Show HN: Continuous-eval – Granular evaluation of GenAI pipelines Hi HN - we are the creators of “continuous-eval”, an open-source tool to test and evaluate generative AI apps. "Continuous-eval" came from our efforts to measure, validate and improve the reliability of a finance AI copilot we were developing for banks. End-to-end evaluation was not enough for us. We wanted to have granular evaluations that help pinpoint the bottlenecks and identify what / how to improve. We’ve since developed more metrics and made the framework more flexible so it can evaluate components like agent tool use, code change, retrieval steps, etc. Let us know what you think of our approach to GenAI App evaluation. https://ift.tt/cFVeQny February 26, 2024 at 12:11AM
Show HN: Continuous-eval – Granular evaluation of GenAI pipelines https://ift.tt/kSEhrdm
Related Articles
Show HN: Exafunction NLP – Run models like GPT-J for 6x cheaper than OpenAI https://ift.tt/39IX4SPShow HN: Exafunction NLP – Run models like GPT-J for 6x cheaper than O… Read More
Show HN: New Oberon+ IDE based on the Mono CLR – lean and fast https://ift.tt/3mjTcgHShow HN: New Oberon+ IDE based on the Mono CLR – lean and fast https:/… Read More
Show HN: Built Codegame for developers to create programming games with Markdown https://ift.tt/3zOGQlvShow HN: Built Codegame for developers to create programming games wit… Read More
Show HN: Automate Job Application Process https://ift.tt/3oiPU05Show HN: Automate Job Application Process https://lazyapply.com Septem… Read More
Show HN: Railway – Configless Software Infrastructure https://ift.tt/3m4CuSyShow HN: Railway – Configless Software Infrastructure https://railway.… Read More
Show HN: Instant Preview for Static Websites https://ift.tt/3zPUNjkShow HN: Instant Preview for Static Websites https://ift.tt/3oktGuv Se… Read More
Show HN: Pepper – Find Low Carbon Food https://ift.tt/3olQA4NShow HN: Pepper – Find Low Carbon Food https://ift.tt/3B0Sqvv October … Read More
Show HN: See HN, Twitter discussions and posts mentioning the page you are on https://ift.tt/3upSKRVShow HN: See HN, Twitter discussions and posts mentioning the page you… Read More
0 Comments: