[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"similar-vibrantlabsai--ragas":3,"tool-vibrantlabsai--ragas":64},[4,17,27,35,43,56],{"id":5,"name":6,"github_repo":7,"description_zh":8,"stars":9,"difficulty_score":10,"last_commit_at":11,"category_tags":12,"status":16},3808,"stable-diffusion-webui","AUTOMATIC1111\u002Fstable-diffusion-webui","stable-diffusion-webui 是一个基于 Gradio 构建的网页版操作界面，旨在让用户能够轻松地在本地运行和使用强大的 Stable Diffusion 图像生成模型。它解决了原始模型依赖命令行、操作门槛高且功能分散的痛点，将复杂的 AI 绘图流程整合进一个直观易用的图形化平台。\n\n无论是希望快速上手的普通创作者、需要精细控制画面细节的设计师，还是想要深入探索模型潜力的开发者与研究人员，都能从中获益。其核心亮点在于极高的功能丰富度：不仅支持文生图、图生图、局部重绘（Inpainting）和外绘（Outpainting）等基础模式，还独创了注意力机制调整、提示词矩阵、负向提示词以及“高清修复”等高级功能。此外，它内置了 GFPGAN 和 CodeFormer 等人脸修复工具，支持多种神经网络放大算法，并允许用户通过插件系统无限扩展能力。即使是显存有限的设备，stable-diffusion-webui 也提供了相应的优化选项，让高质量的 AI 艺术创作变得触手可及。",162132,3,"2026-04-05T11:01:52",[13,14,15],"开发框架","图像","Agent","ready",{"id":18,"name":19,"github_repo":20,"description_zh":21,"stars":22,"difficulty_score":23,"last_commit_at":24,"category_tags":25,"status":16},1381,"everything-claude-code","affaan-m\u002Feverything-claude-code","everything-claude-code 是一套专为 AI 编程助手（如 Claude Code、Codex、Cursor 等）打造的高性能优化系统。它不仅仅是一组配置文件，而是一个经过长期实战打磨的完整框架，旨在解决 AI 代理在实际开发中面临的效率低下、记忆丢失、安全隐患及缺乏持续学习能力等核心痛点。\n\n通过引入技能模块化、直觉增强、记忆持久化机制以及内置的安全扫描功能，everything-claude-code 能显著提升 AI 在复杂任务中的表现，帮助开发者构建更稳定、更智能的生产级 AI 代理。其独特的“研究优先”开发理念和针对 Token 消耗的优化策略，使得模型响应更快、成本更低，同时有效防御潜在的攻击向量。\n\n这套工具特别适合软件开发者、AI 研究人员以及希望深度定制 AI 工作流的技术团队使用。无论您是在构建大型代码库，还是需要 AI 协助进行安全审计与自动化测试，everything-claude-code 都能提供强大的底层支持。作为一个曾荣获 Anthropic 黑客大奖的开源项目，它融合了多语言支持与丰富的实战钩子（hooks），让 AI 真正成长为懂上",138956,2,"2026-04-05T11:33:21",[13,15,26],"语言模型",{"id":28,"name":29,"github_repo":30,"description_zh":31,"stars":32,"difficulty_score":23,"last_commit_at":33,"category_tags":34,"status":16},2271,"ComfyUI","Comfy-Org\u002FComfyUI","ComfyUI 是一款功能强大且高度模块化的视觉 AI 引擎，专为设计和执行复杂的 Stable Diffusion 图像生成流程而打造。它摒弃了传统的代码编写模式，采用直观的节点式流程图界面，让用户通过连接不同的功能模块即可构建个性化的生成管线。\n\n这一设计巧妙解决了高级 AI 绘图工作流配置复杂、灵活性不足的痛点。用户无需具备编程背景，也能自由组合模型、调整参数并实时预览效果，轻松实现从基础文生图到多步骤高清修复等各类复杂任务。ComfyUI 拥有极佳的兼容性，不仅支持 Windows、macOS 和 Linux 全平台，还广泛适配 NVIDIA、AMD、Intel 及苹果 Silicon 等多种硬件架构，并率先支持 SDXL、Flux、SD3 等前沿模型。\n\n无论是希望深入探索算法潜力的研究人员和开发者，还是追求极致创作自由度的设计师与资深 AI 绘画爱好者，ComfyUI 都能提供强大的支持。其独特的模块化架构允许社区不断扩展新功能，使其成为当前最灵活、生态最丰富的开源扩散模型工具之一，帮助用户将创意高效转化为现实。",107662,"2026-04-03T11:11:01",[13,14,15],{"id":36,"name":37,"github_repo":38,"description_zh":39,"stars":40,"difficulty_score":23,"last_commit_at":41,"category_tags":42,"status":16},3704,"NextChat","ChatGPTNextWeb\u002FNextChat","NextChat 是一款轻量且极速的 AI 助手，旨在为用户提供流畅、跨平台的大模型交互体验。它完美解决了用户在多设备间切换时难以保持对话连续性，以及面对众多 AI 模型不知如何统一管理的痛点。无论是日常办公、学习辅助还是创意激发，NextChat 都能让用户随时随地通过网页、iOS、Android、Windows、MacOS 或 Linux 端无缝接入智能服务。\n\n这款工具非常适合普通用户、学生、职场人士以及需要私有化部署的企业团队使用。对于开发者而言，它也提供了便捷的自托管方案，支持一键部署到 Vercel 或 Zeabur 等平台。\n\nNextChat 的核心亮点在于其广泛的模型兼容性，原生支持 Claude、DeepSeek、GPT-4 及 Gemini Pro 等主流大模型，让用户在一个界面即可自由切换不同 AI 能力。此外，它还率先支持 MCP（Model Context Protocol）协议，增强了上下文处理能力。针对企业用户，NextChat 提供专业版解决方案，具备品牌定制、细粒度权限控制、内部知识库整合及安全审计等功能，满足公司对数据隐私和个性化管理的高标准要求。",87618,"2026-04-05T07:20:52",[13,26],{"id":44,"name":45,"github_repo":46,"description_zh":47,"stars":48,"difficulty_score":23,"last_commit_at":49,"category_tags":50,"status":16},2268,"ML-For-Beginners","microsoft\u002FML-For-Beginners","ML-For-Beginners 是由微软推出的一套系统化机器学习入门课程，旨在帮助零基础用户轻松掌握经典机器学习知识。这套课程将学习路径规划为 12 周，包含 26 节精炼课程和 52 道配套测验，内容涵盖从基础概念到实际应用的完整流程，有效解决了初学者面对庞大知识体系时无从下手、缺乏结构化指导的痛点。\n\n无论是希望转型的开发者、需要补充算法背景的研究人员，还是对人工智能充满好奇的普通爱好者，都能从中受益。课程不仅提供了清晰的理论讲解，还强调动手实践，让用户在循序渐进中建立扎实的技能基础。其独特的亮点在于强大的多语言支持，通过自动化机制提供了包括简体中文在内的 50 多种语言版本，极大地降低了全球不同背景用户的学习门槛。此外，项目采用开源协作模式，社区活跃且内容持续更新，确保学习者能获取前沿且准确的技术资讯。如果你正寻找一条清晰、友好且专业的机器学习入门之路，ML-For-Beginners 将是理想的起点。",84991,"2026-04-05T10:45:23",[14,51,52,53,15,54,26,13,55],"数据工具","视频","插件","其他","音频",{"id":57,"name":58,"github_repo":59,"description_zh":60,"stars":61,"difficulty_score":10,"last_commit_at":62,"category_tags":63,"status":16},3128,"ragflow","infiniflow\u002Fragflow","RAGFlow 是一款领先的开源检索增强生成（RAG）引擎，旨在为大语言模型构建更精准、可靠的上下文层。它巧妙地将前沿的 RAG 技术与智能体（Agent）能力相结合，不仅支持从各类文档中高效提取知识，还能让模型基于这些知识进行逻辑推理和任务执行。\n\n在大模型应用中，幻觉问题和知识滞后是常见痛点。RAGFlow 通过深度解析复杂文档结构（如表格、图表及混合排版），显著提升了信息检索的准确度，从而有效减少模型“胡编乱造”的现象，确保回答既有据可依又具备时效性。其内置的智能体机制更进一步，使系统不仅能回答问题，还能自主规划步骤解决复杂问题。\n\n这款工具特别适合开发者、企业技术团队以及 AI 研究人员使用。无论是希望快速搭建私有知识库问答系统，还是致力于探索大模型在垂直领域落地的创新者，都能从中受益。RAGFlow 提供了可视化的工作流编排界面和灵活的 API 接口，既降低了非算法背景用户的上手门槛，也满足了专业开发者对系统深度定制的需求。作为基于 Apache 2.0 协议开源的项目，它正成为连接通用大模型与行业专有知识之间的重要桥梁。",77062,"2026-04-04T04:44:48",[15,14,13,26,54],{"id":65,"github_repo":66,"name":67,"description_en":68,"description_zh":69,"ai_summary_zh":69,"readme_en":70,"readme_zh":71,"quickstart_zh":72,"use_case_zh":73,"hero_image_url":74,"owner_login":75,"owner_name":76,"owner_avatar_url":77,"owner_bio":78,"owner_company":79,"owner_location":79,"owner_email":79,"owner_twitter":79,"owner_website":79,"owner_url":80,"languages":81,"stars":101,"forks":102,"last_commit_at":103,"license":104,"difficulty_score":23,"env_os":105,"env_gpu":105,"env_ram":105,"env_deps":106,"category_tags":111,"github_topics":112,"view_count":116,"oss_zip_url":79,"oss_zip_packed_at":79,"status":16,"created_at":117,"updated_at":118,"faqs":119,"releases":144},2203,"vibrantlabsai\u002Fragas","ragas","Supercharge Your LLM Application Evaluations 🚀","Ragas 是一款专为大语言模型（LLM）应用打造的评估与优化工具箱，旨在帮助开发者建立数据驱动的高效测试流程。在开发基于 LLM 的应用时，传统的人工评估往往耗时费力且主观性强，难以量化效果。Ragas 通过提供客观的评估指标和智能测试数据生成能力，完美解决了这一痛点。\n\n它不仅支持使用基于 LLM 的传统指标进行精准打分，还能自动生成覆盖多种场景的生产级测试数据集，让用户无需手动准备大量测试用例即可快速启动评估。此外，Ragas 能与 LangChain 等主流框架及可观测性工具无缝集成，帮助用户利用生产数据构建持续改进的反馈闭环。其独特的“方面批判”（Aspect Critique）功能，允许用户自定义维度（如摘要准确性），对模型输出进行细粒度的离散值评估。\n\n这款工具非常适合 AI 应用开发者、研究人员以及需要验证 RAG 系统或智能体性能的技术团队。无论是想要基准测试不同模型的表现，还是希望优化提示词策略，Ragas 都能提供科学、自动化的解决方案，让模型迭代更加从容高效。","\u003Ch1 align=\"center\">\n  \u003Cimg style=\"vertical-align:middle\" height=\"200\"\n  src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fvibrantlabsai_ragas_readme_dea4c3187ee1.png\">\n\u003C\u002Fh1>\n\u003Cp align=\"center\">\n  \u003Ci>Supercharge Your LLM Application Evaluations 🚀\u003C\u002Fi>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Freleases\">\n        \u003Cimg alt=\"Latest release\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Frelease\u002Fvibrantlabsai\u002Fragas.svg\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fwww.python.org\u002F\">\n        \u003Cimg alt=\"Made with Python\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FMade%20with-Python-1f425f.svg?color=purple\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fblob\u002Fmaster\u002FLICENSE\">\n        \u003Cimg alt=\"License Apache-2.0\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Flicense\u002Fvibrantlabsai\u002Fragas.svg?color=green\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fpypi.org\u002Fproject\u002Fragas\u002F\">\n        \u003Cimg alt=\"Ragas Downloads per month\" src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fvibrantlabsai_ragas_readme_e61070d2d408.png\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fdiscord.gg\u002F5djav8GGNZ\">\n        \u003Cimg alt=\"Join Ragas community on Discord\" src=\"https:\u002F\u002Fimg.shields.io\u002Fdiscord\u002F1119637219561451644\">\n    \u003C\u002Fa>\n    \u003Ca target=\"_blank\" href=\"https:\u002F\u002Fdeepwiki.com\u002Fvibrantlabsai\u002Fragas\">\n      \u003Cimg \n        src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fvibrantlabsai_ragas_readme_1a081b2fa577.png\" \n        alt=\"Ask DeepWiki.com\" \n        height=\"20\" \n      \u002F>\n    \u003C\u002Fa>\n\u003C\u002Fp>\n\n\u003Ch4 align=\"center\">\n    \u003Cp>\n        \u003Ca href=\"https:\u002F\u002Fdocs.ragas.io\u002F\">Documentation\u003C\u002Fa> |\n        \u003Ca href=\"#fire-quickstart\">Quick start\u003C\u002Fa> |\n        \u003Ca href=\"https:\u002F\u002Fdiscord.gg\u002F5djav8GGNZ\">Join Discord\u003C\u002Fa> |\n        \u003Ca href=\"https:\u002F\u002Fblog.ragas.io\u002F\">Blog\u003C\u002Fa> |\n        \u003Ca href=\"https:\u002F\u002Fnewsletter.ragas.io\u002F\">NewsLetter\u003C\u002Fa> |\n        \u003Ca href=\"https:\u002F\u002Fwww.ragas.io\u002Fcareers\">Careers\u003C\u002Fa>\n    \u003Cp>\n\u003C\u002Fh4>\n\nObjective metrics, intelligent test generation, and data-driven insights for LLM apps\n\nRagas is your ultimate toolkit for evaluating and optimizing Large Language Model (LLM) applications. Say goodbye to time-consuming, subjective assessments and hello to data-driven, efficient evaluation workflows.\nDon't have a test dataset ready? We also do production-aligned test set generation.\n\n## Key Features\n\n- 🎯 Objective Metrics: Evaluate your LLM applications with precision using both LLM-based and traditional metrics.\n- 🧪 Test Data Generation: Automatically create comprehensive test datasets covering a wide range of scenarios.\n- 🔗 Seamless Integrations: Works flawlessly with popular LLM frameworks like LangChain and major observability tools.\n- 📊 Build feedback loops: Leverage production data to continually improve your LLM applications.\n\n## :shield: Installation\n\nPypi:\n\n```bash\npip install ragas\n```\n\nAlternatively, from source:\n\n```bash\npip install git+https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\n```\n\n## :fire: Quickstart\n\n### Clone a Complete Example Project\n\nThe fastest way to get started is to use the `ragas quickstart` command:\n\n```bash\n# List available templates\nragas quickstart\n\n# Create a RAG evaluation project\nragas quickstart rag_eval\n\n# Specify where you want to create it.\nragas quickstart rag_eval -o .\u002Fmy-project\n```\n\nAvailable templates:\n- `rag_eval` - Evaluate RAG systems\n\nComing Soon:\n- `agent_evals` - Evaluate AI agents\n- `benchmark_llm` - Benchmark and compare LLMs\n- `prompt_evals` - Evaluate prompt variations\n- `workflow_eval` - Evaluate complex workflows\n\n### Evaluate your LLM App\n\n`ragas` comes with pre-built metrics for common evaluation tasks. For example, Aspect Critique evaluates any aspect of your output using `DiscreteMetric`:\n\n```python\nimport asyncio\nfrom openai import AsyncOpenAI\nfrom ragas.metrics import DiscreteMetric\nfrom ragas.llms import llm_factory\n\n# Setup your LLM\nclient = AsyncOpenAI()\nllm = llm_factory(\"gpt-4o\", client=client)\n\n# Create a custom aspect evaluator\nmetric = DiscreteMetric(\n    name=\"summary_accuracy\",\n    allowed_values=[\"accurate\", \"inaccurate\"],\n    prompt=\"\"\"Evaluate if the summary is accurate and captures key information.\n\nResponse: {response}\n\nAnswer with only 'accurate' or 'inaccurate'.\"\"\"\n)\n\n# Score your application's output\nasync def main():\n    score = await metric.ascore(\n        llm=llm,\n        response=\"The summary of the text is...\"\n    )\n    print(f\"Score: {score.value}\")  # 'accurate' or 'inaccurate'\n    print(f\"Reason: {score.reason}\")\n\n\nif __name__ == \"__main__\":\n    asyncio.run(main())\n```\n\n> **Note**: Make sure your `OPENAI_API_KEY` environment variable is set.\n\nFind the complete [Quickstart Guide](https:\u002F\u002Fdocs.ragas.io\u002Fen\u002Flatest\u002Fgetstarted\u002Fquickstart)\n\n## Want help in improving your AI application using evals?\n\nIn the past 2 years, we have seen and helped improve many AI applications using evals. If you want help with improving and scaling up your AI application using evals.\n\n🔗 Book a [slot](https:\u002F\u002Fcal.com\u002Fteam\u002Fvibrantlabs\u002Fapp) or drop us a line: [founders@vibrantlabs.com](mailto:founders@vibrantlabs.com).\n\n## 🫂 Community\n\nIf you want to get more involved with Ragas, check out our [discord server](https:\u002F\u002Fdiscord.gg\u002F5qGUJ6mh7C). It's a fun community where we geek out about LLM, Retrieval, Production issues, and more.\n\n## Contributors\n\n```yml\n+----------------------------------------------------------------------------+\n|     +----------------------------------------------------------------+     |\n|     | Developers: Those who built with `ragas`.                      |     |\n|     | (You have `import ragas` somewhere in your project)            |     |\n|     |     +----------------------------------------------------+     |     |\n|     |     | Contributors: Those who make `ragas` better.       |     |     |\n|     |     | (You make PR to this repo)                         |     |     |\n|     |     +----------------------------------------------------+     |     |\n|     +----------------------------------------------------------------+     |\n+----------------------------------------------------------------------------+\n```\n\nWe welcome contributions from the community! Whether it's bug fixes, feature additions, or documentation improvements, your input is valuable.\n\n1. Fork the repository\n2. Create your feature branch (git checkout -b feature\u002FAmazingFeature)\n3. Commit your changes (git commit -m 'Add some AmazingFeature')\n4. Push to the branch (git push origin feature\u002FAmazingFeature)\n5. Open a Pull Request\n\n## 🔍 Open Analytics\n\nAt Ragas, we believe in transparency. We collect minimal, anonymized usage data to improve our product and guide our development efforts.\n\n✅ No personal or company-identifying information\n\n✅ Open-source data collection [code](.\u002Fsrc\u002Fragas\u002F_analytics.py)\n\n✅ Publicly available aggregated [data](https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fissues\u002F49)\n\nTo opt-out, set the `RAGAS_DO_NOT_TRACK` environment variable to `true`.\n\n### Cite Us\n\n```\n@misc{ragas2024,\n  author       = {VibrantLabs},\n  title        = {Ragas: Supercharge Your LLM Application Evaluations},\n  year         = {2024},\n  howpublished = {\\url{https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas}},\n}\n```\n","\u003Ch1 align=\"center\">\n  \u003Cimg style=\"vertical-align:middle\" height=\"200\"\n  src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fvibrantlabsai_ragas_readme_dea4c3187ee1.png\">\n\u003C\u002Fh1>\n\u003Cp align=\"center\">\n  \u003Ci>为您的大模型应用评估注入强大动力 🚀\u003C\u002Fi>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Freleases\">\n        \u003Cimg alt=\"最新版本\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Frelease\u002Fvibrantlabsai\u002Fragas.svg\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fwww.python.org\u002F\">\n        \u003Cimg alt=\"由 Python 构建\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FMade%20with-Python-1f425f.svg?color=purple\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fblob\u002Fmaster\u002FLICENSE\">\n        \u003Cimg alt=\"Apache-2.0 许可证\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Flicense\u002Fvibrantlabsai\u002Fragas.svg?color=green\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fpypi.org\u002Fproject\u002Fragas\u002F\">\n        \u003Cimg alt=\"Ragas 每月下载量\" src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fvibrantlabsai_ragas_readme_e61070d2d408.png\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fdiscord.gg\u002F5djav8GGNZ\">\n        \u003Cimg alt=\"加入 Ragas 社区 Discord\" src=\"https:\u002F\u002Fimg.shields.io\u002Fdiscord\u002F1119637219561451644\">\n    \u003C\u002Fa>\n    \u003Ca target=\"_blank\" href=\"https:\u002F\u002Fdeepwiki.com\u002Fvibrantlabsai\u002Fragas\">\n      \u003Cimg \n        src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fvibrantlabsai_ragas_readme_1a081b2fa577.png\" \n        alt=\"向 DeepWiki.com 提问\" \n        height=\"20\" \n      \u002F>\n    \u003C\u002Fa>\n\u003C\u002Fp>\n\n\u003Ch4 align=\"center\">\n    \u003Cp>\n        \u003Ca href=\"https:\u002F\u002Fdocs.ragas.io\u002F\">文档\u003C\u002Fa> |\n        \u003Ca href=\"#fire-quickstart\">快速入门\u003C\u002Fa> |\n        \u003Ca href=\"https:\u002F\u002Fdiscord.gg\u002F5djav8GGNZ\">加入 Discord\u003C\u002Fa> |\n        \u003Ca href=\"https:\u002F\u002Fblog.ragas.io\u002F\">博客\u003C\u002Fa> |\n        \u003Ca href=\"https:\u002F\u002Fnewsletter.ragas.io\u002F\">新闻通讯\u003C\u002Fa> |\n        \u003Ca href=\"https:\u002F\u002Fwww.ragas.io\u002Fcareers\">职业机会\u003C\u002Fa>\n    \u003Cp>\n\u003C\u002Fh4>\n\n针对 LLM 应用的客观指标、智能测试生成以及数据驱动的洞察\n\nRagas 是您评估和优化大型语言模型（LLM）应用的终极工具包。告别耗时且主观的评估，迎接数据驱动、高效的评估流程。\n还没有准备好测试数据集？我们还能为您生成与生产环境一致的测试集。\n\n## 核心功能\n\n- 🎯 客观指标：使用基于 LLM 的指标和传统指标，精准评估您的 LLM 应用。\n- 🧪 测试数据生成：自动创建覆盖广泛场景的全面测试数据集。\n- 🔗 无缝集成：与 LangChain 等流行 LLM 框架及主流可观测性工具完美兼容。\n- 📊 构建反馈循环：利用生产数据持续改进您的 LLM 应用。\n\n## :shield: 安装\n\n从 PyPI：\n\n```bash\npip install ragas\n```\n\n或者从源代码安装：\n\n```bash\npip install git+https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\n```\n\n## :fire: 快速入门\n\n### 克隆完整示例项目\n\n最快上手的方式是使用 `ragas quickstart` 命令：\n\n```bash\n# 列出可用模板\nragas quickstart\n\n# 创建 RAG 评估项目\nragas quickstart rag_eval\n\n# 指定创建位置\nragas quickstart rag_eval -o .\u002Fmy-project\n```\n\n可用模板：\n- `rag_eval` - 评估 RAG 系统\n\n即将推出：\n- `agent_evals` - 评估 AI 代理\n- `benchmark_llm` - 对 LLM 进行基准测试和比较\n- `prompt_evals` - 评估提示词变体\n- `workflow_eval` - 评估复杂工作流\n\n### 评估您的 LLM 应用\n\n`ragas` 内置了常见评估任务的预设指标。例如，Aspect Critique 使用 `DiscreteMetric` 评估输出的任何方面：\n\n```python\nimport asyncio\nfrom openai import AsyncOpenAI\nfrom ragas.metrics import DiscreteMetric\nfrom ragas.llms import llm_factory\n\n# 设置您的 LLM\nclient = AsyncOpenAI()\nllm = llm_factory(\"gpt-4o\", client=client)\n\n# 创建自定义方面评估器\nmetric = DiscreteMetric(\n    name=\"summary_accuracy\",\n    allowed_values=[\"accurate\", \"inaccurate\"],\n    prompt=\"\"\"评估摘要是否准确并抓住了关键信息。\n\n回答：{response}\n\n请仅回答‘accurate’或‘inaccurate’。”\n)\n\n# 评分您的应用输出\nasync def main():\n    score = await metric.ascore(\n        llm=llm,\n        response=\"文本的摘要是……\"\n    )\n    print(f\"评分：{score.value}\")  # 'accurate' 或 'inaccurate'\n    print(f\"理由：{score.reason}\")\n\n\nif __name__ == \"__main__\":\n    asyncio.run(main())\n```\n\n> **注意**：请确保已设置 `OPENAI_API_KEY` 环境变量。\n\n完整的[快速入门指南](https:\u002F\u002Fdocs.ragas.io\u002Fen\u002Flatest\u002Fgetstarted\u002Fquickstart)\n\n## 需要帮助通过评估改进您的 AI 应用吗？\n\n过去两年里，我们通过评估见证并帮助改进了许多 AI 应用。如果您希望借助评估来提升和扩展您的 AI 事业，\n\n🔗 预约一个[时段](https:\u002F\u002Fcal.com\u002Fteam\u002Fvibrantlabs\u002Fapp)或发送邮件至：[founders@vibrantlabs.com](mailto:founders@vibrantlabs.com)。\n\n## 🫂 社区\n\n如果您想更深入地参与 Ragas 社区，请访问我们的 [Discord 服务器](https:\u002F\u002Fdiscord.gg\u002F5qGUJ6mh7C)。这里是一个充满乐趣的社区，大家热衷于讨论 LLM、检索技术、生产问题等话题。\n\n## 贡献者\n\n```yml\n+----------------------------------------------------------------------------+\n|     +----------------------------------------------------------------+     |\n|     | 开发者：那些使用 `ragas` 构建项目的开发者。                      |     |\n|     | （您的项目中某处有 `import ragas`）                            |     |\n|     |     +----------------------------------------------------+     |     |\n|     |     | 贡献者：那些让 `ragas` 变得更好的人。       |     |     |\n|     |     | （您为此仓库提交过 PR）                         |     |     |\n|     |     +----------------------------------------------------+     |     |\n|     +----------------------------------------------------------------+     |\n+----------------------------------------------------------------------------+\n```\n\n我们欢迎社区的贡献！无论是修复 bug、添加新功能，还是改进文档，您的意见都至关重要。\n\n1. 分支仓库\n2. 创建特性分支（git checkout -b feature\u002FAmazingFeature）\n3. 提交更改（git commit -m '添加 AmazingFeature'）\n4. 推送到分支（git push origin feature\u002FAmazingFeature）\n5. 打开拉取请求\n\n## 🔍 开放式分析\n\n在 Ragas，我们坚信透明度的重要性。我们收集最少且匿名化的使用数据，以改进产品并指导开发工作。\n\n✅ 不包含个人或公司识别信息\n\n✅ 开源的数据收集[代码](.\u002Fsrc\u002Fragas\u002F_analytics.py)\n\n✅ 公开的聚合[数据](https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fissues\u002F49)\n\n如需退出，请将 `RAGAS_DO_NOT_TRACK` 环境变量设置为 `true`。\n\n### 引用我们\n\n```\n@misc{ragas2024,\n  author       = {VibrantLabs},\n  title        = {Ragas：为您的 LLM 应用评估注入强大动力},\n  year         = {2024},\n  howpublished = {\\url{https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas}},\n}\n```","# Ragas 快速上手指南\n\nRagas 是专为大型语言模型（LLM）应用打造的评估与优化工具箱。它提供客观的评估指标、智能测试数据生成以及数据驱动的洞察，帮助你高效地评估 RAG 系统和 LLM 应用。\n\n## 环境准备\n\n在开始之前，请确保你的开发环境满足以下要求：\n\n- **操作系统**：Linux, macOS, 或 Windows\n- **Python 版本**：Python 3.9 或更高版本\n- **依赖项**：\n  - `pip` 包管理工具\n  - 有效的 LLM API 密钥（如 OpenAI API Key），用于运行基于 LLM 的评估指标\n- **环境变量**：使用前请确保已设置 `OPENAI_API_KEY`（或其他对应模型的密钥）。\n\n```bash\nexport OPENAI_API_KEY=\"your-api-key-here\"\n```\n\n## 安装步骤\n\n你可以通过 PyPI 直接安装稳定版，或者从源码安装最新版本。\n\n### 方式一：通过 PyPI 安装（推荐）\n\n```bash\npip install ragas\n```\n\n> **提示**：国内开发者若下载缓慢，可使用清华或阿里镜像源加速：\n> ```bash\n> pip install ragas -i https:\u002F\u002Fpypi.tuna.tsinghua.edu.cn\u002Fsimple\n> ```\n\n### 方式二：从源码安装\n\n```bash\npip install git+https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\n```\n\n## 基本使用\n\nRagas 提供了多种使用方式，最快捷的是通过命令行初始化项目，或直接使用 Python SDK 进行自定义评估。\n\n### 1. 快速初始化项目\n\n使用 `ragas quickstart` 命令可以快速克隆一个完整的示例项目，这是上手最快的方式。\n\n```bash\n# 查看可用模板\nragas quickstart\n\n# 创建一个 RAG 评估项目 (默认当前目录)\nragas quickstart rag_eval\n\n# 指定项目创建路径\nragas quickstart rag_eval -o .\u002Fmy-project\n```\n\n目前支持的模板包括 `rag_eval`（用于评估 RAG 系统）。未来将支持 Agent 评估、LLM 基准测试等更多模板。\n\n### 2. 使用 Python SDK 进行评估\n\n你可以使用预建的指标或创建自定义指标来评估 LLM 的输出。以下示例展示了如何使用 `DiscreteMetric` 对摘要的准确性进行二元评估（准确\u002F不准确）。\n\n```python\nimport asyncio\nfrom openai import AsyncOpenAI\nfrom ragas.metrics import DiscreteMetric\nfrom ragas.llms import llm_factory\n\n# 配置你的 LLM 客户端\nclient = AsyncOpenAI()\nllm = llm_factory(\"gpt-4o\", client=client)\n\n# 创建自定义维度评估器\nmetric = DiscreteMetric(\n    name=\"summary_accuracy\",\n    allowed_values=[\"accurate\", \"inaccurate\"],\n    prompt=\"\"\"Evaluate if the summary is accurate and captures key information.\n\nResponse: {response}\n\nAnswer with only 'accurate' or 'inaccurate'.\"\"\"\n)\n\n# 异步执行评分\nasync def main():\n    score = await metric.ascore(\n        llm=llm,\n        response=\"The summary of the text is...\"\n    )\n    print(f\"Score: {score.value}\")  # 输出: 'accurate' 或 'inaccurate'\n    print(f\"Reason: {score.reason}\")\n\n\nif __name__ == \"__main__\":\n    asyncio.run(main())\n```\n\n运行上述代码后，你将获得评估结果及其理由，从而量化你的 LLM 应用表现。","某电商公司的算法团队正在开发基于 RAG 架构的智能客服系统，急需验证模型在回答商品退换货政策时的准确性与可靠性。\n\n### 没有 ragas 时\n- **评估主观且低效**：团队依赖人工抽检回答质量，不仅耗时费力，还因缺乏统一标准导致不同评估者结论冲突。\n- **测试数据覆盖不足**：手动构造的测试用例场景单一，难以模拟用户千变万化的提问方式，遗漏了长尾问题的风险。\n- **优化方向模糊**：发现回答错误时，无法量化是检索环节出了问题还是生成环节有误，只能凭经验盲目调整 Prompt。\n- **迭代周期漫长**：每次模型更新后需重新组织人力回归测试，严重拖慢了从开发到上线的发布节奏。\n\n### 使用 ragas 后\n- **指标客观自动化**：利用 ragas 内置的“忠实度”和“答案相关性”等指标，自动对成千上万条问答进行打分，结果可量化且可复现。\n- **智能生成测试集**：通过 ragas 的合成数据功能，基于现有文档自动生成涵盖各种边缘场景的测试题，大幅提升了测试覆盖率。\n- **精准定位瓶颈**：借助多维度的诊断报表，团队能清晰看到是检索到的上下文不相关，还是模型未能正确利用上下文，从而针对性优化。\n- **构建持续反馈闭环**：将生产环境的真实用户日志接入 ragas 分析，实时监控模型表现，实现了数据驱动的持续迭代。\n\nragas 将原本模糊、滞后的定性评估转变为精确、实时的数据驱动工作流，让团队能以十倍速交付高质量的 LLM 应用。","https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fvibrantlabsai_ragas_dea4c318.png","vibrantlabsai","Vibrant Labs","https:\u002F\u002Foss.gittoolsai.com\u002Favatars\u002Fvibrantlabsai_e82e8e69.png","",null,"https:\u002F\u002Fgithub.com\u002Fvibrantlabsai",[82,86,90,94,98],{"name":83,"color":84,"percentage":85},"Python","#3572A5",83,{"name":87,"color":88,"percentage":89},"Jupyter Notebook","#DA5B0B",16.7,{"name":91,"color":92,"percentage":93},"Makefile","#427819",0.3,{"name":95,"color":96,"percentage":97},"Dockerfile","#384d54",0,{"name":99,"color":100,"percentage":97},"Shell","#89e051",13228,1333,"2026-04-05T11:33:54","Apache-2.0","未说明",{"notes":107,"python":105,"dependencies":108},"该工具主要通过 pip 安装。运行评估时需要配置 OPENAI_API_KEY 环境变量以调用 LLM（如 GPT-4o）。README 中未明确提及具体的操作系统、GPU、内存或 Python 版本限制，通常意味着它依赖于标准 Python 环境及所调用的外部 LLM 服务，而非本地重型模型推理。",[109,110],"openai","asyncio",[13,26,54],[113,114,115],"llm","llmops","evaluation",4,"2026-03-27T02:49:30.150509","2026-04-06T07:13:43.440080",[120,125,129,134,139],{"id":121,"question_zh":122,"answer_zh":123,"source_url":124},10152,"使用 Ragas 进行评估时是否必须设置 API Key？有免费的使用方式吗？","是的，根据当前的使用情况，使用 LLM 进行评估时必须设置 API Key。错误信息 \"The api_key client option must be set...\" 表明你需要通过两种方式之一设置密钥：1. 在初始化客户端时直接传递 api_key 参数；2. 设置环境变量 OPENAI_API_KEY。目前 Ragas 本身不提供免费的 API Key，你需要自行申请（如 OpenAI 的免费额度或其他提供免费服务的 LLM 提供商）。","https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fissues\u002F1358",{"id":126,"question_zh":127,"answer_zh":128,"source_url":124},10153,"遇到 'LLMResult' 验证错误，提示输入应为 Generation 或 ChatGeneration 实例而不是字符串，如何解决？","该错误是因为自定义 LLM 的 generate_text 方法返回了原始字符串（如 'Answer not found'），而 Ragas 期望返回包装好的对象。解决方法是修改 generate_text 方法，确保返回值被正确包装在 `Generation` 或 `ChatGeneration` 对象中。示例代码：\n```python\nfrom langchain_core.outputs import Generation, LLMResult\n# 在返回前包装结果\ngeneration = Generation(text=\"your_response_string\")\nreturn LLMResult(generations=[[generation]])\n```",{"id":130,"question_zh":131,"answer_zh":132,"source_url":133},10154,"自动语言适配（Automatic Language Adaptation）功能在 v0.1 版本中报错或不工作怎么办？","该问题已在 Ragas v0.2 版本中修复。如果你使用的是 v0.1.x 版本并遇到 JSON 格式错误或方法调用缺失（如 generate_text 或 generate_prompt），请升级到 v0.2 或更高版本。升级后请参考新的文档进行语言适配：https:\u002F\u002Fdocs.ragas.io\u002Fen\u002Fstable\u002Fhowtos\u002Fcustomizations\u002Fmetrics\u002F_metrics_language_adaptation\u002F。如果是从 v0.1 迁移，请查阅迁移指南：https:\u002F\u002Fdocs.ragas.io\u002Fen\u002Fstable\u002Fhowtos\u002Fmigrations\u002Fmigrate_from_v01_to_v02","https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fissues\u002F1101",{"id":135,"question_zh":136,"answer_zh":137,"source_url":138},10155,"评估时报错 'Dataset feature contexts should be of type Sequence[string]'，如何修正数据格式？","该错误表明数据集中的 `contexts` 列格式不正确。Ragas 要求 `contexts` 必须是“字符串列表”（List of Strings），而不是单个字符串或其他类型。即使你的上下文只有一段，也需要将其放入列表中。修正方法：\n1. 如果使用 Pandas DataFrame，可以使用：`df[\"contexts\"] = df[\"chain_context\"].apply(lambda x: [x])`\n2. 确保传入 evaluate 函数的数据集中，每个样本的 contexts 字段都是类似 `[\"context text here\"]` 的格式。","https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fissues\u002F286",{"id":140,"question_zh":141,"answer_zh":142,"source_url":143},10156,"导入 ragas.metrics 时提示 'Faithfulness is not callable' 或无法调用 Faithfulness，如何解决？","这是一个语法或导入方式的问题。在较新版本中，请尝试以下两种方法之一：\n1. 直接导入类并实例化：`from ragas.metrics import Faithfulness`，然后使用 `faithfulness_metric = Faithfulness()`。\n2. 检查并使用新的 `llm_factory` 配置方式。确保不要直接调用未实例化的类或错误的模块路径。","https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fissues\u002F1297",[145,150,155,160,165,170,175,180,185,190,194,199,204,209,213,218,222,227,232,237],{"id":146,"version":147,"summary_zh":148,"released_at":149},107408,"v0.4.3","## What's Changed\n* feat: add `DSPyOptimizer` with `MIPROv2` for advanced prompt optimization by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2537\n* feat(docs): add llms.txt generation for LLM-friendly documentation by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2539\n* feat: dspy caching by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2542\n* feat: add system prompt support for `InstructorLLM` and `LiteLLMStructuredLLM` by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2543\n* feat(docs): add copy-to-llm button for easy AI tool integration by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2541\n* fix: use PAT token for docs-check CI as docs-apply CI by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2546\n* feat: add remaining quickstart templates by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2547\n* fix: enable FactualCorrectness language adaptation by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2555\n* fix: resolve `DiskCacheBackend` pickling issue with `InstructorLLM` by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2556\n* fix: lazy init DEFAULT_TOKENIZER to avoid network calls at import time. by @cgaswin in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2545\n* fix: comment on failed task by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2557\n* docs: fix DiscreteMetric llm examples to match API by @cgaswin in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2558\n* fix: add repository parameter to checkout action for fork PR support by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2559\n\n\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fcompare\u002Fv0.4.2...v0.4.3","2026-01-13T17:47:29",{"id":151,"version":152,"summary_zh":153,"released_at":154},107409,"v0.4.2","## What's Changed\n* feat: migrate SQLSemanticEquivalence to collections API by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2496\n* feat: migrate DataCompyScore to collections API by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2499\n* fix: migrate CHRF Score to new metrics collections by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2500\n* Feat\u002Fimprove rag quickstart by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2501\n* fix: handle classification array length mismatch in TopicAdherence metric by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2503\n* feat: migrate quoted spans metric to collections api by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2504\n* fix: handle instructor modes for json and tools by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2505\n* docs: remove obsolete 'write your own metric' guides by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2488\n* fix: BasePrompt.adapt() structured output + language adaptation guide by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2487\n* Add AG-UI Protocol Integration for Agent Evaluation by @contextablemark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2395\n* docs: fix modifying-prompts-metrics guide with correct API by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2486\n* fix: handle nested dicts\u002Flists in ToolCallF1 args to prevent unhashable type error by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2507\n* feat: add Claude docs auto-update workflow by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2508\n* fix: allow unannotated parameters to accept any type by @dhyaneesh in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2513\n* feat: migrate MultiModalFaithfulness to collections API by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2515\n* feat: migrate MultiModalRelevance to collections API by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2518\n* feat: add support for new google-genai SDK with backwards compatibility for deprecated google-generativeai by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2517\n* fix: use instructor Mode.JSON for litellm and generic providers to fix Dict type validation errors by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2514\n* [AG-UI] Replacing \"OpenAI\" with \"AsyncOpenAI\". by @contextablemark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2520\n* Feat: document PDF export workflow by @cgaswin in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2522\n* refactor: deprecate legacy metrics by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2519\n* fix: claude workflows to use github token by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2527\n* feat: add HuggingFace tokenizer support in knowledge graph operations by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2524\n* feat: add generate_with_chunks for pre-chunked documents by @MinseongS in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2526\n* feat(docs): add offline mermaid support to PDF export by @cgaswin in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2530\n* fix: claude workflows to use pat token instead of github token to support forked PRs by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2531\n* docs: change query execution to use asyncio.run by @yangzq50 in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2532\n* fix: increase max-turns and update prompt by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2534\n* chore: remove survey link by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2535\n* feat: add caching support for metrics collections by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2533\n* feat: add caching support for embeddings by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2536\n\n## New Contributors\n* @contextablemark made their first contribution in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2395\n* @cgaswin made their first contribution in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2522\n* @MinseongS made their first contribution in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2526\n* @yangzq50 made their first contribution in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2532\n\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fcompare\u002Fv0.4.1...v0.4.2","2025-12-23T17:13:41",{"id":156,"version":157,"summary_zh":158,"released_at":159},107410,"v0.4.1","## What's Changed\n* feat: add save\u002Fload methods to BasePrompt by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2465\n* docs: update run_config guide to use collections API by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2468\n* fix: add anthropic and gemini clients for custom clients by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2472\n* feat: migrate `ToolCallAccuracy` to collections API by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2476\n* chore: add survey link to readme and docs banner by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2478\n* Fix: apply_transforms(kg, transforms, run_config=run_config or RunCon… by @narabi in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2480\n* feat: migrate ToolCallF1 to collections API with set-based F1 scoring by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2483\n* New input parameter for the TestsetGenerator Class : LLM_CONTEXT  by @narabi in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2474\n* feat: migrate TopicAdherence and AgentGoalAccuracy to collections API by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2485\n* docs: organize integrations nav into collapsible groups by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2492\n* aembed_text() replace embed_text() in AnswerRelevancy by @anexplore in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2495\n* docs: remove broken reference to train_your_own_metric.md by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2489\n* feat: migrate Rubrics Metrics to collections API by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2494\n\n## New Contributors\n* @narabi made their first contribution in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2480\n* @anexplore made their first contribution in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2495\n\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fcompare\u002Fv0.4.0...v0.4.1","2025-12-10T16:28:51",{"id":161,"version":162,"summary_zh":163,"released_at":164},107411,"v0.4.0","## What's Changed\n* docs: complete collections API documentation for remaining metrics by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2420\n* feat: support GPT-5 and o-series models with automatic `temperature` and `top_p` constraint handling by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2418\n* update: add llm options as tabs to quickstart by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2421\n* feat: migrate to instructor.from_provider for universal provider support by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2424\n* docs: fix typos in some files by @Edge-Seven in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2429\n* feat: implement prompt class for context precision by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2433\n* fix: docs for quickstart by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2434\n* Fix 2432: Update import statements for langchain modules by @rvernica in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2436\n* Fix 2440 Adjust LLM parameters in evals.py by @rvernica in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2441\n* feat: migrate context recall, answer relevancy, and context entity recall metrics to modular BasePrompt architecture by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2435\n* feat: migrate 6 metrics (ContextRelevance, Response Groundedness, AnswerAccuracy, Faithfulness, AnswerCorrectness, SummaryScore) to BasePrompt by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2443\n* feat: migrate final metrics (FactualCorrectness, NoiseSensitivity) to modular BasePrompt architecture and update docs by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2444\n* chore: add COC by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2437\n* docs: clarify MLflow is required, not optional in RAG evaluation guide by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2447\n* chore: cleanup old patterns and update links by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2449\n* chore: rebranding efforts by @jjmachan in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2445\n* feat: dual adapter support (Instructor + LiteLLM) by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2446\n* fix: resolve InstructorLLM detection bug and add EvaluationDataset backend support for experiments by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2451\n* fix: retrieved_contexts string filtering in LangChain integration by @dhyaneesh in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2452\n* fix: correct MultiTurnSample user_input validation logic by @harshil-sanghvi in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2426\n* fix: automatic embedding provider matching for LLMs by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2454\n* fix: make GitPython an optional dependency by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2453\n* docs: Update customizations how-to guides to use collections API and LLM factory by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2425\n* fix: detect async clients in closures for instructor-wrapped litellm routers by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2458\n* fix: quickstart by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2463\n* chore: update calendar email by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2462\n* fix: make GoogleEmbeddings handle GenerativeModel clients by auto-extracting genai module by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2466\n* docs: add migration guide for v0.4 by @anistark in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2461\n\n## New Contributors\n* @Edge-Seven made their first contribution in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2429\n* @rvernica made their first contribution in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2436\n* @dhyaneesh made their first contribution in https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fpull\u002F2452\n\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fcompare\u002Fv0.3.9...v0.4.0","2025-12-03T16:22:29",{"id":166,"version":167,"summary_zh":168,"released_at":169},107412,"v0.3.9","## What's Changed\n* fix(docs): add missing line break so the step title and description a… by @nkch1k in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2391\n* Migrate SummaryScore by @rhlbhatnagar in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2376\n* feat: add metadata fields for synthetic data traceability by @dev-jonathan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2389\n* Migrate noise sensitivity by @rhlbhatnagar in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2379\n* docs: quickstart guide with interactive LLM and project structure by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2380\n* Migrate Faithfullness by @rhlbhatnagar in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2384\n* fix: docs for discrete, numeric and ranking using instructor by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2397\n* Migrate Answer Accuracy + Context Relevance by @rhlbhatnagar in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2390\n* refactor: remove aspect critic and simple criteria metrics with discrete metric examples by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2399\n* Migrate Context Pricision with + without ref by @rhlbhatnagar in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2398\n* docs: fix recall formula label in SQL metrics by @tysoncung in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2405\n* chore: remove deprecated `ground_truths` by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2402\n* docs: Add documentation for metrics.collections API by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2407\n* Migrate factual correctness by @rhlbhatnagar in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2401\n* refactor: remove redundant AnswerSimilarity from collections API in favor of SemanticSimilarity by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2410\n* docs: Update documentation structure to reflect experiments-first paradigm by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2394\n* Response Groundedness by @rhlbhatnagar in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2403\n* fix: office hours link update by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2415\n* Refactor\u002Fremoving deprecated by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2412\n* fix: handle `max_completeion_tokens` error for newer openai models by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2413\n* refactor: make embeddings optional in AnswerCorrectness when using pure factuality mode by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2414\n* Feat\u002Fmigrate context recall by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2372\n* chore: update quickstart llm config by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2417\n\n## New Contributors\n* @nkch1k made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2391\n* @dev-jonathan made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2389\n* @tysoncung made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2405\n\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fcompare\u002Fv0.3.8...v0.3.9","2025-11-11T17:24:46",{"id":171,"version":172,"summary_zh":173,"released_at":174},107413,"v0.3.8","## What's Changed\n* feat: semantic similarity migrated to collections by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2361\n* feat: Add reusable testing infrastructure for metrics migration by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2370\n* add: console scripts for ragas_examples by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2367\n* feat: add quickstart cmd with templates to run by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2374\n* fix: detect uvloop and skip nest_asyncio to prevent patching errors by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2369\n* Remove import not used by @ChenyangLi4288 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2364\n* Migrate answer_correctness by @rhlbhatnagar in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2365\n* Migrate context_entity_recall by @rhlbhatnagar in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2366\n* feat: aspect critic metric for coherence, harmfulness, maliciousness, correctness by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2375\n* Fixed: NameError during evalutation of llamaindex query engine by @Prigoistic in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2331\n* Remove error suppressor in async_utils.py and engine.py by @ChenyangLi4288 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2362\n* docs: clarify Context Relevance implementation differs from paper design by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2378\n* fix: add missing metrics (ToolCallF1, ChrfScore) to sidebar and document deprecated ContextUtilization by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2381\n* refactor: `instructor_llm_factory` merge with `llm_factory` by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2382\n* fix: handle tuple-formatted entities in SingleHopSpecificQuerySynthesizer by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2377\n* feat: simple criteria migrated to collections by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2386\n* chore: remove deprecation warnings for LangchainLLMWrapper, LlamaIndexLLMWrapper, and embedding wrappers by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2387\n\n## New Contributors\n* @ChenyangLi4288 made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2364\n\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fcompare\u002Fv0.3.7...v0.3.8","2025-10-28T19:09:17",{"id":176,"version":177,"summary_zh":178,"released_at":179},107414,"v0.3.7","## What's Changed\n* refactor: improve metrics code quality by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2337\n* chore: remove old analtyics by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2338\n* Fix\u002Fquery distribution robustness by @yatoyun in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2340\n* Simplify earlier how to guides in docs by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2319\n* docs: reorganize prompt evaluation guides in navigation by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2346\n* Metrics migration, migrate rouge + answer relevance by @rhlbhatnagar in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2335\n* fix: streamline theme extraction from overlaps in MultiHopSpecificQue… by @kenzoyan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2347\n* Test\u002Fmetric new compare by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2349\n* feat: bleu score migrated to collections by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2352\n* fix: Add List[List[str]] formats for overlapped items in theme extration (Continuation in #2347) by @kenzoyan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2355\n* feat: string metrics migrated to collections by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2356\n* feat: answer similarity migrated to collections by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2358\n* fix: add missing props token_usage_parser for test generation methods #2359 by @bhkj9999 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2360\n* feat: add bypass_n option to LangchainLLMWrapper for n-completion control by @SimFG in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2354\n* docs: Add how-to guide for aligning LLM-as-Judge by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2348\n\n## New Contributors\n* @yatoyun made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2340\n* @kenzoyan made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2347\n* @bhkj9999 made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2360\n* @SimFG made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2354\n\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fcompare\u002Fv0.3.6...v0.3.7","2025-10-14T16:21:37",{"id":181,"version":182,"summary_zh":183,"released_at":184},107415,"v0.3.6","## What's Changed\n* Feature\u002Fchrf score by @kauabh in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2221\n* Fix\u002Fasyncio by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2294\n* Fix: update simple RAG init to use embed_text(s) (docs) by @s3pi in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2292\n* Update _bleu_score.py by @kauabh in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2297\n* Refactor\u002Fupdate gemini to genai sdk by @sahusiddharth in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2240\n* Feature\u002Fmetrics input flexibility by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2298\n* Ensure old_temperature is set correctly. Fixes #1937 and #2110 by @claudepi in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2295\n* Enhance EmbeddingExtractor to support both async and sync methods for… by @telesoho in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2286\n* Tokens counting by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2299\n* Fix\u002Ftool call accuracy by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2300\n* fix: coroutine warning for bleu by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2301\n* Add base_url parameter to embedding_factor by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2303\n* fix: add disallowed_special  on tiktoken encode by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2304\n* Feat\u002Ftool call f1 1893 by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2305\n* Feature\u002Fazure token usage extraction by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2306\n* fix: improve metric decorators with better validation and error handling by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2302\n* Metric\u002Fparallel tool call by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2307\n* Fix: avoid ambiguous truth value for empty numpy array in HuggingfaceEmbeddings (fixes #2080) by @Rahul2512Chauhan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2308\n* Devpod cn\u002Fmain by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2309\n* Feat\u002Fquoted spans metric by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2311\n* Fix noise sensitivity compute by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2312\n* Corrected numerous typos in Markdown files. by @ker2xu in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F1994\n* Deprecation warnings for LLMs and Prompts by @rhlbhatnagar in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2253\n* Docs\u002Feval_rag_agent - how to evaluate and improve rag app by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2293\n* Add llamaindex agentic evals gemini by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2317\n* fix: type str in tests by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2318\n* Fix generate_multiple caching issue (#1980) by @Rahul2512Chauhan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2314\n* fix: metric inheritance patterns: separate factory-created metrics from class-instantiated metrics by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2316\n* fix: concurrent ResponseRelevancy by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2328\n* fix: answer_relevancy scoring logic to prevent false zero by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2327\n* feat: Add OCI Gen AI Integration for Direct LLM Support by @harshil-sanghvi in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2321\n* feat: Add save\u002Fload functionality and improved repr for LLM-based metrics by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2320\n* Fix: Fixed the Numpy 3.13 issue by @Prigoistic in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2282\n* refactor: docs and warnings for metric base new structure by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2333\n* fix: typing by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2334\n\n## New Contributors\n* @kauabh made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2221\n* @s3pi made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2292\n* @claudepi made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2295\n* @telesoho made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2286\n* @ker2xu made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F1994\n* @harshil-sanghvi made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2321\n* @Prigoistic made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2282\n\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fcompare\u002Fv0.3.5...v0.3.6","2025-10-03T03:56:31",{"id":186,"version":187,"summary_zh":188,"released_at":189},107416,"v0.3.5","## What's Changed\n* Docs\u002Fhowto-texttosqlagent by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2264\n* fix: preview logo was too small.  by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2277\n* modified the documentation to be in sync with current output format by @kotalaraghava in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2281\n* removed some meta properties to test by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2278\n* feature: improve async \u002F executor functionality by @ahgraber in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2070\n* modification of the translate instruction by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2284\n* Remove experimental from docs and fix examples in docs by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2270\n* fix: resolve TypeError in TopicAdherenceScore bitwise operations by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2258\n* Knowledge graph\u002Foptimize for large corpus by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2267\n* Update _nv_metrics.py by @titericz in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2053\n* Add telemetry by @rhlbhatnagar in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2260\n* OpenAI model cost by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2287\n* docs: agent metrics code examples improvement  by @yesidc in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F1983\n* Prompt Optimization Tutorial by @sahusiddharth in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F1993\n* Feature\u002Fmetric type checking by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2288\n* improved the release script for `ragas-examples` by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2289\n* fix: removed the need for regex patterns by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2290\n\n## New Contributors\n* @kotalaraghava made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2281\n* @yesidc made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F1983\n\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fcompare\u002Fv0.3.4...v0.3.5","2025-09-17T19:13:15",{"id":191,"version":192,"summary_zh":79,"released_at":193},107417,"v0.3.5rc2","2025-09-17T17:40:25",{"id":195,"version":196,"summary_zh":197,"released_at":198},107418,"v0.3.5rc1","## What's Changed\n* Docs\u002Fhowto-texttosqlagent by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2264\n* fix: preview logo was too small.  by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2277\n* modified the documentation to be in sync with current output format by @kotalaraghava in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2281\n* removed some meta properties to test by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2278\n* feature: improve async \u002F executor functionality by @ahgraber in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2070\n* modification of the translate instruction by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2284\n* Remove experimental from docs and fix examples in docs by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2270\n* fix: resolve TypeError in TopicAdherenceScore bitwise operations by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2258\n* Knowledge graph\u002Foptimize for large corpus by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2267\n* Update _nv_metrics.py by @titericz in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2053\n* Add telemetry by @rhlbhatnagar in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2260\n* OpenAI model cost by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2287\n* docs: agent metrics code examples improvement  by @yesidc in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F1983\n* Prompt Optimization Tutorial by @sahusiddharth in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F1993\n* Feature\u002Fmetric type checking by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2288\n* improved the release script for `ragas-examples` by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2289\n\n## New Contributors\n* @kotalaraghava made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2281\n* @yesidc made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F1983\n\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fcompare\u002Fv0.3.4...v0.3.5rc1","2025-09-17T17:31:23",{"id":200,"version":201,"summary_zh":202,"released_at":203},107419,"v0.3.4","## What's Changed\n* Update context_precision.md by @anupamck in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2262\n* Reduce find_indirect_clusters() runtime through neighborhood detection and sampling by @ahgraber in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2144\n* fix: handle langchain multiple batching by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2257\n* chore: fix the dead space in the header for docsite by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2265\n* total bin coverage for default_transform() in Knowledge Graph transformations by @tolgaerdonmez in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F1950\n* Improved context precision documentation by @anupamck in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2266\n* Fix docs: correct inheritance class for Non-LLM metrics by @AlanPonnachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2272\n* added rb2b analytics by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2273\n\n## New Contributors\n* @anupamck made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2262\n* @tolgaerdonmez made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F1950\n* @AlanPonnachan made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2272\n\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fcompare\u002Fv0.3.3...v0.3.4","2025-09-10T23:57:28",{"id":205,"version":206,"summary_zh":207,"released_at":208},107420,"v0.3.3","## What's Changed\n* feat: refactor Ragas `embeddings` to maintain backward compatibility by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2184\n* feat: move `tracing` from experimental to ragas\u002Fintegrations by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2188\n* feat: Refactor LLM structure and add InstructorLLM by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2191\n* fix: setup and local ci by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2199\n* docs: correct small typos in Testset Generation page and related md components by @deibid in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2123\n* Docs\u002Fhow-to-benchmark-new-llm-guide by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2158\n* fix: access model fields via class by @emmanuel-ferdman in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2146\n* add: PR template for better review and overview of changes by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2202\n* Move `prompts` to ragas by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2197\n* Fix: make the answer be the same as the split sentences by @wlbksy in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2128\n* Merge workspace.toml into pyproject.toml by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2189\n* Cleanup\u002Fmove-benchmark-llm-to-examples by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2203\n* fix: shields badge for downloads replace with pepy. by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2204\n* refactor: removing some section to streamline pr template by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2206\n* Move dataset module to main package and update imports by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2205\n* fix: type check for pyright by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2211\n* refactor: move ragas out to project root. by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2209\n* chore: fix the OIDC error in claude code by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2212\n* fix: _version.py is to be ignored in git by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2213\n* chore: only review for `\u002Fclaude-review` command by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2214\n* Fix a couple of typos in `docs\u002Fgetstarted\u002Fevals.md` by @sadra-barikbin in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2081\n* Fix repeating template bug in _nv_metrics.py by @dev-nid in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2216\n* fix: ragas dir structure path fixes by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2215\n* fix: minor typo in  context_precision.md by @AbhishekRP2002 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2050\n* fix: update docstrings for single-hop use case by @terry07 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F1947\n* Fix issue with cache hits by @Nicba1010 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F1977\n* Use generate_multiple instead of generate in a loop in ResponseRelevancy by @aabanic in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F1978\n* fix: update required columns in RefusalRate (docs) by @DmitrySirakov in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2008\n* Add support for make install-minimal to speed up CI + dev by @rhlbhatnagar in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2218\n* Move experimental metrics to main package by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2220\n* Fix e2e test failures with datasets>=4.0.0 and missing dependencies by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2222\n* Move Utils \u002F Exceptions module to main package and update imports by @rhlbhatnagar in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2219\n* update compare_embeddings doc links by @resitaydin in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2037\n* chore: add `py.typed` marker file to support type checking by @sisp in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2029\n* Retire `ragas.experimental` namespace by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2223\n* chore: cleanup docs for ragas directory by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2224\n* fix: update make commands to use uv for docs by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2217\n* Remove experimental folder by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2225\n* Docs\u002Fhow-to-iterate-prompt by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2226\n* security: stop supporting v0.2 security releases by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2230\n* Optimize factual correctness metric runtime by 50% by @vignesh14052002 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2153\n* Fixed the mermaid which was breaking on navigation by @ganeshrvel in https:\u002F\u002Fgithub.com\u002Fe","2025-09-04T17:59:51",{"id":210,"version":211,"summary_zh":207,"released_at":212},107421,"v0.3.3rc1","2025-09-04T17:46:09",{"id":214,"version":215,"summary_zh":216,"released_at":217},107422,"v0.3.2","## What's Changed\n* feat: save and load prompt object by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2152\n* refactor: use basemetric as base for metric by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2163\n* add: pre-commit hook to run ci on every commit by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2164\n* Make `AnswerSimilarity` a dataclass by @hutec in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2159\n* docs: Add discord link to Community page in docs by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2157\n* Fixed a small typo in context_precision.md by @ImadSaddik in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2166\n* Move Experimental to inside Ragas by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2167\n* refactor: experiment function to use model as additional optional arg by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2165\n* Add python 3.13 target, refactor on pre-commit and benchmarking tests by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2169\n* fix: enum values instead of names in testset synthesizers by @lovets18 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2137\n* Cleaner, simpler pyproject.toml by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2173\n* Move Examples to root by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2176\n* Add `backend` to ragas main by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2174\n* Remove txt requirements by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2180\n* Remove black by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2178\n* feat: move `experiment` from experimental to ragas main by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2175\n* feat: migrate CLI from experimental to main package by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2183\n* Reformat examples by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2186\n* Remove simulation by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2187\n* refactor: avoid notebook in test. run all in test-all by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2171\n* Add Claude Code GitHub Workflow by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2192\n* hotfix: Include LICENSE and README.md in PyPI package by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2195\n* fix: pypi requirements and image absolute path by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2196\n* chore: fix release script by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2198\n\n## New Contributors\n* @anistark made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2163\n* @hutec made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2159\n* @ImadSaddik made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2166\n* @lovets18 made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2137\n\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fcompare\u002Fv0.3.1...v0.3.2","2025-08-19T12:03:50",{"id":219,"version":220,"summary_zh":79,"released_at":221},107423,"v0.3.2rc3","2025-08-19T12:01:12",{"id":223,"version":224,"summary_zh":225,"released_at":226},107424,"v0.3.2-rc2","## What's Changed\n* fix: pypi requirements and image absolute path by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2196\n\n\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fcompare\u002Fv0.3.2-rc1...v0.3.2-rc2","2025-08-19T11:49:23",{"id":228,"version":229,"summary_zh":230,"released_at":231},107425,"v0.3.2-rc1","## What's Changed\n* feat: save and load prompt object by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2152\n* refactor: use basemetric as base for metric by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2163\n* add: pre-commit hook to run ci on every commit by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2164\n* Make `AnswerSimilarity` a dataclass by @hutec in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2159\n* docs: Add discord link to Community page in docs by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2157\n* Fixed a small typo in context_precision.md by @ImadSaddik in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2166\n* Move Experimental to inside Ragas by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2167\n* refactor: experiment function to use model as additional optional arg by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2165\n* Add python 3.13 target, refactor on pre-commit and benchmarking tests by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2169\n* fix: enum values instead of names in testset synthesizers by @lovets18 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2137\n* Cleaner, simpler pyproject.toml by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2173\n* Move Examples to root by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2176\n* Add `backend` to ragas main by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2174\n* Remove txt requirements by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2180\n* Remove black by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2178\n* feat: move `experiment` from experimental to ragas main by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2175\n* feat: migrate CLI from experimental to main package by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2183\n* Reformat examples by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2186\n* Remove simulation by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2187\n* refactor: avoid notebook in test. run all in test-all by @anistark in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2171\n* Add Claude Code GitHub Workflow by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2192\n* hotfix: Include LICENSE and README.md in PyPI package by @NirantK in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2195\n\n## New Contributors\n* @anistark made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2163\n* @hutec made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2159\n* @ImadSaddik made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2166\n* @lovets18 made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2137\n\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fcompare\u002Fv0.3.1...v0.3.2-rc1","2025-08-19T10:28:33",{"id":233,"version":234,"summary_zh":235,"released_at":236},107426,"v0.3.1","## What's Changed\n* docs: improve hello world by @shahules786 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2127\n* examples: fix api changes by @shahules786 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2129\n* docs: add explanation by @shahules786 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2139\n* adds Google Drive backend for storing datasets as sheets by @dmikey in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2138\n* Fix experimental docs navigation, fix broken tutorials, improve examples for better user understanding by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2156\n* fix: correct mkdocstrings path configuration for local development by @sanjeed5 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2155\n\n## New Contributors\n* @dmikey made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2138\n* @sanjeed5 made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2156\n\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fcompare\u002Fv0.3.0...v0.3.1","2025-08-11T11:15:03",{"id":238,"version":239,"summary_zh":240,"released_at":241},107427,"v0.3.0","## What's Changed\n* llamaindex agentic integration by @sahusiddharth in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2026\n* CVE-2025-45691 Fix: Arbitrary File Read Vulnerability + Feat: Security settings by @adithyan-ak in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F1991\n* feat: add devin's deepwiki by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2039\n* feat: added `ragas_experimental` back into main by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2031\n* fix: dataset slicing by @shahules786 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2057\n* feat: user simulator by @shahules786 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2062\n* chore: make CI pass by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2063\n* Update analytics source by @emmanuel-ferdman in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2059\n* fix: update embedding extraction to use appropriate async method by @ahgraber in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2068\n* feat: check embedding shapes in CosineSimilarityBuilders by @ahgraber in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2069\n* feat: llm as prompt as optional by @shahules786 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2084\n* chore: added claude code workflow by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2089\n* chore: refactor the experimental code off nbdev by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2087\n* feat: align and validate alignment rate by @shahules786 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2094\n* Fix\u002Fdocs upgrade shade 2 by @ganeshrvel in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2097\n* feat: ragas evals CLI by @shahules786 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2086\n* docs: hello world for ragas experimental by @shahules786 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2100\n* feat: refactored out `Project`  by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2106\n* Updated the header color and moved away from yellow shade to more dar… by @ganeshrvel in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2104\n* examples : example evals by @shahules786 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2109\n* Fix\u002Ftheme black primary color by @ganeshrvel in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2113\n* fix: cleaned up some metrics by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2111\n* feat: add `llm_factory` and `embedding_factory` by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2112\n* chore: release scripts and cleanup for monorepo by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2117\n* fix: release script by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2118\n* docs: hello world & explanation by @shahules786 in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2114\n* chore: added `ragas.experimental` and cleaned dependencies by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2119\n* fix: metric alignment with `inmemory` backend by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2116\n* docs: added structure to sidebar by @jjmachan in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2120\n\n## New Contributors\n* @emmanuel-ferdman made their first contribution in https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fpull\u002F2059\n\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas\u002Fcompare\u002Fv0.2.15...v0.3.0","2025-07-17T05:32:24"]