[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"similar-NirDiamant--RAG_Techniques":3,"tool-NirDiamant--RAG_Techniques":61},[4,18,26,36,44,53],{"id":5,"name":6,"github_repo":7,"description_zh":8,"stars":9,"difficulty_score":10,"last_commit_at":11,"category_tags":12,"status":17},4358,"openclaw","openclaw\u002Fopenclaw","OpenClaw 是一款专为个人打造的本地化 AI 助手，旨在让你在自己的设备上拥有完全可控的智能伙伴。它打破了传统 AI 助手局限于特定网页或应用的束缚，能够直接接入你日常使用的各类通讯渠道，包括微信、WhatsApp、Telegram、Discord、iMessage 等数十种平台。无论你在哪个聊天软件中发送消息，OpenClaw 都能即时响应，甚至支持在 macOS、iOS 和 Android 设备上进行语音交互，并提供实时的画布渲染功能供你操控。\n\n这款工具主要解决了用户对数据隐私、响应速度以及“始终在线”体验的需求。通过将 AI 部署在本地，用户无需依赖云端服务即可享受快速、私密的智能辅助，真正实现了“你的数据，你做主”。其独特的技术亮点在于强大的网关架构，将控制平面与核心助手分离，确保跨平台通信的流畅性与扩展性。\n\nOpenClaw 非常适合希望构建个性化工作流的技术爱好者、开发者，以及注重隐私保护且不愿被单一生态绑定的普通用户。只要具备基础的终端操作能力（支持 macOS、Linux 及 Windows WSL2），即可通过简单的命令行引导完成部署。如果你渴望拥有一个懂你",349277,3,"2026-04-06T06:32:30",[13,14,15,16],"Agent","开发框架","图像","数据工具","ready",{"id":19,"name":20,"github_repo":21,"description_zh":22,"stars":23,"difficulty_score":10,"last_commit_at":24,"category_tags":25,"status":17},3808,"stable-diffusion-webui","AUTOMATIC1111\u002Fstable-diffusion-webui","stable-diffusion-webui 是一个基于 Gradio 构建的网页版操作界面，旨在让用户能够轻松地在本地运行和使用强大的 Stable Diffusion 图像生成模型。它解决了原始模型依赖命令行、操作门槛高且功能分散的痛点，将复杂的 AI 绘图流程整合进一个直观易用的图形化平台。\n\n无论是希望快速上手的普通创作者、需要精细控制画面细节的设计师，还是想要深入探索模型潜力的开发者与研究人员，都能从中获益。其核心亮点在于极高的功能丰富度：不仅支持文生图、图生图、局部重绘（Inpainting）和外绘（Outpainting）等基础模式，还独创了注意力机制调整、提示词矩阵、负向提示词以及“高清修复”等高级功能。此外，它内置了 GFPGAN 和 CodeFormer 等人脸修复工具，支持多种神经网络放大算法，并允许用户通过插件系统无限扩展能力。即使是显存有限的设备，stable-diffusion-webui 也提供了相应的优化选项，让高质量的 AI 艺术创作变得触手可及。",162132,"2026-04-05T11:01:52",[14,15,13],{"id":27,"name":28,"github_repo":29,"description_zh":30,"stars":31,"difficulty_score":32,"last_commit_at":33,"category_tags":34,"status":17},1381,"everything-claude-code","affaan-m\u002Feverything-claude-code","everything-claude-code 是一套专为 AI 编程助手（如 Claude Code、Codex、Cursor 等）打造的高性能优化系统。它不仅仅是一组配置文件，而是一个经过长期实战打磨的完整框架，旨在解决 AI 代理在实际开发中面临的效率低下、记忆丢失、安全隐患及缺乏持续学习能力等核心痛点。\n\n通过引入技能模块化、直觉增强、记忆持久化机制以及内置的安全扫描功能，everything-claude-code 能显著提升 AI 在复杂任务中的表现，帮助开发者构建更稳定、更智能的生产级 AI 代理。其独特的“研究优先”开发理念和针对 Token 消耗的优化策略，使得模型响应更快、成本更低，同时有效防御潜在的攻击向量。\n\n这套工具特别适合软件开发者、AI 研究人员以及希望深度定制 AI 工作流的技术团队使用。无论您是在构建大型代码库，还是需要 AI 协助进行安全审计与自动化测试，everything-claude-code 都能提供强大的底层支持。作为一个曾荣获 Anthropic 黑客大奖的开源项目，它融合了多语言支持与丰富的实战钩子（hooks），让 AI 真正成长为懂上",142651,2,"2026-04-06T23:34:12",[14,13,35],"语言模型",{"id":37,"name":38,"github_repo":39,"description_zh":40,"stars":41,"difficulty_score":32,"last_commit_at":42,"category_tags":43,"status":17},2271,"ComfyUI","Comfy-Org\u002FComfyUI","ComfyUI 是一款功能强大且高度模块化的视觉 AI 引擎，专为设计和执行复杂的 Stable Diffusion 图像生成流程而打造。它摒弃了传统的代码编写模式，采用直观的节点式流程图界面，让用户通过连接不同的功能模块即可构建个性化的生成管线。\n\n这一设计巧妙解决了高级 AI 绘图工作流配置复杂、灵活性不足的痛点。用户无需具备编程背景，也能自由组合模型、调整参数并实时预览效果，轻松实现从基础文生图到多步骤高清修复等各类复杂任务。ComfyUI 拥有极佳的兼容性，不仅支持 Windows、macOS 和 Linux 全平台，还广泛适配 NVIDIA、AMD、Intel 及苹果 Silicon 等多种硬件架构，并率先支持 SDXL、Flux、SD3 等前沿模型。\n\n无论是希望深入探索算法潜力的研究人员和开发者，还是追求极致创作自由度的设计师与资深 AI 绘画爱好者，ComfyUI 都能提供强大的支持。其独特的模块化架构允许社区不断扩展新功能，使其成为当前最灵活、生态最丰富的开源扩散模型工具之一，帮助用户将创意高效转化为现实。",107888,"2026-04-06T11:32:50",[14,15,13],{"id":45,"name":46,"github_repo":47,"description_zh":48,"stars":49,"difficulty_score":32,"last_commit_at":50,"category_tags":51,"status":17},4721,"markitdown","microsoft\u002Fmarkitdown","MarkItDown 是一款由微软 AutoGen 团队打造的轻量级 Python 工具，专为将各类文件高效转换为 Markdown 格式而设计。它支持 PDF、Word、Excel、PPT、图片（含 OCR）、音频（含语音转录）、HTML 乃至 YouTube 链接等多种格式的解析，能够精准提取文档中的标题、列表、表格和链接等关键结构信息。\n\n在人工智能应用日益普及的今天，大语言模型（LLM）虽擅长处理文本，却难以直接读取复杂的二进制办公文档。MarkItDown 恰好解决了这一痛点，它将非结构化或半结构化的文件转化为模型“原生理解”且 Token 效率极高的 Markdown 格式，成为连接本地文件与 AI 分析 pipeline 的理想桥梁。此外，它还提供了 MCP（模型上下文协议）服务器，可无缝集成到 Claude Desktop 等 LLM 应用中。\n\n这款工具特别适合开发者、数据科学家及 AI 研究人员使用，尤其是那些需要构建文档检索增强生成（RAG）系统、进行批量文本分析或希望让 AI 助手直接“阅读”本地文件的用户。虽然生成的内容也具备一定可读性，但其核心优势在于为机器",93400,"2026-04-06T19:52:38",[52,14],"插件",{"id":54,"name":55,"github_repo":56,"description_zh":57,"stars":58,"difficulty_score":10,"last_commit_at":59,"category_tags":60,"status":17},4487,"LLMs-from-scratch","rasbt\u002FLLMs-from-scratch","LLMs-from-scratch 是一个基于 PyTorch 的开源教育项目，旨在引导用户从零开始一步步构建一个类似 ChatGPT 的大型语言模型（LLM）。它不仅是同名技术著作的官方代码库，更提供了一套完整的实践方案，涵盖模型开发、预训练及微调的全过程。\n\n该项目主要解决了大模型领域“黑盒化”的学习痛点。许多开发者虽能调用现成模型，却难以深入理解其内部架构与训练机制。通过亲手编写每一行核心代码，用户能够透彻掌握 Transformer 架构、注意力机制等关键原理，从而真正理解大模型是如何“思考”的。此外，项目还包含了加载大型预训练权重进行微调的代码，帮助用户将理论知识延伸至实际应用。\n\nLLMs-from-scratch 特别适合希望深入底层原理的 AI 开发者、研究人员以及计算机专业的学生。对于不满足于仅使用 API，而是渴望探究模型构建细节的技术人员而言，这是极佳的学习资源。其独特的技术亮点在于“循序渐进”的教学设计：将复杂的系统工程拆解为清晰的步骤，配合详细的图表与示例，让构建一个虽小但功能完备的大模型变得触手可及。无论你是想夯实理论基础，还是为未来研发更大规模的模型做准备",90106,"2026-04-06T11:19:32",[35,15,13,14],{"id":62,"github_repo":63,"name":64,"description_en":65,"description_zh":66,"ai_summary_zh":66,"readme_en":67,"readme_zh":68,"quickstart_zh":69,"use_case_zh":70,"hero_image_url":71,"owner_login":72,"owner_name":73,"owner_avatar_url":74,"owner_bio":75,"owner_company":76,"owner_location":73,"owner_email":73,"owner_twitter":77,"owner_website":78,"owner_url":79,"languages":80,"stars":89,"forks":90,"last_commit_at":91,"license":92,"difficulty_score":32,"env_os":93,"env_gpu":93,"env_ram":93,"env_deps":94,"category_tags":97,"github_topics":98,"view_count":32,"oss_zip_url":73,"oss_zip_packed_at":73,"status":17,"created_at":108,"updated_at":109,"faqs":110,"releases":146},4809,"NirDiamant\u002FRAG_Techniques","RAG_Techniques","This repository showcases various advanced techniques for Retrieval-Augmented Generation (RAG) systems. RAG systems combine information retrieval with generative models to provide accurate and contextually rich responses.","RAG_Techniques 是一个专注于检索增强生成（RAG）系统的开源知识库，旨在汇集并展示各种先进的 RAG 优化技巧。它通过将信息检索技术与生成式模型相结合，帮助开发者构建能够提供更准确、上下文更丰富回答的 AI 应用。\n\n在实际应用中，基础版的 RAG 系统常面临检索内容不相关、关键信息丢失或回答缺乏深度等挑战。RAG_Techniques 正是为了解决这些痛点而生，它提供了一套动态更新的技术教程集合，涵盖了从数据分块策略、高级检索算法到上下文优化等多维度的解决方案，从而显著提升系统的准确性与运行效率。\n\n该项目非常适合 AI 开发者、研究人员以及希望深入理解大模型落地技术的工程师使用。无论是想要优化现有生产环境中的问答机器人，还是致力于探索前沿学术研究的团队，都能从中获得极具价值的实践指南。其核心亮点在于内容的全面性与前瞻性，不仅整理了当前业界领先的实施方法，还持续跟进最新技术动态，堪称提升 RAG 系统性能的实战宝典。通过参考这里的方案，用户可以轻松跨越从理论到高效落地的鸿沟。","[![PRs Welcome](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPRs-welcome-brightgreen.svg?style=flat-square)](http:\u002F\u002Fmakeapullrequest.com)\n[![LinkedIn](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLinkedIn-Connect-blue)](https:\u002F\u002Fwww.linkedin.com\u002Fin\u002Fnir-diamant-759323134\u002F)\n[![Twitter](https:\u002F\u002Fimg.shields.io\u002Ftwitter\u002Ffollow\u002FNirDiamantAI?label=Follow%20@NirDiamantAI&style=social)](https:\u002F\u002Ftwitter.com\u002FNirDiamantAI)\n[![Reddit](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FReddit-Join%20our%20subreddit-FF4500?style=flat-square&logo=reddit&logoColor=white)](https:\u002F\u002Fwww.reddit.com\u002Fr\u002FEducationalAI\u002F)\n[![Discord](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FDiscord-Join%20our%20community-7289da?style=flat-square&logo=discord&logoColor=white)](https:\u002F\u002Fdiscord.gg\u002FcA6Aa4uyDX)\n[![Sponsor](https:\u002F\u002Fimg.shields.io\u002Fstatic\u002Fv1?label=Sponsor&message=%E2%9D%A4&logo=GitHub&color=ff69b4)](https:\u002F\u002Fgithub.com\u002Fsponsors\u002FNirDiamant)\n\n\n\n> 🌟 **Support This Project:** Your sponsorship fuels innovation in RAG technologies. **[Become a sponsor](https:\u002F\u002Fwww.diamant-ai.com\u002Fsponsorship)** to help maintain and expand this valuable resource!\n\n## Sponsors ❤️\n\nWe gratefully acknowledge the organizations and individuals who have made significant contributions to this project.\n\n**Company Sponsors**\n\n\u003Cdiv align=\"center\">\n\n\u003Ctable style=\"border: none; border-collapse: collapse; width: 100%; max-width: 840px; margin: 0 auto; background: transparent; table-layout: fixed;\">\n  \u003Ctr>\n    \u003Ctd style=\"border: none; text-align: center; padding: 16px 24px; width: 33.33%; vertical-align: top;\">\n      \u003Ca href=\"https:\u002F\u002Fapp.contextual.ai?utm_campaign=rag-techniques&utm_source=diamantai&utm_medium=github&utm_content=notebook\" target=\"_blank\" style=\"text-decoration: none; display: inline-block; transition: transform 0.2s ease;\">\n        \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FNirDiamant_RAG_Techniques_readme_12d53258e26d.png\" \n             alt=\"Contextual AI\"\n             style=\"height: 28px; width: auto; border-radius: 12px; vertical-align: middle; box-shadow: 0 2px 8px rgba(0,0,0,0.1);\">\n        \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FNirDiamant_RAG_Techniques_readme_6ec4792e925c.png\" \n             alt=\"Contextual AI\"\n             style=\"height: 28px; width: auto; border-radius: 12px; vertical-align: middle; box-shadow: 0 2px 8px rgba(0,0,0,0.1);\">\n      \u003C\u002Fa>\n    \u003C\u002Ftd>\n    \u003Ctd style=\"border: none; text-align: center; padding: 16px 24px; width: 33.33%; vertical-align: middle;\">\n      \u003Ca href=\"https:\u002F\u002Fcoderabbit.link\u002Fnir\" target=\"_blank\" style=\"text-decoration: none; display: inline-block; transition: transform 0.2s ease;\">\n        \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FNirDiamant_RAG_Techniques_readme_fdef4d816bf0.png\"\n             alt=\"CodeRabbit\"\n             style=\"height: 28px; width: auto; border-radius: 12px; vertical-align: middle; box-shadow: 0 2px 8px rgba(0,0,0,0.1);\">\n        \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FNirDiamant_RAG_Techniques_readme_07395db6faa4.png\"\n             alt=\"CodeRabbit\"\n             style=\"height: 28px; width: auto; border-radius: 12px; vertical-align: middle; box-shadow: 0 2px 8px rgba(0,0,0,0.1);\">\n      \u003C\u002Fa>\n    \u003C\u002Ftd>\n  \u003C\u002Ftr>\n\u003C\u002Ftable>\n\u003C\u002Fdiv>\n\n\n\n\n\n**Individual Sponsors**\n\n\u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fsponsors\u002FEisenh\">\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FNirDiamant_RAG_Techniques_readme_61c61852bc80.png\" style=\"border-radius: 50%;\" width=\"64\" height=\"64\" alt=\"\">\u003C\u002Fa>\n\n# Advanced RAG Techniques: Elevating Your Retrieval-Augmented Generation Systems 🚀\n\nWelcome to one of the most comprehensive and dynamic collections of Retrieval-Augmented Generation (RAG) tutorials available today. This repository serves as a hub for cutting-edge techniques aimed at enhancing the accuracy, efficiency, and contextual richness of RAG systems.\n\n## 📫 Stay Updated!\n\n\u003Cdiv align=\"center\">\n\u003Ctable>\n\u003Ctr>\n\u003Ctd align=\"center\">🚀\u003Cbr>\u003Cb>Cutting-edge\u003Cbr>Updates\u003C\u002Fb>\u003C\u002Ftd>\n\u003Ctd align=\"center\">💡\u003Cbr>\u003Cb>Expert\u003Cbr>Insights\u003C\u002Fb>\u003C\u002Ftd>\n\u003Ctd align=\"center\">🎯\u003Cbr>\u003Cb>Top 0.1%\u003Cbr>Content\u003C\u002Fb>\u003C\u002Ftd>\n\n\u003C\u002Ftr>\n\u003C\u002Ftable>\n\n[![Subscribe to DiamantAI Newsletter](images\u002Fsubscribe-button.svg)](https:\u002F\u002Fdiamantai.substack.com\u002F?r=336pe4&utm_campaign=pub-share-checklist)\n\n*Join over 50,000 AI enthusiasts getting unique cutting-edge insights and free tutorials!* ***Plus, subscribers get exclusive early access and special 33% discounts to my book and the upcoming RAG Techniques course!***\n\u003C\u002Fdiv>\n\n\n\n[![DiamantAI's newsletter](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FNirDiamant_RAG_Techniques_readme_bd4960312f91.png)](https:\u002F\u002Fdiamantai.substack.com\u002F?r=336pe4&utm_campaign=pub-share-checklist)\n\n\n## Introduction\n\nRetrieval-Augmented Generation (RAG) is revolutionizing the way we combine information retrieval with generative AI. This repository showcases a curated collection of advanced techniques designed to supercharge your RAG systems, enabling them to deliver more accurate, contextually relevant, and comprehensive responses.\n\nOur goal is to provide a valuable resource for researchers and practitioners looking to push the boundaries of what's possible with RAG. By fostering a collaborative environment, we aim to accelerate innovation in this exciting field.\n\n## Related Projects\n\n🚀 Level up with my **[Agents Towards Production](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002Fagents-towards-production)** repository. It delivers horizontal, code-first tutorials that cover every tool and step in the lifecycle of building production-grade GenAI agents, guiding you from spark to scale with proven patterns and reusable blueprints for real-world launches, making it the smartest place to start if you're serious about shipping agents to production.\n\n🤖 Explore my **[GenAI Agents Repository](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FGenAI_Agents)** to discover a variety of AI agent implementations and tutorials, showcasing how different AI technologies can be combined to create powerful, interactive systems.\n\n\n🖋️ Check out my  **[Prompt Engineering Techniques guide](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FPrompt_Engineering)** for a comprehensive collection of prompting strategies, from basic concepts to advanced techniques, enhancing your ability to interact effectively with AI language models.\n\n\n## A Community-Driven Knowledge Hub\n\n**This repository grows stronger with your contributions!** Join our vibrant communities - the central hubs for shaping and advancing this project together 🤝\n\n**[Educational AI Subreddit](https:\u002F\u002Fwww.reddit.com\u002Fr\u002FEducationalAI\u002F)**\n\n**[RAG Techniques Discord Community](https:\u002F\u002Fdiscord.gg\u002FcA6Aa4uyDX)**\n\nWhether you're an expert or just starting out, your insights can shape the future of RAG. Join us to propose ideas, get feedback, and collaborate on innovative techniques. For contribution guidelines, please refer to our **[CONTRIBUTING.md](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002FCONTRIBUTING.md)** file. Let's advance RAG technology together!\n\n🔗 For discussions on GenAI, RAG, or custom agents, or to explore knowledge-sharing opportunities, feel free to **[connect on LinkedIn](https:\u002F\u002Fwww.linkedin.com\u002Fin\u002Fnir-diamant-759323134\u002F)**.\n\n## Key Features\n\n- 🧠 State-of-the-art RAG enhancements\n- 📚 Comprehensive documentation for each technique\n- 🛠️ Practical implementation guidelines\n- 🌟 Regular updates with the latest advancements\n\n## Advanced Techniques\n\nExplore our extensive list of cutting-edge RAG techniques:\n\n| # | Category | Technique | View |\n|---|----------|-----------|------|\n| 1 | ⭐ Key Collaboration | Agentic RAG with Contextual AI | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FAgentic_RAG.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FAgentic_RAG.ipynb) |\n| 2 | Foundational 🌱 | Basic RAG | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_rag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_rag.ipynb) |\n| 3 | Foundational 🌱 | RAG with CSV Files | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_csv_rag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_csv_rag.ipynb) |\n| 4 | Foundational 🌱 | Reliable RAG | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Freliable_rag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Freliable_rag.ipynb) |\n| 5 | Foundational 🌱 | Optimizing Chunk Sizes | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fchoose_chunk_size.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fchoose_chunk_size.ipynb) |\n| 6 | Foundational 🌱 | Proposition Chunking | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fproposition_chunking.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fproposition_chunking.ipynb) |\n| 7 | Query Enhancement 🔍 | Query Transformations | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fquery_transformations.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fquery_transformations.ipynb) |\n| 8 | Query Enhancement 🔍 | HyDE (Hypothetical Document Embedding) | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FHyDe_Hypothetical_Document_Embedding.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FHyDe_Hypothetical_Document_Embedding.ipynb) |\n| 9 | Query Enhancement 🔍 | HyPE (Hypothetical Prompt Embedding) | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FHyPE_Hypothetical_Prompt_Embeddings.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FHyPE_Hypothetical_Prompt_Embeddings.ipynb) |\n| 10 | Context Enrichment 📚 | Contextual Chunk Headers | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontextual_chunk_headers.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontextual_chunk_headers.ipynb) |\n| 11 | Context Enrichment 📚 | Relevant Segment Extraction | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Frelevant_segment_extraction.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Frelevant_segment_extraction.ipynb) |\n| 12 | Context Enrichment 📚 | Context Window Enhancement | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontext_enrichment_window_around_chunk.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontext_enrichment_window_around_chunk.ipynb) |\n| 13 | Context Enrichment 📚 | Semantic Chunking | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsemantic_chunking.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsemantic_chunking.ipynb) |\n| 14 | Context Enrichment 📚 | Contextual Compression | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontextual_compression.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontextual_compression.ipynb) |\n| 15 | Context Enrichment 📚 | Document Augmentation | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fdocument_augmentation.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fdocument_augmentation.ipynb) |\n| 16 | Advanced Retrieval 🚀 | Fusion Retrieval | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Ffusion_retrieval.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Ffusion_retrieval.ipynb) |\n| 17 | Advanced Retrieval 🚀 | Reranking | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Freranking.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Freranking.ipynb) |\n| 18 | Advanced Retrieval 🚀 | Multi-faceted Filtering | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fmulti_faceted_filtering.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fmulti_faceted_filtering.ipynb) |\n| 19 | Advanced Retrieval 🚀 | Hierarchical Indices | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fhierarchical_indices.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fhierarchical_indices.ipynb) |\n| 20 | Advanced Retrieval 🚀 | Ensemble Retrieval | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fensemble_retrieval.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fensemble_retrieval.ipynb) |\n| 21 | Advanced Retrieval 🚀 | Dartboard Retrieval | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fdartboard.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fdartboard.ipynb) |\n| 22 | Advanced Retrieval 🚀 | Multi-modal RAG with Captioning | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fmulti_model_rag_with_captioning.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fmulti_model_rag_with_captioning.ipynb) |\n| 23 | Iterative Techniques 🔁 | Retrieval with Feedback Loop | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fretrieval_with_feedback_loop.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fretrieval_with_feedback_loop.ipynb) |\n| 24 | Iterative Techniques 🔁 | Adaptive Retrieval | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fadaptive_retrieval.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fadaptive_retrieval.ipynb) |\n| 25 | Iterative Retrieval 🔄 | Iterative Retrieval | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fiterative_retrieval.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fiterative_retrieval.ipynb) |\n| 26 | Evaluation 📊 | DeepEval | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fevaluation\u002Fevaluation_deep_eval.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fevaluation\u002Fevaluation_deep_eval.ipynb) |\n| 27 | Evaluation 📊 | GroUSE | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fevaluation\u002Fevaluation_grouse.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fevaluation\u002Fevaluation_grouse.ipynb) |\n| 28 | Explainability 🔬 | Explainable Retrieval | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fexplainable_retrieval.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fexplainable_retrieval.ipynb) |\n| 29 | Advanced Architecture 🏗️ | Graph RAG with LangChain | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fgraph_rag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fgraph_rag.ipynb) |\n| 30 | Advanced Architecture 🏗️ | Microsoft GraphRAG | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FMicrosoft_GraphRag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FMicrosoft_GraphRag.ipynb) |\n| 31 | Advanced Architecture 🏗️ | RAPTOR | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fraptor.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fraptor.ipynb) |\n| 32 | Advanced Architecture 🏗️ | Self-RAG | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fself_rag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fself_rag.ipynb) |\n| 33 | Advanced Architecture 🏗️ | Corrective RAG (CRAG) | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcrag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcrag.ipynb) |\n| 34 | Special Technique 🌟 | Sophisticated Controllable Agent | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FControllable-RAG-Agent) |\n\n### 🌱 Foundational RAG Techniques\n\n1. Simple RAG 🌱\n   - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_rag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_rag.ipynb)\n   - **LlamaIndex**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_rag_with_llamaindex.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_rag_with_llamaindex.ipynb)\n   - **[Runnable Script](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques_runnable_scripts\u002Fsimple_rag.py)**\n\n   #### Overview 🔎\n   Introducing basic RAG techniques ideal for newcomers.\n\n   #### Implementation 🛠️\n   Start with basic retrieval queries and integrate incremental learning mechanisms.\n\n2. Simple RAG using a CSV file 🧩\n   - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_csv_rag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_csv_rag.ipynb)\n   - **LlamaIndex**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_csv_rag_with_llamaindex.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_csv_rag_with_llamaindex.ipynb)\n   \n    #### Overview 🔎\n    Introducing basic RAG using CSV files.\n\n    #### Implementation 🛠️\n    This uses CSV files to create basic retrieval and integrates with openai to create question and answering system.\n\n3. **Reliable RAG 🏷️**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Freliable_rag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Freliable_rag.ipynb)\n\n   #### Overview 🔎\n   Enhances the Simple RAG by adding validation and refinement to ensure the accuracy and relevance of retrieved information.\n\n   #### Implementation 🛠️\n   Check for retrieved document relevancy and highlight the segment of docs used for answering.\n\n4. Choose Chunk Size 📏  \n   - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fchoose_chunk_size.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fchoose_chunk_size.ipynb)  \n   - **[Runnable Script](all_rag_techniques_runnable_scripts\u002Fchoose_chunk_size.py)**\n\n   #### Overview 🔎\n   Selecting an appropriate fixed size for text chunks to balance context preservation and retrieval efficiency.\n\n   #### Implementation 🛠️\n   Experiment with different chunk sizes to find the optimal balance between preserving context and maintaining retrieval speed for your specific use case.\n\n5. **Proposition Chunking ⛓️‍💥**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fproposition_chunking.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fproposition_chunking.ipynb)\n\n   #### Overview 🔎\n   Breaking down the text into concise, complete, meaningful sentences allowing for better control and handling of specific queries (especially extracting knowledge).\n\n   #### Implementation 🛠️\n   - 💪 **Proposition Generation:** The LLM is used in conjunction with a custom prompt to generate factual statements from the document chunks.\n   - ✅ **Quality Checking:** The generated propositions are passed through a grading system that evaluates accuracy, clarity, completeness, and conciseness.\n\n#### Additional Resources 📚\n   - **[The Propositions Method: Enhancing Information Retrieval for AI Systems](https:\u002F\u002Fopen.substack.com\u002Fpub\u002Fdiamantai\u002Fp\u002Fthe-propositions-method-enhancing?r=336pe4&utm_campaign=post&utm_medium=web)** - A comprehensive blog post exploring the benefits and implementation of proposition chunking in RAG systems.\n\n### 🔍 Query Enhancement\n\n6. Query Transformations 🔄  \n   - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fquery_transformations.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fquery_transformations.ipynb)  \n   - **[Runnable Script](all_rag_techniques_runnable_scripts\u002Fquery_transformations.py)**\n\n   #### Overview 🔎\n   Modifying and expanding queries to improve retrieval effectiveness.\n\n   #### Implementation 🛠️\n   - ✍️ **Query Rewriting:** Reformulate queries to improve retrieval.\n   - 🔙 **Step-back Prompting:** Generate broader queries for better context retrieval.\n   - 🧩 **Sub-query Decomposition:** Break complex queries into simpler sub-queries.\n\n7. Hypothetical Questions (HyDE Approach) ❓  \n   - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FHyDe_Hypothetical_Document_Embedding.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FHyDe_Hypothetical_Document_Embedding.ipynb)  \n   - **[Runnable Script](all_rag_techniques_runnable_scripts\u002FHyDe_Hypothetical_Document_Embedding.py)**\n\n   #### Overview 🔎\n   Generating hypothetical questions to improve alignment between queries and data.\n\n   #### Implementation 🛠️\n   Create hypothetical questions that point to relevant locations in the data, enhancing query-data matching.\n\n   #### Additional Resources 📚\n   - **[HyDE: Exploring Hypothetical Document Embeddings for AI Retrieval](https:\u002F\u002Fopen.substack.com\u002Fpub\u002Fdiamantai\u002Fp\u002Fhyde-exploring-hypothetical-document?r=336pe4&utm_campaign=post&utm_medium=web)** - A short blog post explaining this method clearly.\n\n### 📚 Context and Content Enrichment\n\n8. Hypothetical Prompt Embeddings (HyPE) ❓🚀  \n   - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FHyPE_Hypothetical_Prompt_Embedding.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FHyPE_Hypothetical_Prompt_Embedding.ipynb)  \n   - **[Runnable Script](all_rag_techniques_runnable_scripts\u002FHyPE_Hypothetical_Prompt_Embeddings.py)**  \n\n   #### Overview 🔎  \n   HyPE (Hypothetical Prompt Embeddings) is an enhancement to traditional RAG retrieval that **precomputes hypothetical prompts at the indexing stage**, but inseting the chunk in their place. This transforms retrieval into a **question-question matching task**. This avoids the need for runtime synthetic answer generation, reducing inference-time computational overhead while **improving retrieval alignment**.  \n\n   #### Implementation 🛠️  \n   - 📖 **Precomputed Questions:** Instead of embedding document chunks, HyPE **generates multiple hypothetical queries per chunk** at indexing time.  \n   - 🔍 **Question-Question Matching:** User queries are matched against stored hypothetical questions, leading to **better retrieval alignment**.  \n   - ⚡ **No Runtime Overhead:** Unlike HyDE, HyPE does **not require LLM calls at query time**, making retrieval **faster and cheaper**.  \n   - 📈 **Higher Precision & Recall:** Improves retrieval **context precision by up to 42 percentage points** and **claim recall by up to 45 percentage points**.  \n\n   #### Additional Resources 📚  \n   - **[Preprint: Hypothetical Prompt Embeddings (HyPE)](https:\u002F\u002Fpapers.ssrn.com\u002Fsol3\u002Fpapers.cfm?abstract_id=5139335)** - Research paper detailing the method, evaluation, and benchmarks.  \n\n\n9. **Contextual Chunk Headers :label:**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontextual_chunk_headers.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontextual_chunk_headers.ipynb)\n\n    #### Overview 🔎\n    Contextual chunk headers (CCH) is a method of creating document-level and section-level context, and prepending those chunk headers to the chunks prior to embedding them.\n\n    #### Implementation 🛠️\n    Create a chunk header that includes context about the document and\u002For section of the document, and prepend that to each chunk in order to improve the retrieval accuracy.\n\n    #### Additional Resources 📚\n    **[dsRAG](https:\u002F\u002Fgithub.com\u002FD-Star-AI\u002FdsRAG)**: open-source retrieval engine that implements this technique (and a few other advanced RAG techniques)\n\n10. **Relevant Segment Extraction 🧩**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Frelevant_segment_extraction.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Frelevant_segment_extraction.ipynb)\n\n    #### Overview 🔎\n    Relevant segment extraction (RSE) is a method of dynamically constructing multi-chunk segments of text that are relevant to a given query.\n\n    #### Implementation 🛠️\n    Perform a retrieval post-processing step that analyzes the most relevant chunks and identifies longer multi-chunk segments to provide more complete context to the LLM.\n\n11. Context Enrichment Techniques 📝  \n   - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontext_enrichment_window_around_chunk.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontext_enrichment_window_around_chunk.ipynb)  \n   - **LlamaIndex**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontext_enrichment_window_around_chunk_with_llamaindex.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontext_enrichment_window_around_chunk_with_llamaindex.ipynb)\n   - **[Runnable Script](all_rag_techniques_runnable_scripts\u002Fcontext_enrichment_window_around_chunk.py)**\n  \n   #### Overview 🔎\n   Enhancing retrieval accuracy by embedding individual sentences and extending context to neighboring sentences.\n\n   #### Implementation 🛠️\n   Retrieve the most relevant sentence while also accessing the sentences before and after it in the original text.\n\n12. Semantic Chunking 🧠\n   - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsemantic_chunking.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsemantic_chunking.ipynb)\n   - **[Runnable Script](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques_runnable_scripts\u002Fsemantic_chunking.py)**\n\n   #### Overview 🔎\n   Dividing documents based on semantic coherence rather than fixed sizes.\n\n   #### Implementation 🛠️\n   Use NLP techniques to identify topic boundaries or coherent sections within documents for more meaningful retrieval units.\n\n   #### Additional Resources 📚\n   - **[Semantic Chunking: Improving AI Information Retrieval](https:\u002F\u002Fopen.substack.com\u002Fpub\u002Fdiamantai\u002Fp\u002Fsemantic-chunking-improving-ai-information?r=336pe4&utm_campaign=post&utm_medium=web)** - A comprehensive blog post exploring the benefits and implementation of semantic chunking in RAG systems.\n\n13. Contextual Compression 🗜️  \n   - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontextual_compression.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontextual_compression.ipynb)  \n   - **[Runnable Script](all_rag_techniques_runnable_scripts\u002Fcontextual_compression.py)**\n\n   #### Overview 🔎\n   Compressing retrieved information while preserving query-relevant content.\n\n   #### Implementation 🛠️\n   Use an LLM to compress or summarize retrieved chunks, preserving key information relevant to the query.\n\n14. Document Augmentation through Question Generation for Enhanced Retrieval  \n   - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fdocument_augmentation.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fdocument_augmentation.ipynb)  \n   - **[Runnable Script](all_rag_techniques_runnable_scripts\u002Fdocument_augmentation.py)**\n\n   #### Overview 🔎\n   This implementation demonstrates a text augmentation technique that leverages additional question generation to improve document retrieval within a vector database. By generating and incorporating various questions related to each text fragment, the system enhances the standard retrieval process, thus increasing the likelihood of finding relevant documents that can be utilized as context for generative question answering.\n\n   #### Implementation 🛠️\n   Use an LLM to augment text dataset with all possible questions that can be asked to each document.\n\n### 🚀 Advanced Retrieval Methods\n\n15. Fusion Retrieval 🔗  \n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Ffusion_retrieval.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Ffusion_retrieval.ipynb)  \n    - **LlamaIndex**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Ffusion_retrieval_with_llamaindex.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Ffusion_retrieval_with_llamaindex.ipynb)\n    - **[Runnable Script](all_rag_techniques_runnable_scripts\u002Ffusion_retrieval.py)**\n \n    #### Overview 🔎\n    Optimizing search results by combining different retrieval methods.\n    \n    #### Implementation 🛠️\n    Combine keyword-based search with vector-based search for more comprehensive and accurate retrieval.\n\n16. Intelligent Reranking 📈  \n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Freranking.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Freranking.ipynb)  \n    - **LlamaIndex**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Freranking_with_llamaindex.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Freranking_with_llamaindex.ipynb)\n    - **[Runnable Script](all_rag_techniques_runnable_scripts\u002Freranking.py)**\n\n    #### Overview 🔎\n    Applying advanced scoring mechanisms to improve the relevance ranking of retrieved results.\n    \n    #### Implementation 🛠️\n    - 🧠 **LLM-based Scoring:** Use a language model to score the relevance of each retrieved chunk.\n    - 🔀 **Cross-Encoder Models:** Re-encode both the query and retrieved documents jointly for similarity scoring.\n    - 🏆 **Metadata-enhanced Ranking:** Incorporate metadata into the scoring process for more nuanced ranking.\n\n    #### Additional Resources 📚\n    - **[Relevance Revolution: How Re-ranking Transforms RAG Systems](https:\u002F\u002Fopen.substack.com\u002Fpub\u002Fdiamantai\u002Fp\u002Frelevance-revolution-how-re-ranking?r=336pe4&utm_campaign=post&utm_medium=web)** - A comprehensive blog post exploring the power of re-ranking in enhancing RAG system performance.\n\n17. Multi-faceted Filtering 🔍\n\n    #### Overview 🔎\n    Applying various filtering techniques to refine and improve the quality of retrieved results.\n\n    #### Implementation 🛠️\n    - 🏷️ **Metadata Filtering:** Apply filters based on attributes like date, source, author, or document type.\n    - 📊 **Similarity Thresholds:** Set thresholds for relevance scores to keep only the most pertinent results.\n    - 📄 **Content Filtering:** Remove results that don't match specific content criteria or essential keywords.\n    - 🌈 **Diversity Filtering:** Ensure result diversity by filtering out near-duplicate entries.\n\n18. Hierarchical Indices 🗂️  \n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fhierarchical_indices.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fhierarchical_indices.ipynb)  \n    - **[Runnable Script](all_rag_techniques_runnable_scripts\u002Fhierarchical_indices.py)**\n\n    #### Overview 🔎\n    Creating a multi-tiered system for efficient information navigation and retrieval.\n    \n    #### Implementation 🛠️\n    Implement a two-tiered system for document summaries and detailed chunks, both containing metadata pointing to the same location in the data.\n    \n    #### Additional Resources 📚\n    - **[Hierarchical Indices: Enhancing RAG Systems](https:\u002F\u002Fopen.substack.com\u002Fpub\u002Fdiamantai\u002Fp\u002Fhierarchical-indices-enhancing-rag?r=336pe4&utm_campaign=post&utm_medium=web)** - A comprehensive blog post exploring the power of hierarchical indices in enhancing RAG system performance.\n\n19. Ensemble Retrieval 🎭\n\n    #### Overview 🔎\n    Combining multiple retrieval models or techniques for more robust and accurate results.\n\n    #### Implementation 🛠️\n    Apply different embedding models or retrieval algorithms and use voting or weighting mechanisms to determine the final set of retrieved documents.\n\n20. Dartboard Retrieval 🎯\n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fdartboard.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fdartboard.ipynb) \n    #### Overview 🔎\n    Optimizing over Relevant Information Gain in Retrieval\n\n    #### Implementation 🛠️\n    - Combine both relevance and diversity into a single scoring function and directly optimize for it.\n    - POC showing plain simple RAG underperforming when the database is dense, and the dartboard retrieval outperforming it.\n\n21. Multi-modal Retrieval 📽️\n\n    #### Overview 🔎\n    Extending RAG capabilities to handle diverse data types for richer responses.\n\n    #### Implementation 🛠️\n    - **Multi-model RAG with Multimedia Captioning**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fmulti_model_rag_with_captioning.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fmulti_model_rag_with_captioning.ipynb) - Caption and store all the other multimedia data like pdfs, ppts, etc., with text data in vector store and retrieve them together.\n    - **Multi-model RAG with Colpali**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fmulti_model_rag_with_colpali.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fmulti_model_rag_with_colpali.ipynb) - Instead of captioning convert all the data into image, then find the most relevant images and pass them to a vision large language model.\n    \n\n### 🔁 Iterative and Adaptive Techniques\n\n22. Retrieval with Feedback Loops 🔁  \n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fretrieval_with_feedback_loop.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fretrieval_with_feedback_loop.ipynb)  \n    - **[Runnable Script](all_rag_techniques_runnable_scripts\u002Fretrieval_with_feedback_loop.py)**\n\n    #### Overview 🔎\n    Implementing mechanisms to learn from user interactions and improve future retrievals.\n\n    #### Implementation 🛠️\n    Collect and utilize user feedback on the relevance and quality of retrieved documents and generated responses to fine-tune retrieval and ranking models.\n\n23. Adaptive Retrieval 🎯  \n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fadaptive_retrieval.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fadaptive_retrieval.ipynb)  \n    - **[Runnable Script](all_rag_techniques_runnable_scripts\u002Fadaptive_retrieval.py)**\n\n    #### Overview 🔎\n    Dynamically adjusting retrieval strategies based on query types and user contexts.\n\n    #### Implementation 🛠️\n    Classify queries into different categories and use tailored retrieval strategies for each, considering user context and preferences.\n\n24. Iterative Retrieval 🔄\n\n    #### Overview 🔎\n    Performing multiple rounds of retrieval to refine and enhance result quality.\n\n    #### Implementation 🛠️\n    Use the LLM to analyze initial results and generate follow-up queries to fill in gaps or clarify information.\n\n### 📊 Evaluation\n\n25. **DeepEval Evaluation**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fevaluation\u002Fevaluation_deep_eval.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fevaluation\u002Fevaluation_deep_eval.ipynb) | Comprehensive RAG system evaluation |\n\n    #### Overview 🔎\n    Performing evaluations Retrieval-Augmented Generation systems, by covering several metrics and creating test cases.\n\n    #### Implementation 🛠️\n    Use the `deepeval` library to conduct test cases on correctness, faithfulness and contextual relevancy of RAG systems.\n    \n\n26. **GroUSE Evaluation**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fevaluation\u002Fevaluation_grouse.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fevaluation\u002Fevaluation_grouse.ipynb) | Contextually-grounded LLM evaluation |\n\n    #### Overview 🔎\n    Evaluate the final stage of Retrieval-Augmented Generation using metrics of the GroUSE framework and meta-evaluate your custom LLM judge on GroUSE unit tests.\n\n    #### Implementation 🛠️\n    Use the `grouse` package to evaluate contextually-grounded LLM generations with GPT-4 on the 6 metrics of the GroUSE framework and use unit tests to evaluate a custom Llama 3.1 405B evaluator.\n    \n\n### 🔬 Explainability and Transparency\n\n27. Explainable Retrieval 🔍  \n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fexplainable_retrieval.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fexplainable_retrieval.ipynb)  \n    - **[Runnable Script](all_rag_techniques_runnable_scripts\u002Fexplainable_retrieval.py)**\n\n    #### Overview 🔎\n    Providing transparency in the retrieval process to enhance user trust and system refinement.\n    \n    #### Implementation 🛠️\n    Explain why certain pieces of information were retrieved and how they relate to the query.\n\n### 🏗️ Advanced Architectures\n\n28. Agentic RAG with Contextual AI 🤖 \n    - **Agentic RAG**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FAgentic_RAG.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FAgentic_RAG.ipynb)  \n\n    #### Overview 🔎\n    Building production-ready agentic RAG pipelines for financial document analysis with Contextual AI's managed platform. This comprehensive tutorial demonstrates how to leverage agentic RAG to solve complex queries through intelligent query reformulation, document parsing, reranking, and grounded language models.\n    \n    #### Implementation 🛠️\n    - **Document Parser**: Enterprise-grade parsing with vision models for complex tables, charts, and multi-page documents\n    - **Instruction-Following Reranker**: SOTA reranker with instruction-following capabilities for handling conflicting information\n    - **Grounded Language Model (GLM)**: World's most grounded LLM specifically engineered to minimize hallucinations for RAG use cases\n    - **LMUnit**: Natural language unit testing framework for evaluating and optimizing RAG system performance\n\n29. Graph RAG with Milvus Vector Database 🔍 \n    - **Graph RAG with Milvus**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fgraphrag_with_milvus_vectordb.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fgraphrag_with_milvus_vectordb.ipynb)  \n\n    #### Overview 🔎\n    A simple yet powerful approach to implement Graph RAG using Milvus vector databases. This technique significantly improves performance on complex multi-hop questions by combining relationship-based retrieval with vector search and reranking.\n    \n    #### Implementation 🛠️\n    - Store both text passages and relationship triplets (subject-predicate-object) in separate Milvus collections\n    - Perform multi-way retrieval by querying both collections\n    - Use an LLM to rerank retrieved relationships based on their relevance to the query \n    - Retrieve the final passages based on the most relevant relationships\n\n30. Knowledge Graph Integration (Graph RAG) 🕸️  \n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fgraph_rag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fgraph_rag.ipynb)  \n    - **[Runnable Script](all_rag_techniques_runnable_scripts\u002Fgraph_rag.py)**\n\n    #### Overview 🔎\n    Incorporating structured data from knowledge graphs to enrich context and improve retrieval.\n    \n    #### Implementation 🛠️\n    Retrieve entities and their relationships from a knowledge graph relevant to the query, combining this structured data with unstructured text for more informative responses.\n\n31. GraphRag (Microsoft) 🎯\n    - **GraphRag**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FMicrosoft_GraphRag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FMicrosoft_GraphRag.ipynb)  \n\n    #### Overview 🔎\n    Microsoft GraphRAG (Open Source) is an advanced RAG system that integrates knowledge graphs to improve the performance of LLMs\n    \n    #### Implementation 🛠️\n    • Analyze an input corpus by extracting entities, relationships from text units. generates summaries of each community and its constituents from the bottom-up.\n\n32. RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval 🌳  \n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fraptor.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fraptor.ipynb)  \n    - **[Runnable Script](all_rag_techniques_runnable_scripts\u002Fraptor.py)**\n\n    #### Overview 🔎\n    Implementing a recursive approach to process and organize retrieved information in a tree structure.\n    \n    #### Implementation 🛠️\n    Use abstractive summarization to recursively process and summarize retrieved documents, organizing the information in a tree structure for hierarchical context.\n\n33. Self RAG 🔁  \n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fself_rag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fself_rag.ipynb)  \n    - **[Runnable Script](all_rag_techniques_runnable_scripts\u002Fself_rag.py)**\n\n    #### Overview 🔎\n    A dynamic approach that combines retrieval-based and generation-based methods, adaptively deciding whether to use retrieved information and how to best utilize it in generating responses.\n    \n    #### Implementation 🛠️\n    • Implement a multi-step process including retrieval decision, document retrieval, relevance evaluation, response generation, support assessment, and utility evaluation to produce accurate, relevant, and useful outputs.\n\n34. Corrective RAG 🔧  \n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcrag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcrag.ipynb)  \n    - **[Runnable Script](all_rag_techniques_runnable_scripts\u002Fcrag.py)**\n\n    #### Overview 🔎\n    A sophisticated RAG approach that dynamically evaluates and corrects the retrieval process, combining vector databases, web search, and language models for highly accurate and context-aware responses.\n    \n    #### Implementation 🛠️\n    • Integrate Retrieval Evaluator, Knowledge Refinement, Web Search Query Rewriter, and Response Generator components to create a system that adapts its information sourcing strategy based on relevance scores and combines multiple sources when necessary.\n\n## 🌟 Special Advanced Technique 🌟\n\n35. **[Sophisticated Controllable Agent for Complex RAG Tasks 🤖](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FControllable-RAG-Agent)**\n\n    #### Overview 🔎\n    An advanced RAG solution designed to tackle complex questions that simple semantic similarity-based retrieval cannot solve. This approach uses a sophisticated deterministic graph as the \"brain\" 🧠 of a highly controllable autonomous agent, capable of answering non-trivial questions from your own data.\n\n    #### Implementation 🛠️\n    • Implement a multi-step process involving question anonymization, high-level planning, task breakdown, adaptive information retrieval and question answering, continuous re-planning, and rigorous answer verification to ensure grounded and accurate responses.\n\n\n\n## Getting Started\n\nTo begin implementing these advanced RAG techniques in your projects:\n\n1. Clone this repository:\n   ```\n   git clone https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_Techniques.git\n   ```\n2. Navigate to the technique you're interested in:\n   ```\n   cd all_rag_techniques\u002Ftechnique-name\n   ```\n3. Follow the detailed implementation guide in each technique's directory.\n\n## Contributing\n\nWe welcome contributions from the community! If you have a new technique or improvement to suggest:\n\n1. Fork the repository\n2. Create your feature branch: `git checkout -b feature\u002FAmazingFeature`\n3. Commit your changes: `git commit -m 'Add some AmazingFeature'`\n4. Push to the branch: `git push origin feature\u002FAmazingFeature`\n5. Open a pull request\n\n## Contributors\n\n[![Contributors](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FNirDiamant_RAG_Techniques_readme_31ed344e7518.png)](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_Techniques\u002Fgraphs\u002Fcontributors)\n\n## License\n\nThis project is licensed under a custom non-commercial license - see the [LICENSE](LICENSE) file for details.\n\n---\n\n⭐️ If you find this repository helpful, please consider giving it a star!\n\n![](https:\u002F\u002Feurope-west1-rag-techniques-views-tracker.cloudfunctions.net\u002Frag-techniques-tracker?notebook=main-readme)\n\nKeywords: RAG, Retrieval-Augmented Generation, NLP, AI, Machine Learning, Information Retrieval, Natural Language Processing, LLM, Embeddings, Semantic Search\n","[![欢迎提交PR](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPRs-welcome-brightgreen.svg?style=flat-square)](http:\u002F\u002Fmakeapullrequest.com)\n[![LinkedIn](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLinkedIn-Connect-blue)](https:\u002F\u002Fwww.linkedin.com\u002Fin\u002Fnir-diamant-759323134\u002F)\n[![Twitter](https:\u002F\u002Fimg.shields.io\u002Ftwitter\u002Ffollow\u002FNirDiamantAI?label=Follow%20@NirDiamantAI&style=social)](https:\u002F\u002Ftwitter.com\u002FNirDiamantAI)\n[![Reddit](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FReddit-Join%20our%20subreddit-FF4500?style=flat-square&logo=reddit&logoColor=white)](https:\u002F\u002Fwww.reddit.com\u002Fr\u002FEducationalAI\u002F)\n[![Discord](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FDiscord-Join%20our%20community-7289da?style=flat-square&logo=discord&logoColor=white)](https:\u002F\u002Fdiscord.gg\u002FcA6Aa4uyDX)\n[![赞助](https:\u002F\u002Fimg.shields.io\u002Fstatic\u002Fv1?label=Sponsor&message=%E2%9D%A4&logo=GitHub&color=ff69b4)](https:\u002F\u002Fgithub.com\u002Fsponsors\u002FNirDiamant)\n\n\n\n> 🌟 **支持本项目：** 您的赞助将为RAG技术的创新提供动力。**[成为赞助者](https:\u002F\u002Fwww.diamant-ai.com\u002Fsponsorship)**，帮助维护并扩展这一宝贵资源！\n\n## 赞助商 ❤️\n\n我们衷心感谢那些为本项目做出重大贡献的组织和个人。\n\n**公司赞助商**\n\n\u003Cdiv align=\"center\">\n\n\u003Ctable style=\"border: none; border-collapse: collapse; width: 100%; max-width: 840px; margin: 0 auto; background: transparent; table-layout: fixed;\">\n  \u003Ctr>\n    \u003Ctd style=\"border: none; text-align: center; padding: 16px 24px; width: 33.33%; vertical-align: top;\">\n      \u003Ca href=\"https:\u002F\u002Fapp.contextual.ai?utm_campaign=rag-techniques&utm_source=diamantai&utm_medium=github&utm_content=notebook\" target=\"_blank\" style=\"text-decoration: none; display: inline-block; transition: transform 0.2s ease;\">\n        \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FNirDiamant_RAG_Techniques_readme_12d53258e26d.png\" \n             alt=\"Contextual AI\"\n             style=\"height: 28px; width: auto; border-radius: 12px; vertical-align: middle; box-shadow: 0 2px 8px rgba(0,0,0,0.1);\">\n        \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FNirDiamant_RAG_Techniques_readme_6ec4792e925c.png\" \n             alt=\"Contextual AI\"\n             style=\"height: 28px; width: auto; border-radius: 12px; vertical-align: middle; box-shadow: 0 2px 8px rgba(0,0,0,0.1);\">\n      \u003C\u002Fa>\n    \u003C\u002Ftd>\n    \u003Ctd style=\"border: none; text-align: center; padding: 16px 24px; width: 33.33%; vertical-align: middle;\">\n      \u003Ca href=\"https:\u002F\u002Fcoderabbit.link\u002Fnir\" target=\"_blank\" style=\"text-decoration: none; display: inline-block; transition: transform 0.2s ease;\">\n        \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FNirDiamant_RAG_Techniques_readme_fdef4d816bf0.png\"\n             alt=\"CodeRabbit\"\n             style=\"height: 28px; width: auto; border-radius: 12px; vertical-align: middle; box-shadow: 0 2px 8px rgba(0,0,0,0.1);\">\n        \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FNirDiamant_RAG_Techniques_readme_07395db6faa4.png\"\n             alt=\"CodeRabbit\"\n             style=\"height: 28px; width: auto; border-radius: 12px; vertical-align: middle; box-shadow: 0 2px 8px rgba(0,0,0,0.1);\">\n      \u003C\u002Fa>\n    \u003C\u002Ftd>\n  \u003C\u002Ftr>\n\u003C\u002Ftable>\n\u003C\u002Fdiv>\n\n\n\n\n\n**个人赞助商**\n\n\u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fsponsors\u002FEisenh\">\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FNirDiamant_RAG_Techniques_readme_61c61852bc80.png\" style=\"border-radius: 50%;\" width=\"64\" height=\"64\" alt=\"\">\u003C\u002Fa>\n\n# 高级RAG技术：提升你的检索增强生成系统 🚀\n\n欢迎来到当今最全面、最具活力的检索增强生成（RAG）教程集合之一。本仓库是用于提升RAG系统准确性、效率和上下文丰富性的尖端技术中心。\n\n## 📫 保持更新！\n\n\u003Cdiv align=\"center\">\n\u003Ctable>\n\u003Ctr>\n\u003Ctd align=\"center\">🚀\u003Cbr>\u003Cb>前沿\u003Cbr>更新\u003C\u002Fb>\u003C\u002Ftd>\n\u003Ctd align=\"center\">💡\u003Cbr>\u003Cb>专家\u003Cbr>见解\u003C\u002Fb>\u003C\u002Ftd>\n\u003Ctd align=\"center\">🎯\u003Cbr>\u003Cb>顶尖0.1%\u003Cbr>内容\u003C\u002Fb>\u003C\u002Ftd>\n\n\u003C\u002Ftr>\n\u003C\u002Ftable>\n\n[![订阅DiamantAI通讯](images\u002Fsubscribe-button.svg)](https:\u002F\u002Fdiamantai.substack.com\u002F?r=336pe4&utm_campaign=pub-share-checklist)\n\n*加入超过5万名人工智能爱好者，获取独特的前沿见解和免费教程！* ***此外，订阅用户还可享受我书籍及即将推出的RAG技术课程的独家抢先体验和特别33%折扣！***\n\u003C\u002Fdiv>\n\n\n\n[![DiamantAI的通讯](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FNirDiamant_RAG_Techniques_readme_bd4960312f91.png)](https:\u002F\u002Fdiamantai.substack.com\u002F?r=336pe4&utm_campaign=pub-share-checklist)\n\n\n## 简介\n\n检索增强生成（RAG）正在革新我们结合信息检索与生成式AI的方式。本仓库展示了一系列精心挑选的高级技术，旨在大幅提升您的RAG系统性能，使其能够提供更准确、更具上下文相关性且更全面的回答。\n\n我们的目标是为希望突破RAG技术边界的研究人员和从业者提供宝贵的资源。通过营造协作环境，我们致力于加速这一激动人心领域的创新步伐。\n\n## 相关项目\n\n🚀 使用我的**[Agents Towards Production](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002Fagents-towards-production)**仓库进一步提升技能。它提供了以代码为中心的横向教程，涵盖了构建生产级GenAI代理生命周期中的每一个工具和步骤，从萌芽到规模化，采用经过验证的模式和可重用蓝图实现真实世界的部署，是您认真考虑将代理投入生产的最佳起点。\n\n🤖 探索我的**[GenAI Agents Repository](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FGenAI_Agents)**，发现各种AI代理实现和教程，展示如何结合不同的AI技术创建功能强大、交互式的系统。\n\n\n🖋️ 查看我的**[Prompt Engineering Techniques guide](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FPrompt_Engineering)**，这是一份从基础概念到高级技巧的全面提示策略汇编，可提升您与AI语言模型有效互动的能力。\n\n\n## 社区驱动的知识中心\n\n**本仓库因您的贡献而日益壮大！** 加入我们的活跃社区——共同塑造和推进该项目的核心枢纽 🤝\n\n**[Educational AI Subreddit](https:\u002F\u002Fwww.reddit.com\u002Fr\u002FEducationalAI\u002F)**\n\n**[RAG Techniques Discord Community](https:\u002F\u002Fdiscord.gg\u002FcA6Aa4uyDX)**\n\n无论您是专家还是初学者，您的见解都能影响RAG技术的未来。加入我们，提出想法、获取反馈，并合作开发创新技术。有关贡献指南，请参阅我们的**[CONTRIBUTING.md](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002FCONTRIBUTING.md)**文件。让我们携手推动RAG技术的进步！\n\n🔗 如需讨论GenAI、RAG或自定义代理，或探索知识共享机会，欢迎随时**[在LinkedIn上联系](https:\u002F\u002Fwww.linkedin.com\u002Fin\u002Fnir-diamant-759323134\u002F)**。\n\n## 核心特性\n\n- 🧠 最先进的RAG增强技术\n- 📚 每项技术的全面文档\n- 🛠️ 实用的实施指南\n- 🌟 定期更新最新进展\n\n## 高级技术\n\n探索我们丰富的前沿 RAG 技术列表：\n\n| # | 类别 | 技术 | 查看 |\n|---|----------|-----------|------|\n| 1 | ⭐ 关键协作 | 基于上下文的智能体RAG | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FAgentic_RAG.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FAgentic_RAG.ipynb) |\n| 2 | 基础 🌱 | 基本RAG | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_rag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_rag.ipynb) |\n| 3 | 基础 🌱 | 使用CSV文件的RAG | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_csv_rag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_csv_rag.ipynb) |\n| 4 | 基础 🌱 | 可靠的RAG | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Freliable_rag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Freliable_rag.ipynb) |\n| 5 | 基础 🌱 | 优化分块大小 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fchoose_chunk_size.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fchoose_chunk_size.ipynb) |\n| 6 | 基础 🌱 | 命题式分块 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fproposition_chunking.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fproposition_chunking.ipynb) |\n| 7 | 查询增强 🔍 | 查询变换 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fquery_transformations.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fquery_transformations.ipynb) |\n| 8 | 查询增强 🔍 | HyDE（假设文档嵌入） | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FHyDe_Hypothetical_Document_Embedding.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FHyDe_Hypothetical_Document_Embedding.ipynb) |\n| 9 | 查询增强 🔍 | HyPE（假设提示嵌入） | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FHyPE_Hypothetical_Prompt_Embeddings.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FHyPE_Hypothetical_Prompt_Embeddings.ipynb) |\n| 10 | 上下文增强 📚 | 上下文分块头 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontextual_chunk_headers.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontextual_chunk_headers.ipynb) |\n| 11 | 上下文增强 📚 | 相关片段提取 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Frelevant_segment_extraction.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Frelevant_segment_extraction.ipynb) |\n| 12 | 上下文增强 📚 | 上下文窗口增强 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontext_enrichment_window_around_chunk.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontext_enrichment_window_around_chunk.ipynb) |\n| 13 | 上下文增强 📚 | 语义分块 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsemantic_chunking.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsemantic_chunking.ipynb) |\n| 14 | 上下文增强 📚 | 上下文压缩 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontextual_compression.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontextual_compression.ipynb) |\n| 15 | 上下文增强 📚 | 文档增广 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fdocument_augmentation.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fdocument_augmentation.ipynb) |\n| 16 | 高级检索 🚀 | 融合检索 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Ffusion_retrieval.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Ffusion_retrieval.ipynb) |\n| 17 | 高级检索 🚀 | 重排序 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Freranking.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Freranking.ipynb) |\n| 18 | 高级检索 🚀 | 多维度过滤 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fmulti_faceted_filtering.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fmulti_faceted_filtering.ipynb) |\n| 19 | 高级检索 🚀 | 层次索引 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fhierarchical_indices.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fhierarchical_indices.ipynb) |\n| 20 | 高级检索 🚀 | 集成检索 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fensemble_retrieval.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fensemble_retrieval.ipynb) |\n| 21 | 高级检索 🚀 | 飞镖盘检索 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fdartboard.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fdartboard.ipynb) |\n| 22 | 高级检索 🚀 | 多模态RAG与字幕生成 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fmulti_model_rag_with_captioning.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fmulti_model_rag_with_captioning.ipynb) |\n| 23 | 迭代技术 🔁 | 带反馈循环的检索 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fretrieval_with_feedback_loop.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fretrieval_with_feedback_loop.ipynb) |\n| 24 | 迭代技术 🔁 | 自适应检索 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fadaptive_retrieval.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fadaptive_retrieval.ipynb) |\n| 25 | 迭代检索 🔄 | 迭代检索 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fiterative_retrieval.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fiterative_retrieval.ipynb) |\n| 26 | 评估 📊 | DeepEval | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fevaluation\u002Fevaluation_deep_eval.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fevaluation\u002Fevaluation_deep_eval.ipynb) |\n| 27 | 评估 📊 | GroUSE | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fevaluation\u002Fevaluation_grouse.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fevaluation\u002Fevaluation_grouse.ipynb) |\n| 28 | 可解释性 🔬 | 可解释的检索 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fexplainable_retrieval.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fexplainable_retrieval.ipynb) |\n| 29 | 高级架构 🏗️ | 使用LangChain的图RAG | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fgraph_rag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fgraph_rag.ipynb) |\n| 30 | 高级架构 🏗️ | 微软GraphRAG | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FMicrosoft_GraphRag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FMicrosoft_GraphRag.ipynb) |\n| 31 | 高级架构 🏗️ | RAPTOR | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fraptor.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fraptor.ipynb) |\n| 32 | 高级架构 🏗️ | 自我RAG | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fself_rag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fself_rag.ipynb) |\n| 33 | 高级架构 🏗️ | 纠正型RAG（CRAG） | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcrag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcrag.ipynb) |\n| 34 | 特殊技术 🌟 | 精湛可控代理 | [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FControllable-RAG-Agent) |\n\n### 🌱 基础 RAG 技术\n\n1. 简单 RAG 🌱\n   - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_rag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_rag.ipynb)\n   - **LlamaIndex**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_rag_with_llamaindex.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_rag_with_llamaindex.ipynb)\n   - **[可运行脚本](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques_runnable_scripts\u002Fsimple_rag.py)**\n\n   #### 概述 🔎\n   介绍适合初学者的基础 RAG 技术。\n\n   #### 实现 🛠️\n   从基础的检索查询开始，并集成增量学习机制。\n\n2. 使用 CSV 文件的简单 RAG 🧩\n   - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_csv_rag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_csv_rag.ipynb)\n   - **LlamaIndex**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_csv_rag_with_llamaindex.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsimple_csv_rag_with_llamaindex.ipynb)\n   \n    #### 概述 🔎\n    介绍使用 CSV 文件的基础 RAG。\n\n    #### 实现 🛠️\n    利用 CSV 文件创建基础检索功能，并与 OpenAI 集成以构建问答系统。\n\n3. **可靠 RAG 🏷️**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Freliable_rag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Freliable_rag.ipynb)\n\n   #### 概述 🔎\n   在简单 RAG 的基础上增加验证和精炼步骤，以确保检索信息的准确性和相关性。\n\n   #### 实现 🛠️\n   检查检索文档的相关性，并高亮显示用于回答问题的文档片段。\n\n4. 选择分块大小 📏  \n   - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fchoose_chunk_size.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fchoose_chunk_size.ipynb)  \n   - **[可运行脚本](all_rag_techniques_runnable_scripts\u002Fchoose_chunk_size.py)**\n\n   #### 概述 🔎\n   选择合适的固定文本分块大小，以在保持上下文完整性和检索效率之间取得平衡。\n\n   #### 实现 🛠️\n   尝试不同的分块大小，找到最适合您特定应用场景的、能够在保留上下文的同时维持检索速度的最佳平衡点。\n\n5. **命题分块 ⛓️‍💥**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fproposition_chunking.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fproposition_chunking.ipynb)\n\n   #### 概述 🔎\n   将文本分解为简洁、完整、有意义的句子，从而更好地控制和处理特定查询（尤其是知识提取）。\n\n   #### 实现 🛠️\n   - 💪 **命题生成:** 结合自定义提示词，利用大模型从文档分块中生成事实性陈述。\n   - ✅ **质量检查:** 将生成的命题送入评分系统，评估其准确性、清晰度、完整性和简洁性。\n\n#### 其他资源 📚\n   - **[命题方法：提升 AI 系统的信息检索能力](https:\u002F\u002Fopen.substack.com\u002Fpub\u002Fdiamantai\u002Fp\u002Fthe-propositions-method-enhancing?r=336pe4&utm_campaign=post&utm_medium=web)** - 一篇全面的博客文章，探讨命题分块在 RAG 系统中的优势及实现方法。\n\n### 🔍 查询增强\n\n6. 查询变换 🔄  \n   - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fquery_transformations.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fquery_transformations.ipynb)  \n   - **[可运行脚本](all_rag_techniques_runnable_scripts\u002Fquery_transformations.py)**\n\n   #### 概述 🔎\n   通过修改和扩展查询来提升检索效果。\n\n   #### 实现 🛠️\n   - ✍️ **查询重写:** 重新表述查询以提高检索质量。\n   - 🔙 **退一步提示:** 生成更宽泛的查询，以便更好地检索上下文。\n   - 🧩 **子查询分解:** 将复杂查询拆分为更简单的子查询。\n\n7. 假设性问题（HyDE 方法） ❓  \n   - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FHyDe_Hypothetical_Document_Embedding.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FHyDe_Hypothetical_Document_Embedding.ipynb)  \n   - **[可运行脚本](all_rag_techniques_runnable_scripts\u002FHyDe_Hypothetical_Document_Embedding.py)**\n\n   #### 概述 🔎\n   通过生成假设性问题来改善查询与数据之间的匹配度。\n\n   #### 实现 🛠️\n   创建指向数据中相关位置的假设性问题，从而增强查询与数据的匹配效果。\n\n   #### 补充资源 📚\n   - **[HyDE：探索用于 AI 检索的假设性文档嵌入](https:\u002F\u002Fopen.substack.com\u002Fpub\u002Fdiamantai\u002Fp\u002Fhyde-exploring-hypothetical-document?r=336pe4&utm_campaign=post&utm_medium=web)** - 一篇简明易懂的博客文章，详细解释了该方法。\n\n### 📚 上下文与内容增强\n\n8. 假设性提示嵌入（HyPE） ❓🚀  \n   - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FHyPE_Hypothetical_Prompt_Embedding.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FHyPE_Hypothetical_Prompt_Embedding.ipynb)  \n   - **[可运行脚本](all_rag_techniques_runnable_scripts\u002FHyPE_Hypothetical_Prompt_Embeddings.py)**  \n\n   #### 概述 🔎  \n   HyPE（假设性提示嵌入）是对传统 RAG 检索的一种改进，它在索引阶段**预先计算假设性提示**，并将这些提示插入到相应的位置。这样，检索就转变为一个**问题与问题匹配的任务**。这种方法避免了在运行时合成答案的需求，从而减少了推理时的计算开销，同时**提升了检索的匹配精度**。  \n\n   #### 实现 🛠️  \n   - 📖 **预计算问题:** 与直接嵌入文档片段不同，HyPE 在索引时会为每个片段**生成多个假设性查询**。  \n   - 🔍 **问题与问题匹配:** 用户查询将与存储的假设性问题进行匹配，从而实现**更好的检索匹配**。  \n   - ⚡ **无运行时开销:** 与 HyDE 不同，HyPE**无需在查询时调用 LLM**，因此检索过程**更快、更经济**。  \n   - 📈 **更高的精确率与召回率:** 可以将检索的**上下文精确率提高多达 42 个百分点**，并将**证据召回率提高多达 45 个百分点**。  \n\n   #### 补充资源 📚  \n   - **[预印本：假设性提示嵌入（HyPE）](https:\u002F\u002Fpapers.ssrn.com\u002Fsol3\u002Fpapers.cfm?abstract_id=5139335)** - 详细介绍了该方法、评估结果及基准测试的研究论文。  \n\n\n9. **上下文片段头 :label:**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontextual_chunk_headers.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontextual_chunk_headers.ipynb)\n\n    #### 概述 🔎\n    上下文片段头（CCH）是一种创建文档级和章节级上下文的方法，并在嵌入片段之前将其添加到片段的开头。\n\n    #### 实现 🛠️\n    创建包含文档和\u002F或章节上下文信息的片段头，并将其添加到每个片段的开头，以提高检索准确性。\n\n    #### 补充资源 📚\n    **[dsRAG](https:\u002F\u002Fgithub.com\u002FD-Star-AI\u002FdsRAG)**：一个实现了该技术（以及一些其他高级 RAG 技术）的开源检索引擎。\n\n10. **相关段落提取 🧩**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Frelevant_segment_extraction.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Frelevant_segment_extraction.ipynb)\n\n    #### 概述 🔎\n    相关段落提取（RSE）是一种动态构建与给定查询相关的多片段文本段落的方法。\n\n    #### 实现 🛠️\n    在检索后处理步骤中，分析最相关的片段并识别出较长的多片段段落，以便为 LLM 提供更完整的上下文。\n\n11. 上下文增强技术 📝  \n   - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontext_enrichment_window_around_chunk.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontext_enrichment_window_around_chunk.ipynb)  \n   - **LlamaIndex**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontext_enrichment_window_around_chunk_with_llamaindex.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontext_enrichment_window_around_chunk_with_llamaindex.ipynb)\n   - **[可运行脚本](all_rag_techniques_runnable_scripts\u002Fcontext_enrichment_window_around_chunk.py)**\n  \n   #### 概述 🔎\n   通过嵌入单个句子并将上下文扩展到相邻句子，从而提高检索准确性。\n\n   #### 实现 🛠️\n   检索最相关的句子，同时访问原文中该句子前后相邻的句子。\n\n12. 语义分块 🧠\n   - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsemantic_chunking.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fsemantic_chunking.ipynb)\n   - **[可运行脚本](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques_runnable_scripts\u002Fsemantic_chunking.py)**\n\n   #### 概述 🔎\n   根据语义连贯性而非固定大小来划分文档。\n\n   #### 实现 🛠️\n   使用自然语言处理技术识别文档中的主题边界或连贯段落，以获得更有意义的检索单元。\n\n   #### 补充资源 📚\n   - **[语义分块：提升AI信息检索](https:\u002F\u002Fopen.substack.com\u002Fpub\u002Fdiamantai\u002Fp\u002Fsemantic-chunking-improving-ai-information?r=336pe4&utm_campaign=post&utm_medium=web)** - 一篇全面的博客文章，探讨语义分块在RAG系统中的优势及实现方法。\n\n13. 上下文压缩 🗜️  \n   - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontextual_compression.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcontextual_compression.ipynb)  \n   - **[可运行脚本](all_rag_techniques_runnable_scripts\u002Fcontextual_compression.py)**\n\n   #### 概述 🔎\n   在保留与查询相关的内容的同时，对检索到的信息进行压缩。\n\n   #### 实现 🛠️\n   使用大语言模型对检索到的文本块进行压缩或摘要，以保留与查询相关的关键信息。\n\n14. 通过问题生成进行文档增强以提升检索效果  \n   - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fdocument_augmentation.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fdocument_augmentation.ipynb)  \n   - **[可运行脚本](all_rag_techniques_runnable_scripts\u002Fdocument_augmentation.py)**\n\n   #### 概述 🔎\n   该实现展示了一种文本增强技术，通过额外的问题生成来改善向量数据库中的文档检索。通过对每个文本片段生成并纳入各种相关问题，系统能够优化标准检索流程，从而提高找到可用于生成式问答背景的相关文档的可能性。\n\n   #### 实现 🛠️\n   使用大语言模型为文本数据集生成与每份文档相关的所有可能问题，以进行扩充。\n\n\n\n### 🚀 高级检索方法\n\n15. 融合检索 🔗  \n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Ffusion_retrieval.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Ffusion_retrieval.ipynb)  \n    - **LlamaIndex**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Ffusion_retrieval_with_llamaindex.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Ffusion_retrieval_with_llamaindex.ipynb)\n    - **[可运行脚本](all_rag_techniques_runnable_scripts\u002Ffusion_retrieval.py)**\n \n    #### 概述 🔎\n    通过结合不同的检索方法来优化搜索结果。\n    \n    #### 实现 🛠️\n    将基于关键词的搜索与基于向量的搜索相结合，以实现更全面、更准确的检索。\n\n16. 智能重排序 📈  \n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Freranking.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Freranking.ipynb)  \n    - **LlamaIndex**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Freranking_with_llamaindex.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Freranking_with_llamaindex.ipynb)\n    - **[可运行脚本](all_rag_techniques_runnable_scripts\u002Freranking.py)**\n\n    #### 概述 🔎\n    应用先进的打分机制，以提升检索结果的相关性排序。\n\n    #### 实现方式 🛠️\n    - 🧠 **基于LLM的打分：** 使用语言模型对每个检索到的片段进行相关性评分。\n    - 🔀 **交叉编码器模型：** 将查询和检索到的文档联合重新编码，用于相似度打分。\n    - 🏆 **元数据增强排序：** 在打分过程中引入元数据，以实现更精细的排序。\n\n    #### 补充资源 📚\n    - **[相关性革命：重排序如何改变RAG系统](https:\u002F\u002Fopen.substack.com\u002Fpub\u002Fdiamantai\u002Fp\u002Frelevance-revolution-how-re-ranking?r=336pe4&utm_campaign=post&utm_medium=web)** - 一篇全面的博客文章，探讨重排序在提升RAG系统性能方面的强大作用。\n\n17. 多维度过滤 🔍\n\n    #### 概述 🔎\n    应用多种过滤技术，以精炼并提升检索结果的质量。\n\n    #### 实现方式 🛠️\n    - 🏷️ **元数据过滤：** 根据日期、来源、作者或文档类型等属性应用过滤条件。\n    - 📊 **相似度阈值：** 设置相关性得分阈值，仅保留最相关的结果。\n    - 📄 **内容过滤：** 移除不符合特定内容标准或关键关键词的结果。\n    - 🌈 **多样性过滤：** 通过过滤掉近似重复条目，确保结果的多样性。\n\n18. 层次化索引 🗂️  \n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fhierarchical_indices.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fhierarchical_indices.ipynb)  \n    - **[可运行脚本](all_rag_techniques_runnable_scripts\u002Fhierarchical_indices.py)**\n\n    #### 概述 🔎\n    构建多层级系统，以实现高效的信息导航与检索。\n\n    #### 实现方式 🛠️\n    实现一个由文档摘要和详细片段组成的双层系统，两者均包含指向数据中同一位置的元数据。\n\n    #### 补充资源 📚\n    - **[层次化索引：增强RAG系统](https:\u002F\u002Fopen.substack.com\u002Fpub\u002Fdiamantai\u002Fp\u002Fhierarchical-indices-enhancing-rag?r=336pe4&utm_campaign=post&utm_medium=web)** - 一篇全面的博客文章，探讨层次化索引在提升RAG系统性能方面的强大作用。\n\n19. 集成检索 🎭\n\n    #### 概述 🔎\n    结合多种检索模型或技术，以获得更稳健、更准确的结果。\n\n    #### 实现方式 🛠️\n    应用不同的嵌入模型或检索算法，并使用投票或加权机制来确定最终的检索文档集合。\n\n20. 飞镖盘检索 🎯\n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fdartboard.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fdartboard.ipynb) \n    #### 概述 🔎\n    优化检索中的相关信息增益\n\n    #### 实现方式 🛠️\n    - 将相关性和多样性结合为单一的打分函数，并直接对其进行优化。\n    - POC显示，在数据库较为密集的情况下，简单的RAG表现不佳，而飞镖盘检索则表现出色。\n\n21. 多模态检索 📽️\n\n    #### 概述 🔎\n    扩展RAG的能力，以处理多样化的数据类型，从而生成更丰富的响应。\n\n    #### 实现方式 🛠️\n    - **多模态RAG与多媒体字幕生成**： [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fmulti_model_rag_with_captioning.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fmulti_model_rag_with_captioning.ipynb) - 将PDF、PPT等其他多媒体数据连同文本数据一起进行字幕生成并存储到向量数据库中，随后一并检索。\n    - **多模态RAG与Colpali**： [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fmulti_model_rag_with_colpali.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fmulti_model_rag_with_colpali.ipynb) - 不再进行字幕生成，而是将所有数据转换为图像，然后找到最相关的图像，并将其传递给视觉大语言模型。\n\n### 🔁 迭代与自适应技术\n\n22. 带反馈循环的检索 🔁  \n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fretrieval_with_feedback_loop.ipynb) [\u003Cimg src=\"https6:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fretrieval_with_feedback_loop.ipynb)  \n    - **[可运行脚本](all_rag_techniques_runnable_scripts\u002Fretrieval_with_feedback_loop.py)**\n\n    #### 概述 🔎\n    实现从用户交互中学习并改进未来检索的机制。\n\n    #### 实现 🛠️\n    收集并利用用户对检索文档和生成回复的相关性和质量的反馈，以微调检索和排序模型。\n\n23. 自适应检索 🎯  \n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fadaptive_retrieval.ipynb) [\u003Cimg src=\"https6:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fadaptive_retrieval.ipynb)  \n    - **[可运行脚本](all_rag_techniques_runnable_scripts\u002Fadaptive_retrieval.py)**\n\n    #### 概述 🔎\n    根据查询类型和用户上下文动态调整检索策略。\n\n    #### 实现 🛠️\n    将查询分类为不同类别，并针对每类查询使用量身定制的检索策略，同时考虑用户上下文和偏好。\n\n24. 迭代式检索 🔄\n\n    #### 概述 🔎\n    通过多轮检索来优化和提升结果质量。\n\n    #### 实现 🛠️\n    使用LLM分析初始结果，并生成后续查询以填补空白或澄清信息。\n\n### 📊 评估\n\n25. **DeepEval 评估**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fevaluation\u002Fevaluation_deep_eval.ipynb) [\u003Cimg src=\"https6:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fevaluation\u002Fevaluation_deep_eval.ipynb) | 全面的RAG系统评估 |\n\n    #### 概述 🔎\n    对检索增强生成系统进行评估，涵盖多项指标并创建测试用例。\n\n    #### 实现 🛠️\n    使用`deepeval`库对RAG系统的正确性、忠实性和上下文相关性进行测试。\n\n26. **GroUSE 评估**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fevaluation\u002Fevaluation_grouse.ipynb) [\u003Cimg src=\"https6:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fevaluation\u002Fevaluation_grouse.ipynb) | 上下文相关的LLM评估 |\n\n    #### 概述 🔎\n    使用GroUSE框架的指标评估检索增强生成的最终阶段，并在GroUSE单元测试上对自定义的LLM评判器进行元评估。\n\n    #### 实现 🛠️\n    使用`grouse`包，借助GPT-4评估基于上下文的LLM生成结果，涵盖GroUSE框架的6项指标；同时利用单元测试评估自定义的Llama 3.1 405B评判器。\n\n### 🔬 可解释性与透明度\n\n27. 可解释的检索 🔍  \n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fexplainable_retrieval.ipynb) [\u003Cimg src=\"https6:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fexplainable_retrieval.ipynb)  \n    - **[可运行脚本](all_rag_techniques_runnable_scripts\u002Fexplainable_retrieval.py)**\n\n    #### 概述 🔎\n    提供检索过程的透明度，以增强用户信任并优化系统。\n\n    #### 实现 🛠️\n    解释为何检索到某些信息以及这些信息如何与查询相关联。\n\n### 🏗️ 高级架构\n\n28. 基于情境AI的代理式RAG 🤖 \n    - **代理式RAG**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FAgentic_RAG.ipynb) [\u003Cimg src=\"https6:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FAgentic_RAG.ipynb)  \n\n    #### 概述 🔎\n    利用Contextual AI的托管平台构建用于金融文档分析的生产级代理式RAG流水线。本综合教程展示了如何通过智能查询改写、文档解析、重排序和基于上下文的语言模型，利用代理式RAG解决复杂查询。\n\n    #### 实现 🛠️\n    - **文档解析器**: 采用视觉模型的企业级解析能力，适用于复杂表格、图表及多页文档\n    - **指令遵循型重排序器**: 具备指令遵循能力的SOTA重排序器，可处理冲突信息\n    - **基于上下文的语言模型 (GLM)**: 专为减少RAG场景中的幻觉而设计的世界最可靠的LLM\n    - **LMUnit**: 用于评估和优化RAG系统性能的自然语言单元测试框架\n\n29. 基于Milvus向量数据库的图RAG 🔍 \n    - **图RAG与Milvus**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fgraphrag_with_milvus_vectordb.ipynb) [\u003Cimg src=\"https6:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fgraphrag_with_milvus_vectordb.ipynb)\n\n#### 概述 🔎\n    一种简单而强大的方法，使用 Milvus 向量数据库实现图 RAG。该技术通过将基于关系的检索与向量搜索和重排序相结合，显著提升了对复杂多跳问题的性能。\n    \n    #### 实现 🛠️\n    - 将文本段落和关系三元组（主语-谓语-宾语）分别存储在不同的 Milvus 集合中\n    - 通过查询两个集合进行多路检索\n    - 使用 LLM 根据检索到的关系与查询的相关性对其进行重排序\n    - 根据最相关的关系检索最终的文本段落\n\n30. 知识图谱集成（Graph RAG） 🕸️  \n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fgraph_rag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fgraph_rag.ipynb)  \n    - **[可运行脚本](all_rag_techniques_runnable_scripts\u002Fgraph_rag.py)**\n\n    #### 概述 🔎\n    将知识图谱中的结构化数据融入其中，以丰富上下文并改进检索效果。\n    \n    #### 实现 🛠️\n    从知识图谱中检索与查询相关的实体及其关系，将这些结构化数据与非结构化文本结合，从而生成更具信息量的响应。\n\n31. GraphRag（微软） 🎯\n    - **GraphRag**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FMicrosoft_GraphRag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002FMicrosoft_GraphRag.ipynb)  \n\n    #### 概述 🔎\n    微软 GraphRAG（开源）是一种先进的 RAG 系统，通过集成知识图谱来提升 LLM 的性能。\n    \n    #### 实现 🛠️\n    • 分析输入语料，从文本单元中提取实体和关系；自下而上地为每个社区及其组成部分生成摘要。\n\n32. RAPTOR：面向树形组织检索的递归抽象处理 🌳  \n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fraptor.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fraptor.ipynb)  \n    - **[可运行脚本](all_rag_techniques_runnable_scripts\u002Fraptor.py)**\n\n    #### 概述 🔎\n    实现一种递归方法，将检索到的信息以树形结构进行处理和组织。\n    \n    #### 实现 🛠️\n    使用抽象摘要技术递归地处理和总结检索到的文档，将信息组织成树形结构，以提供分层的上下文。\n\n33. 自我 RAG 🔁  \n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fself_rag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fself_rag.ipynb)  \n    - **[可运行脚本](all_rag_techniques_runnable_scripts\u002Fself_rag.py)**\n\n    #### 概述 🔎\n    一种动态方法，结合基于检索和基于生成的方法，自适应地决定是否使用检索到的信息以及如何最佳利用这些信息来生成响应。\n    \n    #### 实现 🛠️\n    • 实施一个多步骤流程，包括检索决策、文档检索、相关性评估、响应生成、支持度评估和效用评估，以产生准确、相关且有用的结果。\n\n34. 纠正型 RAG 🔧  \n    - **LangChain**: [\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-View-blue\" height=\"20\">](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_TECHNIQUES\u002Fblob\u002Fmain\u002Fhttps:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcrag.ipynb) [\u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" height=\"20\">](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002FNirDiamant\u002FRAG_Techniques\u002Fblob\u002Fmain\u002Fall_rag_techniques\u002Fcrag.ipynb)  \n    - **[可运行脚本](all_rag_techniques_runnable_scripts\u002Fcrag.py)**\n\n    #### 概述 🔎\n    一种复杂的 RAG 方法，能够动态评估和纠正检索过程，结合向量数据库、网络搜索和语言模型，以生成高度准确且具有上下文感知能力的响应。\n    \n    #### 实现 🛠️\n    • 集成检索评估器、知识精炼模块、网络搜索查询改写器和响应生成器组件，构建一个系统，该系统会根据相关性得分调整其信息获取策略，并在必要时整合多个来源。\n\n\n\n## 🌟 特别高级技术 🌟\n\n35. **[用于复杂 RAG 任务的精密可控智能体 🤖](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FControllable-RAG-Agent)**\n\n    #### 概述 🔎\n    一种先进的 RAG 解决方案，旨在应对简单的语义相似度检索无法解决的复杂问题。该方法使用一个精密的确定性图作为高度可控自主智能体的“大脑” 🧠，能够根据您自己的数据回答非平凡的问题。\n    \n    #### 实现 🛠️\n    • 实施一个多步骤流程，包括问题匿名化、高层次规划、任务分解、自适应信息检索与问答、持续重新规划以及严格的答案验证，以确保生成有依据且准确的回应。\n\n\n\n## 开始使用\n\n要在您的项目中开始实施这些高级 RAG 技术：\n\n1. 克隆此仓库：\n   ```\n   git clone https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_Techniques.git\n   ```\n2. 导航到您感兴趣的技术目录：\n   ```\n   cd all_rag_techniques\u002Ftechnique-name\n   ```\n3. 按照各技术目录中的详细实施指南操作。\n\n## 贡献\n\n我们欢迎社区的贡献！如果您有新的技术或改进建议：\n\n1. 分支此仓库\n2. 创建您的功能分支：`git checkout -b feature\u002FAmazingFeature`\n3. 提交您的更改：`git commit -m 'Add some AmazingFeature'`\n4. 推送到分支：`git push origin feature\u002FAmazingFeature`\n5. 打开拉取请求\n\n## 贡献者\n\n[![贡献者](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FNirDiamant_RAG_Techniques_readme_31ed344e7518.png)](https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_Techniques\u002Fgraphs\u002Fcontributors)\n\n## 许可证\n\n本项目采用自定义的非商业许可协议授权——详情请参阅 [LICENSE](LICENSE) 文件。\n\n---\n\n⭐️ 如果您觉得这个仓库很有帮助，请考虑给它点个赞！\n\n![](https:\u002F\u002Feurope-west1-rag-techniques-views-tracker.cloudfunctions.net\u002Frag-techniques-tracker?notebook=main-readme)\n\n关键词：RAG，检索增强生成，NLP，AI，机器学习，信息检索，自然语言处理，LLM，嵌入，语义搜索","# RAG_Techniques 快速上手指南\n\nRAG_Techniques 是一个汇集了最先进检索增强生成（RAG）技术的开源项目，旨在通过多种高级策略提升 RAG 系统的准确性、效率和上下文丰富度。本指南将帮助你快速搭建环境并运行基础示例。\n\n## 环境准备\n\n在开始之前，请确保你的开发环境满足以下要求：\n\n*   **操作系统**: Linux, macOS 或 Windows (推荐 WSL2)\n*   **Python 版本**: Python 3.9 或更高版本\n*   **包管理工具**: pip 或 conda\n*   **API 密钥**: 根据你选择的具体技术（如调用 OpenAI、Anthropic 等模型），需提前准备相应的 API Key。\n\n**前置依赖建议**：\n由于本项目包含大量不同的 Notebook 教程，建议为每个实验或整体项目创建独立的虚拟环境，以避免依赖冲突。\n\n## 安装步骤\n\n### 1. 克隆仓库\n首先，从 GitHub 克隆项目到本地：\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_Techniques.git\ncd RAG_Techniques\n```\n\n### 2. 创建虚拟环境\n推荐使用 `venv` 或 `conda` 创建隔离环境：\n\n```bash\n# 使用 venv\npython -m venv rag_env\nsource rag_env\u002Fbin\u002Factivate  # Windows 用户请使用: rag_env\\Scripts\\activate\n\n# 或者使用 conda\nconda create -n rag_env python=3.10\nconda activate rag_env\n```\n\n### 3. 安装依赖\n项目中的每个 Notebook 可能有特定的依赖需求，但通常核心依赖可以通过以下方式安装。如果根目录没有统一的 `requirements.txt`，请参考具体 Notebook 单元格中的安装命令，或安装通用的 RAG 生态库：\n\n```bash\npip install langchain langchain-community langchain-openai faiss-cpu chromadb pypdf tiktoken notebook\n```\n\n> **提示**：针对国内网络环境，建议使用清华源或阿里源加速安装：\n> ```bash\n> pip install -i https:\u002F\u002Fpypi.tuna.tsinghua.edu.cn\u002Fsimple langchain langchain-community langchain-openai faiss-cpu chromadb pypdf tiktoken notebook\n> ```\n\n## 基本使用\n\n本项目以 Jupyter Notebook 形式提供了多种技术实现。以下是运行最基础的 **Simple RAG** 示例的步骤：\n\n### 1. 启动 Jupyter Notebook\n在项目根目录下启动服务：\n\n```bash\njupyter notebook\n```\n\n### 2. 打开示例文件\n在浏览器中导航至 `all_rag_techniques` 文件夹，选择并打开 `simple_rag.ipynb`。\n\n### 3. 配置 API 密钥\n在 Notebook 的第一个代码单元格中，设置你的大模型 API 密钥（以 OpenAI 为例）：\n\n```python\nimport os\nos.environ[\"OPENAI_API_KEY\"] = \"your-api-key-here\"\n```\n\n### 4. 运行单元格\n按顺序执行 Notebook 中的所有单元格。该示例将演示：\n1.  加载文档（通常是 PDF 或文本文件）。\n2.  将文档切分为块（Chunking）。\n3.  创建向量存储（Vector Store）。\n4.  构建检索器并生成回答。\n\n### 5. 尝试其他高级技术\n熟悉基础流程后，你可以探索其他高级技术笔记本，例如：\n*   **查询增强**: `query_transformations.ipynb`, `HyDe_Hypothetical_Document_Embedding.ipynb`\n*   **上下文优化**: `contextual_chunk_headers.ipynb`\n*   **代理 RAG**: `Agentic_RAG.ipynb`\n\n只需打开对应的 `.ipynb` 文件，按照单元格说明配置参数并运行即可。","某大型金融科技公司的技术团队正在构建内部智能合规助手，旨在帮助员工快速从数千页不断更新的法律文档和内部审计报告中检索准确条款并生成解读。\n\n### 没有 RAG_Techniques 时\n- **检索精度低**：传统的向量检索仅依赖语义相似度，常返回看似相关但实际过期的旧版法规，导致模型基于错误信息生成建议。\n- **上下文丢失**：面对长篇复杂的法律条文，简单的切片策略切断了关键的前后文逻辑，模型无法理解特定条款的适用前提。\n- **多跳推理失败**：当问题需要跨文档关联（如“结合 A 报告的风险点与 B 法规的处罚标准”）时，系统只能抓取单一片段，无法串联线索。\n- **幻觉频发**：由于缺乏有效的引用验证机制，模型在信息缺失时倾向于编造看似合理的虚假条款，带来严重的合规风险。\n\n### 使用 RAG_Techniques 后\n- **混合检索增强**：利用仓库中的高级混合检索策略，结合关键词匹配与向量搜索，精准锁定最新生效的法规版本，过滤过时干扰项。\n- **智能分块优化**：应用父子索引（Parent-Child Indexing）等技术，在保留细粒度检索的同时自动还原完整段落上下文，确保模型理解条款全貌。\n- **多步推理链支持**：通过部署图谱增强或迭代检索技巧，系统能自动拆解复杂问题，跨多个文档提取关联信息并综合推导结论。\n- **可信引用溯源**：引入重排序（Re-ranking）与严格的答案生成约束，强制模型仅依据检索到的高置信度片段作答，并附带精确出处，大幅降低幻觉。\n\nRAG_Techniques 通过集成前沿的检索与生成优化策略，将原本不可靠的内部问答系统转化为精准、可追溯且具备深度推理能力的企业级合规专家。","https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FNirDiamant_RAG_Techniques_b9086042.png","NirDiamant",null,"https:\u002F\u002Foss.gittoolsai.com\u002Favatars\u002FNirDiamant_846222c2.jpg","GenAI Consultant | Open Source Hub | Upwork: upwork.com\u002Ffreelancers\u002F~016cfe264764ea59ba","DiamantAI","NirDiamantAI","https:\u002F\u002Fdiamantai.substack.com","https:\u002F\u002Fgithub.com\u002FNirDiamant",[81,85],{"name":82,"color":83,"percentage":84},"Jupyter Notebook","#DA5B0B",94.9,{"name":86,"color":87,"percentage":88},"Python","#3572A5",5.1,26505,3177,"2026-04-06T22:38:48","NOASSERTION","未说明",{"notes":95,"python":93,"dependencies":96},"README 内容主要介绍了项目背景、赞助信息、社区链接以及包含的各种 RAG 技术教程列表（如 Agentic RAG, Basic RAG, HyDE 等）。提供的文本片段中未包含具体的安装指南、环境配置要求、依赖库列表或硬件需求说明。这些技术以 Jupyter Notebook (.ipynb) 形式提供，并支持在 Google Colab 上运行，具体依赖可能因各个 Notebook 的具体实现而异，需查看具体文件内容。",[93],[15,35,14,13],[99,100,101,102,103,104,105,106,107],"rag","tutorials","langchain","llama-index","llms","opeani","python","ai","llm","2026-03-27T02:49:30.150509","2026-04-07T11:44:37.640765",[111,116,121,126,131,136,141],{"id":112,"question_zh":113,"answer_zh":114,"source_url":115},21848,"运行代码时提示缺少依赖包（如 sklearn），requirements.txt 中没有包含所有需要的包，该怎么办？","维护者指出，由于不同技术教程所需的依赖项差异较大，为了避免安装不必要的包，未来的计划是将依赖项按技术分离，而不是在一个文件中安装所有内容。目前建议用户只针对“技术无法运行”的问题提交 Issue，算法类问题请前往项目的 Discord 频道讨论。如果遇到缺失包的情况，可能需要根据具体运行的 Notebook 手动安装缺失的库（例如 sklearn）。","https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_Techniques\u002Fissues\u002F73",{"id":117,"question_zh":118,"answer_zh":119,"source_url":120},21849,"无法克隆资源中提到的仓库（报错地址不可用），正确的仓库地址是什么？","这是一个文档链接错误。正确的仓库地址是 https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_Techniques.git。如果在其他资源中看到指向 `N7\u002FRAG_TECHNIQUES` 的旧链接，请替换为上述正确地址。该问题已被修复。","https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_Techniques\u002Fissues\u002F100",{"id":122,"question_zh":123,"answer_zh":124,"source_url":125},21850,"运行 Fusion Retrieval 时出现 'RuntimeWarning: invalid value encountered in divide' 警告，如何解决？","该警告是由于归一化计算中分母可能为零导致的。解决方案是在分母中添加一个极小值（epsilon）以防止除以零。修改代码如下：\n\n```python\nepsilon = 1e-8\nvector_scores = 1 - (vector_scores - np.min(vector_scores)) \u002F (np.max(vector_scores) - np.min(vector_scores) + epsilon)\nbm25_scores = (bm25_scores - np.min(bm25_scores)) \u002F (np.max(bm25_scores) - np.min(bm25_scores) + epsilon)\n```\n维护者已合并了包含此修复的 PR。","https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_Techniques\u002Fissues\u002F68",{"id":127,"question_zh":128,"answer_zh":129,"source_url":130},21851,"运行 simple RAG 系统时出现 'BaseRetriever.get_relevant_documents is deprecated' 弃用警告，该如何更新代码？","该警告表明 `BaseRetriever.get_relevant_documents` 方法已在 langchain-core 0.1.46 中被弃用，并将在 0.3.0 中移除。解决方法是将代码中的调用方式从 `.get_relevant_documents()` 更改为使用 `.invoke()` 方法。例如：\n\n原代码：`retriever.get_relevant_documents(query)`\n新代码：`retriever.invoke(query)`\n\n这将消除警告并确保与未来版本的 LangChain 兼容。","https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_Techniques\u002Fissues\u002F99",{"id":132,"question_zh":133,"answer_zh":134,"source_url":135},21852,"在 reliable_rag.ipynb 中运行 PydanticOutputParser 时报错 'AttributeError: ... has no attribute model_json_schema'，如何修复？","该错误是因为使用了 `langchain_core.pydantic_v1` 导入 BaseModel，但该模块不支持 Pydantic v2 的 `model_json_schema` 方法。修复方法是修改导入语句，直接使用标准的 `pydantic` 库。请将代码中的：\n\n```python\nfrom langchain_core.pydantic_v1 import BaseModel, Field\n```\n\n更改为：\n\n```python\nfrom pydantic import BaseModel, Field\n```\n\n这适用于较新版本的 Pydantic (v2+) 和 LangChain。","https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_Techniques\u002Fissues\u002F75",{"id":137,"question_zh":138,"answer_zh":139,"source_url":140},21853,"运行 evaluate_rag.py 时找不到 helper_functions.py 文件，提示路径错误，如何解决？","这是由于脚本中尝试追加当前目录上一级目录的路径逻辑未按预期工作导致的。维护者已修复了该路径查找逻辑。如果您遇到类似问题，请确保拉取了最新的代码版本，或者检查脚本中关于 `sys.path` 的添加逻辑，确保其正确指向了包含 `helper_functions.py` 的目录。","https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_Techniques\u002Fissues\u002F49",{"id":142,"question_zh":143,"answer_zh":144,"source_url":145},21854,"在使用 llama_index 创建向量索引时，遇到 'ServiceContext' 相关的错误或弃用提示，应该如何更新代码？","llama_index 库已将 `ServiceContext` 更新为 `Settings`。如果在 `choose_chink_size.ipynb` 等文件中遇到相关错误，需要将旧的 `ServiceContext.from_defaults(...)` 用法迁移到新的 `Settings` 配置模式。通常涉及将 `service_context` 参数替换为全局 `Settings` 配置或新的初始化方式。建议查阅 llama_index 最新文档以获取具体的迁移语法，因为这是库本身的重大版本更新。","https:\u002F\u002Fgithub.com\u002FNirDiamant\u002FRAG_Techniques\u002Fissues\u002F39",[]]