[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"similar-imoneoi--openchat":3,"tool-imoneoi--openchat":64},[4,17,27,35,48,56],{"id":5,"name":6,"github_repo":7,"description_zh":8,"stars":9,"difficulty_score":10,"last_commit_at":11,"category_tags":12,"status":16},1381,"everything-claude-code","affaan-m\u002Feverything-claude-code","everything-claude-code 是一套专为 AI 编程助手（如 Claude Code、Codex、Cursor 等）打造的高性能优化系统。它不仅仅是一组配置文件，而是一个经过长期实战打磨的完整框架，旨在解决 AI 代理在实际开发中面临的效率低下、记忆丢失、安全隐患及缺乏持续学习能力等核心痛点。\n\n通过引入技能模块化、直觉增强、记忆持久化机制以及内置的安全扫描功能，everything-claude-code 能显著提升 AI 在复杂任务中的表现，帮助开发者构建更稳定、更智能的生产级 AI 代理。其独特的“研究优先”开发理念和针对 Token 消耗的优化策略，使得模型响应更快、成本更低，同时有效防御潜在的攻击向量。\n\n这套工具特别适合软件开发者、AI 研究人员以及希望深度定制 AI 工作流的技术团队使用。无论您是在构建大型代码库，还是需要 AI 协助进行安全审计与自动化测试，everything-claude-code 都能提供强大的底层支持。作为一个曾荣获 Anthropic 黑客大奖的开源项目，它融合了多语言支持与丰富的实战钩子（hooks），让 AI 真正成长为懂上",141543,2,"2026-04-06T11:32:54",[13,14,15],"开发框架","Agent","语言模型","ready",{"id":18,"name":19,"github_repo":20,"description_zh":21,"stars":22,"difficulty_score":23,"last_commit_at":24,"category_tags":25,"status":16},4487,"LLMs-from-scratch","rasbt\u002FLLMs-from-scratch","LLMs-from-scratch 是一个基于 PyTorch 的开源教育项目，旨在引导用户从零开始一步步构建一个类似 ChatGPT 的大型语言模型（LLM）。它不仅是同名技术著作的官方代码库，更提供了一套完整的实践方案，涵盖模型开发、预训练及微调的全过程。\n\n该项目主要解决了大模型领域“黑盒化”的学习痛点。许多开发者虽能调用现成模型，却难以深入理解其内部架构与训练机制。通过亲手编写每一行核心代码，用户能够透彻掌握 Transformer 架构、注意力机制等关键原理，从而真正理解大模型是如何“思考”的。此外，项目还包含了加载大型预训练权重进行微调的代码，帮助用户将理论知识延伸至实际应用。\n\nLLMs-from-scratch 特别适合希望深入底层原理的 AI 开发者、研究人员以及计算机专业的学生。对于不满足于仅使用 API，而是渴望探究模型构建细节的技术人员而言，这是极佳的学习资源。其独特的技术亮点在于“循序渐进”的教学设计：将复杂的系统工程拆解为清晰的步骤，配合详细的图表与示例，让构建一个虽小但功能完备的大模型变得触手可及。无论你是想夯实理论基础，还是为未来研发更大规模的模型做准备",90106,3,"2026-04-06T11:19:32",[15,26,14,13],"图像",{"id":28,"name":29,"github_repo":30,"description_zh":31,"stars":32,"difficulty_score":10,"last_commit_at":33,"category_tags":34,"status":16},3704,"NextChat","ChatGPTNextWeb\u002FNextChat","NextChat 是一款轻量且极速的 AI 助手，旨在为用户提供流畅、跨平台的大模型交互体验。它完美解决了用户在多设备间切换时难以保持对话连续性，以及面对众多 AI 模型不知如何统一管理的痛点。无论是日常办公、学习辅助还是创意激发，NextChat 都能让用户随时随地通过网页、iOS、Android、Windows、MacOS 或 Linux 端无缝接入智能服务。\n\n这款工具非常适合普通用户、学生、职场人士以及需要私有化部署的企业团队使用。对于开发者而言，它也提供了便捷的自托管方案，支持一键部署到 Vercel 或 Zeabur 等平台。\n\nNextChat 的核心亮点在于其广泛的模型兼容性，原生支持 Claude、DeepSeek、GPT-4 及 Gemini Pro 等主流大模型，让用户在一个界面即可自由切换不同 AI 能力。此外，它还率先支持 MCP（Model Context Protocol）协议，增强了上下文处理能力。针对企业用户，NextChat 提供专业版解决方案，具备品牌定制、细粒度权限控制、内部知识库整合及安全审计等功能，满足公司对数据隐私和个性化管理的高标准要求。",87618,"2026-04-05T07:20:52",[13,15],{"id":36,"name":37,"github_repo":38,"description_zh":39,"stars":40,"difficulty_score":10,"last_commit_at":41,"category_tags":42,"status":16},2268,"ML-For-Beginners","microsoft\u002FML-For-Beginners","ML-For-Beginners 是由微软推出的一套系统化机器学习入门课程，旨在帮助零基础用户轻松掌握经典机器学习知识。这套课程将学习路径规划为 12 周，包含 26 节精炼课程和 52 道配套测验，内容涵盖从基础概念到实际应用的完整流程，有效解决了初学者面对庞大知识体系时无从下手、缺乏结构化指导的痛点。\n\n无论是希望转型的开发者、需要补充算法背景的研究人员，还是对人工智能充满好奇的普通爱好者，都能从中受益。课程不仅提供了清晰的理论讲解，还强调动手实践，让用户在循序渐进中建立扎实的技能基础。其独特的亮点在于强大的多语言支持，通过自动化机制提供了包括简体中文在内的 50 多种语言版本，极大地降低了全球不同背景用户的学习门槛。此外，项目采用开源协作模式，社区活跃且内容持续更新，确保学习者能获取前沿且准确的技术资讯。如果你正寻找一条清晰、友好且专业的机器学习入门之路，ML-For-Beginners 将是理想的起点。",85013,"2026-04-06T11:09:19",[26,43,44,45,14,46,15,13,47],"数据工具","视频","插件","其他","音频",{"id":49,"name":50,"github_repo":51,"description_zh":52,"stars":53,"difficulty_score":23,"last_commit_at":54,"category_tags":55,"status":16},3128,"ragflow","infiniflow\u002Fragflow","RAGFlow 是一款领先的开源检索增强生成（RAG）引擎，旨在为大语言模型构建更精准、可靠的上下文层。它巧妙地将前沿的 RAG 技术与智能体（Agent）能力相结合，不仅支持从各类文档中高效提取知识，还能让模型基于这些知识进行逻辑推理和任务执行。\n\n在大模型应用中，幻觉问题和知识滞后是常见痛点。RAGFlow 通过深度解析复杂文档结构（如表格、图表及混合排版），显著提升了信息检索的准确度，从而有效减少模型“胡编乱造”的现象，确保回答既有据可依又具备时效性。其内置的智能体机制更进一步，使系统不仅能回答问题，还能自主规划步骤解决复杂问题。\n\n这款工具特别适合开发者、企业技术团队以及 AI 研究人员使用。无论是希望快速搭建私有知识库问答系统，还是致力于探索大模型在垂直领域落地的创新者，都能从中受益。RAGFlow 提供了可视化的工作流编排界面和灵活的 API 接口，既降低了非算法背景用户的上手门槛，也满足了专业开发者对系统深度定制的需求。作为基于 Apache 2.0 协议开源的项目，它正成为连接通用大模型与行业专有知识之间的重要桥梁。",77062,"2026-04-04T04:44:48",[14,26,13,15,46],{"id":57,"name":58,"github_repo":59,"description_zh":60,"stars":61,"difficulty_score":23,"last_commit_at":62,"category_tags":63,"status":16},519,"PaddleOCR","PaddlePaddle\u002FPaddleOCR","PaddleOCR 是一款基于百度飞桨框架开发的高性能开源光学字符识别工具包。它的核心能力是将图片、PDF 等文档中的文字提取出来，转换成计算机可读取的结构化数据，让机器真正“看懂”图文内容。\n\n面对海量纸质或电子文档，PaddleOCR 解决了人工录入效率低、数字化成本高的问题。尤其在人工智能领域，它扮演着连接图像与大型语言模型（LLM）的桥梁角色，能将视觉信息直接转化为文本输入，助力智能问答、文档分析等应用场景落地。\n\nPaddleOCR 适合开发者、算法研究人员以及有文档自动化需求的普通用户。其技术优势十分明显：不仅支持全球 100 多种语言的识别，还能在 Windows、Linux、macOS 等多个系统上运行，并灵活适配 CPU、GPU、NPU 等各类硬件。作为一个轻量级且社区活跃的开源项目，PaddleOCR 既能满足快速集成的需求，也能支撑前沿的视觉语言研究，是处理文字识别任务的理想选择。",74991,"2026-04-06T23:16:49",[15,26,13,46],{"id":65,"github_repo":66,"name":67,"description_en":68,"description_zh":69,"ai_summary_zh":69,"readme_en":70,"readme_zh":71,"quickstart_zh":72,"use_case_zh":73,"hero_image_url":74,"owner_login":75,"owner_name":76,"owner_avatar_url":77,"owner_bio":78,"owner_company":79,"owner_location":79,"owner_email":79,"owner_twitter":79,"owner_website":80,"owner_url":81,"languages":82,"stars":99,"forks":100,"last_commit_at":101,"license":102,"difficulty_score":23,"env_os":103,"env_gpu":104,"env_ram":105,"env_deps":106,"category_tags":113,"github_topics":114,"view_count":10,"oss_zip_url":79,"oss_zip_packed_at":79,"status":16,"created_at":118,"updated_at":119,"faqs":120,"releases":150},4669,"imoneoi\u002Fopenchat","openchat","OpenChat: Advancing Open-source Language Models with Imperfect Data","OpenChat 是一个致力于推动开源语言模型发展的创新项目，旨在利用混合质量的数据训练出高性能的对话模型。它主要解决了传统大模型训练过度依赖昂贵且稀缺的高质量标注数据这一难题，通过独特的 C-RLFT（条件强化学习微调）策略，让模型能够直接从无需偏好标签的混杂数据中高效学习。\n\n得益于这一技术突破，OpenChat 在仅拥有 70 亿至 80 亿参数量的情况下，便能展现出媲美 ChatGPT 的卓越性能，甚至在代码生成、数学推理及综合基准测试中实现超越。其显著优势在于极高的部署效率，普通用户仅需一张消费级显卡（如 RTX 3090）即可在本地流畅运行，大大降低了使用门槛。此外，该项目保持快速迭代，最新推出的基于 Llama 3 架构的版本进一步提升了整体表现，并明确支持商业免费使用。\n\nOpenChat 非常适合希望低成本私有化部署大模型的开发者、需要高效基座模型进行二次研究的研究人员，以及追求高性能但受限于硬件资源的普通技术爱好者。无论是构建智能客服、辅助编程工具，还是探索大模型应用边界，OpenChat 都提供了一个强大且易用的开源选择。","# OpenChat: Advancing Open-source Language Models with Mixed-Quality Data\n\n\u003Cdiv align=\"center\">\n  \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fimoneoi_openchat_readme_2508a546dfb3.png\" style=\"width: 65%\">\n\u003C\u002Fdiv>\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Fopenchat.team\">💻Online Demo\u003C\u002Fa> |\n  \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fopenchat\">🤗Huggingface\u003C\u002Fa> |\n  \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F2309.11235.pdf\">📃Paper\u003C\u002Fa> |\n  \u003Ca href=\"https:\u002F\u002Fdiscord.gg\u002FpQjnXvNKHY\">💭Discord\u003C\u002Fa> \n\u003C\u002Fp>\n\n- OpenChat is an innovative library of **open-source language models**, fine-tuned with [**C-RLFT**](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2309.11235.pdf) - a strategy inspired by offline reinforcement learning.\n- Our models learn from mixed-quality data without preference labels, delivering exceptional performance on par with `ChatGPT`, even with a `7B` model which can be run on a **consumer GPU (e.g. RTX 3090)**.\n- Despite our simple approach, we are committed to developing a high-performance, commercially viable, open-source large language model, and we continue to make significant strides toward this vision.\n\n[![DOI](https:\u002F\u002Fzenodo.org\u002Fbadge\u002F645397533.svg)](https:\u002F\u002Fzenodo.org\u002Fbadge\u002Flatestdoi\u002F645397533)\n\n# ✨ News\n\n - [2024\u002F05\u002F22] We released the Llama-3 based version [OpenChat 3.6 20240522](https:\u002F\u002Fhuggingface.co\u002Fopenchat\u002Fopenchat-3.6-8b-20240522), outperforming official Llama 3 8B Instruct and open-source finetunes\u002Fmerges.\n\n- [2024\u002F01\u002F06] We released the second update, [OpenChat 3.5 0106](openchat\u002Fopenchat-3.5-0106), further improved coding and overall performance 🏆.\n\n- [2023\u002F12\u002F10] We released the first update, [OpenChat 3.5 1210](openchat\u002Fopenchat-3.5-1210), improved coding by 15 points 🚀.\n\n- [2023\u002F11\u002F01] We released the [OpenChat-3.5-7B](https:\u002F\u002Fhuggingface.co\u002Fopenchat\u002Fopenchat_3.5) model, surpassing ChatGPT on various benchmarks 🔥.\n\n- [2023\u002F09\u002F21] We released our paper [OpenChat: Advancing Open-source Language Models with Mixed-Quality Data](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2309.11235.pdf).\n  \n\u003Cdetails>\n  \u003Csummary>Read more\u003C\u002Fsummary>\n  \n- [2023\u002F09\u002F03] We released the [OpenChat V3.2 SUPER]([#models](https:\u002F\u002Fhuggingface.co\u002Fopenchat\u002Fopenchat_v3.2_super)) model.\n\n- [2023\u002F08\u002F04] We have launched an [Online Demo](https:\u002F\u002Fopenchat.team) featuring the latest version, OpenChat 3.2.\n\n- [2023\u002F07\u002F30] We are thrilled to introduce the [OpenChat V3 model series](#models), based on Llama 2, and now available for free for commercial use!\n\n- [2023\u002F07\u002F07] We released the [OpenChat V2 model series](#legacy-models).\n\n- [2023\u002F07\u002F01] We released the [OpenChat V1 model series](#legacy-models).\n\u003C\u002Fdetails>\n\n# 🏷️ Benchmarks - OpenChat 3.6\n\n\u003Cdiv align=\"center\">\n  \u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002Fimoneoi\u002Fopenchat\u002Fmaster\u002Fassets\u002Fbenchmarks-openchat-3.6-20240522.svg\" style=\"width: 95%;\">\n\u003C\u002Fdiv>\n\n\u003Cdetails>\n  \u003Csummary>Reproducing benchmarks\u003C\u002Fsummary>\n\nNote: Please run the following commands at the base directory of this repository.\n\n```bash\npython -m ochat.evaluation.run_eval --condition \"GPT4 Correct\" --model openchat\u002Fopenchat-3.6-8b-20240522 --eval_sets fs_cothub\u002Fmmlu fs_cothub\u002Fgsm8k fs_cothub\u002Fmath\npython -m ochat.evaluation.run_eval --condition \"GPT4\" --model openchat\u002Fopenchat-3.6-8b-20240522 --eval_sets zs\u002Fgpqa\n```\n\nHumanEval is run using the official [EvalPlus repository](https:\u002F\u002Fgithub.com\u002Fevalplus\u002Fevalplus).\n\u003C\u002Fdetails>\n\n# 🏷️ Benchmarks - OpenChat 3.5\n\n| Model                 | # Params | Average  | MT-Bench     | HumanEval       | BBH MC   | AGIEval  | TruthfulQA    | MMLU         | GSM8K        | BBH CoT     |\n|-----------------------|----------|----------|--------------|-----------------|----------|----------|---------------|--------------|--------------|-------------|\n| **OpenChat-3.5-0106** | **7B**   | **64.5** | 7.8          | **71.3**        | **51.5** | **49.1** | 61.0          | 65.8         | **77.4**     | 62.2        |\n| ChatGPT (March)*      | ???B     | 61.5     | **7.94**     | 48.1            | 47.6     | 47.1     | 57.7          | **67.3**     | 74.9         | **70.1**    |\n|                       |          |          |              |                 |          |          |               |              |              |             |\n| OpenHermes 2.5        | 7B       | 59.3     | 7.54         | 48.2            | 49.4     | 46.5     | 57.5          | 63.8         | 73.5         | 59.9        |\n| OpenOrca Mistral      | 7B       | 52.7     | 6.86         | 38.4            | 49.4     | 42.9     | 45.9          | 59.3         | 59.1         | 58.1        |\n| Zephyr-β^             | 7B       | 34.6     | 7.34         | 22.0            | 40.6     | 39.0     | 40.8          | 39.8         | 5.1          | 16.0        |\n| Mistral               | 7B       | -        | 6.84         | 30.5            | 39.0     | 38.0     | -             | 60.1         | 52.2         | -           |\n| Open-source SOTA**    | 13B-70B  | 61.4     | 7.71         | 73.2            | 49.7     | 41.7     | 62.3          | 63.7         | 82.3         | 41.4        |\n|                       |          |          | WizardLM 70B | WizardCoder 34B | Orca 13B | Orca 13B | Platypus2 70B | WizardLM 70B | MetaMath 70B | Flan-T5 11B |\n\n🔥 OpenChat-3.5-0106 (7B) now outperforms Grok-0 (33B) on **all 4 benchmarks** and Grok-1 (314B) on average and **3\u002F4 benchmarks**.\n\n|                       | License     | # Param | Average  | MMLU   | HumanEval | MATH     | GSM8k    |\n|-----------------------|-------------|---------|----------|--------|-----------|----------|----------|\n| **OpenChat-3.5-0106** | Apache-2.0  | **7B**  | **61.0** | 65.8   | **71.3**  | **29.3** | **77.4** |\n| Grok-0                | Proprietary | 33B     | 44.5     | 65.7   | 39.7      | 15.7     | 56.8     |\n| Grok-1                | Proprietary | 314B    | 55.8     | **73** | 63.2      | 23.9     | 62.9     |\n\n\u003Cdetails>\n  \u003Csummary>Evaluation details\u003C\u002Fsummary>\n*: ChatGPT (March) results are from GPT-4 Technical Report, Chain-of-Thought Hub, and our evaluation.\n\n^: Zephyr-β often fails to follow few-shot CoT instructions, likely because it was aligned with only chat data but not trained on few-shot data.\n\n **: Mistral and Open-source SOTA results are taken from reported results in instruction-tuned model papers and official repositories.\n\nAll models are evaluated in chat mode (e.g. with the respective conversation template applied). All zero-shot benchmarks follow the same setting as in the AGIEval paper and Orca paper. CoT tasks use the same configuration as Chain-of-Thought Hub, HumanEval is evaluated with EvalPlus, and MT-bench is run using FastChat. To reproduce our results, follow the instructions below.\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n  \u003Csummary>Reproducing benchmarks\u003C\u002Fsummary>\n\nReasoning and Coding:\n\nNote: Please run the following commands at the base directory of this repository.\n\n```bash\npython -m ochat.evaluation.run_eval --condition \"GPT4 Correct\" --model openchat\u002Fopenchat-3.5-0106 --eval_sets coding fs_cothub\u002Fbbh fs_cothub\u002Fmmlu zs\u002Fagieval zs\u002Fbbh_mc_orca zs\u002Ftruthfulqa_orca\npython ochat\u002Fevaluation\u002Fview_results.py\npython ochat\u002Fevaluation\u002Fconvert_to_evalplus.py\n```\n\nThen all humaneval code samples are placed in `ochat\u002Fevaluation\u002Fevalplus_codegen`. Use the following command to evaluate an individual code sample named `samples.jsonl` using Docker as a sandbox.\n\n```bash\ndocker run -v $(pwd):\u002Fapp ganler\u002Fevalplus:latest --dataset humaneval --samples samples.jsonl\n```\n\nMathematical Reasoning:\n\nNote: Please run the following commands at the base directory of this repository.\n\n```bash\npython -m ochat.evaluation.run_eval --condition \"Math Correct\" --model openchat\u002Fopenchat-3.5-0106 --eval_sets fs_cothub\u002Fgsm8k zs\u002Fmath\npython ochat\u002Fevaluation\u002Fview_results.py\n```\n\nMT-Bench:\n\nPlease first launch a local API server, then download FastChat and run the following commands.\n\nNote: Due to non-zero temperature and GPT-4 API changes over time, there might be variations in the results.\n\n```bash\ncd fastchat\u002Fllm_judge\npython gen_api_answer.py --model openchat-3.5-0106 --max-tokens 4096 --parallel 128 --openai-api-base http:\u002F\u002Flocalhost:18888\u002Fv1\npython gen_judgment.py --model-list openchat-3.5-0106 --parallel 8 --mode single\n```\n\n\u003C\u002Fdetails>\n\n# ⬇️ Installation\n> [!NOTE]\n> Need [`pytorch`](https:\u002F\u002Fpytorch.org\u002Fget-started\u002Flocally\u002F#start-locally) and [CUDA](https:\u002F\u002Fdeveloper.nvidia.com\u002Fcuda-toolkit-archive) to run OpenChat\n\n## pip\n\n```bash\npip3 install ochat\n```\n> [!IMPORTANT]\n> If you are facing package compatibility issues with pip, try the conda method below or check [this issue](https:\u002F\u002Fgithub.com\u002Fimoneoi\u002Fopenchat\u002Fissues\u002F41)\n\n## conda\n\n```bash\nconda create -y --name openchat python=3.11\nconda activate openchat\n\npip3 install ochat\n```\n\n## Windows (WSL 1.x, Ubuntu-22.04)\n\n```bash\nsudo apt update\nsudo apt install build-essential\n\nsudo apt install -y curl\ncurl -o miniconda.sh https:\u002F\u002Frepo.anaconda.com\u002Fminiconda\u002FMiniconda3-latest-Linux-x86_64.sh\nbash miniconda.sh\n\n# Restart WSL terminal if the following conda command does not work\n\nconda create -y --name openchat python=3.11\nconda activate openchat\n\npip3 install ochat\n```\n\n## From source\n\n\u003Cdetails>\n  \u003Csummary>Clone this repo and install openchat from source in editable mode\u003C\u002Fsummary>\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002Fimoneoi\u002Fopenchat\ncd openchat\n\npip3 install --upgrade pip  # enable PEP 660 support\npip3 install -e .  # Editable mode, you can make changes in this cloned repo\n```\n\u003C\u002Fdetails>\n\n# 🚀 Deploying API server\n\n⚡ Our API server is ready for production use and compatible with the OpenAI API protocol. It is highly optimized with vLLM and can dynamically batch requests.\n\n📎 Note: For 20 series or older GPUs that do not support `bfloat16`, add `--dtype float16` to the server args.\n\n### List of currently supported models\n\n| MODEL_TYPE   | MODEL_REPO                                                                                    | License    |\n|--------------|-----------------------------------------------------------------------------------------------|------------|\n| openchat_3.6 | [openchat\u002Fopenchat-3.6-8b-20240522](https:\u002F\u002Fhuggingface.co\u002Fopenchat\u002Fopenchat-3.6-8b-20240522) | Llama 3    |\n| openchat_3.5 | [openchat\u002Fopenchat-3.5-0106](https:\u002F\u002Fhuggingface.co\u002Fopenchat\u002Fopenchat-3.5-0106)               | Apache 2.0 |\n\n### For a single GPU (e.g. RTX 3090, 4090)\n\n```bash\npython -m ochat.serving.openai_api_server --model MODEL_REPO\n```\n\n### For multiple GPUs (tensor parallel)\n\n```bash\n# N is the number of tensor parallel GPUs\npython -m ochat.serving.openai_api_server --model MODEL_REPO --engine-use-ray --worker-use-ray --tensor-parallel-size N\n```\n\nuse `-h` to see more settings\n```bash\npython -m ochat.serving.openai_api_server --model MODEL_REPO -h\n```\n\n\u003Cdetails>\n  \u003Csummary>Deploy as online service\u003C\u002Fsummary>\n\nIf you want to deploy the server as an online service, you can use `--api-keys sk-KEY1 sk-KEY2 ...` to specify allowed API keys and `--disable-log-requests --disable-log-stats --log-file openchat.log` for logging only to a file. For security purposes, we recommend using an [HTTPS gateway](https:\u002F\u002Ffastapi.tiangolo.com\u002Fes\u002Fdeployment\u002Fconcepts\u002F#security-https) in front of the server.\n\n\u003C\u002Fdetails>\n\n## Request example\n\nOnce started, the server listens at `localhost:18888` for requests and is compatible with the [OpenAI ChatCompletion API specifications](https:\u002F\u002Fplatform.openai.com\u002Fdocs\u002Fapi-reference\u002Fchat). \n\n💡 **Default Mode (GPT4 Correct)**: Best for coding, chat and general tasks\n\n```bash\ncurl http:\u002F\u002Flocalhost:18888\u002Fv1\u002Fchat\u002Fcompletions \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\n    \"model\": \"MODEL_TYPE\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"You are a large language model named OpenChat. Write a poem to describe yourself\"}]\n  }'\n```\n\n🧮 **Mathematical Reasoning Mode**: Tailored for solving math problems\n\n```bash\ncurl http:\u002F\u002Flocalhost:18888\u002Fv1\u002Fchat\u002Fcompletions \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\n    \"model\": \"MODEL_TYPE\",\n    \"condition\": \"Math Correct\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"10.3 − 7988.8133 = \"}]\n  }'\n```\n\n# \u003Ca id=\"web-ui\">\u003C\u002Fa> 🌐 Web UI - [OpenChat-UI](https:\u002F\u002Fgithub.com\u002Fimoneoi\u002Fopenchat-ui)\n\nAfter launching the API server, OpenChat provide user interface that easy to interact with. [Click here to check Web UI](https:\u002F\u002Fgithub.com\u002Fimoneoi\u002Fopenchat-ui)\n\n# 🤗 Inference with Transformers\n\n> [!WARNING]\n> It's recommended to use our optimized API server for deployment. Inferencing with Transformers will be slower.\n\n💡 **Default Mode (GPT4 Correct)**: Best for coding, chat and general tasks\n\n```\nGPT4 Correct User: Hello\u003C|end_of_turn|>GPT4 Correct Assistant: Hi\u003C|end_of_turn|>GPT4 Correct User: How are you today?\u003C|end_of_turn|>GPT4 Correct Assistant:\n```\n\n🧮 **Mathematical Reasoning Mode**: Tailored for solving math problems\n\n```\nMath Correct User: 10.3 − 7988.8133=\u003C|end_of_turn|>Math Correct Assistant:\n```\n\n⚠️ **Notice:** Remember to set `\u003C|end_of_turn|>` as end of generation token.\n\nThe default (GPT4 Correct) template is also available as the integrated `tokenizer.chat_template`, which can be used instead of manually specifying the template.\n\n# \u003Ca id=\"training\">\u003C\u002Fa> 🛠️ Training\n\nThe OpenChat training system utilizes padding-free training and the [Multipack Sampler](https:\u002F\u002Fgithub.com\u002Fimoneoi\u002Fmultipack_sampler), achieving a **3~10x** speedup compared to the conventional padded training.\n\n## Choose a base model\n\nOpenChat supports Llama 3 and Mistral models. Please first choose a base model to fit your needs. Each base model has a corresponding weight repo, model type, and recommended batch size as listed below, they should be filled into `BASE_REPO`, `MODEL_TYPE`, and `BATCH_SIZE` in the following instructions.\n\n| Base Model | Size | Weights (with EOT token)                   | Model Type              | Recommended Batch Size per GPU (8xA100 80GB) |\n|------------|------|--------------------------------------------|-------------------------|----------------------------------------------|\n| Llama 3    | 8B   | `imone\u002FLlama-3-8B-fixed-special-embedding` | `openchat_3.6`          | 40960                                        |\n| Mistral    | 7B   | `imone\u002FMistral_7B_with_EOT_token`          | `openchat_v3.2_mistral` | 77824                                        |\n\nNote: The OpenChat conversation template requires `\u003C|eot_id|>, \u003C|start_header_id|>, \u003C|end_header_id|>` (Llama 3) `\u003C|end_of_turn|>` (Mistral) special tokens. The base model specified must include these tokens with initialized embeddings. Our provided weights are the original base weights with this token added and embeddings initialized. If you want to add them manually, use the `init_special_embedding_llama3.py` or `mistral_add_tokens.py` in the `scripts` directory.\n\n## Installing DeepSpeed and Flash Attention\n\nFirst, ensure that the CUDA `nvcc` compiler is available in your environment. If it is not, install the CUDA toolkit that matches the version used by PyTorch.\n\nNext, install building dependencies:\n\n```bash\npip install packaging ninja\n```\n\nFinally, install the packages:\n\n```bash\npip install deepspeed flash-attn\n```\n\n### Preparing Your Data\n\nTo utilize the OpenChat trainer, prepare your SFT data into a JSON Lines format where each line corresponds to a `Conversation` object:\n\n```python\nclass Message(BaseModel):\n    role: str     # Must be \"user\" or \"assistant\"\n    content: str  # Message content\n    weight: Optional[float] = None  # Loss weight for this message. Typically 0 for user and 1 for assistant to supervise assistant's responses only\n\n\nclass Conversation(BaseModel):\n    items: List[Message]  # All messages within the conversation\n    condition: str = \"\"  # C-RLFT condition, can be any string or empty.\n    system: str = \"\"  # System message for this conversation\n```\n\nFor basic SFT, assign `weight` as `0` for human messages and `1` for assistant responses.\n\nSFT example:\n\n```json\n{\"items\":[{\"role\":\"user\",\"content\":\"Hello\",\"weight\":0.0},{\"role\":\"assistant\",\"content\":\"Hi\",\"weight\":1.0},{\"role\":\"user\",\"content\":\"How are you today?\",\"weight\":0.0},{\"role\":\"assistant\",\"content\":\"I'm fine.\",\"weight\":1.0}],\"system\":\"\"}\n{\"items\":[{\"role\":\"user\",\"content\":\"Who are you?\",\"weight\":0.0},{\"role\":\"assistant\",\"content\":\"I'm OpenChat.\",\"weight\":1.0}],\"system\":\"You are a helpful assistant named OpenChat.\"}\n```\n\nFor C-RLFT, `condition` should be set as the class the conversation belongs to (e.g. `GPT3` or `GPT4`). The `weight` is assigned as `0` for human messages and `w` for assistant responses, where `w` is the weight of the class (e.g. `0.1` for `GPT3` and `1` for `GPT4`, as found in our C-RLFT paper).\n\nC-RLFT example:\n\n```json\n{\"items\":[{\"role\":\"user\",\"content\":\"What is C-RLFT?\",\"weight\":0.0},{\"role\":\"assistant\",\"content\":\"C-RLFT is a method for improving open-source LLMs with mixed-quality data.\",\"weight\":1.0}],\"condition\":\"GPT4\",\"system\":\"\"}\n{\"items\":[{\"role\":\"user\",\"content\":\"What is C-RLFT?\",\"weight\":0.0},{\"role\":\"assistant\",\"content\":\"I don't know.\",\"weight\":0.1}],\"condition\":\"GPT3\",\"system\":\"\"}\n```\n\n### Pre-tokenizing the Dataset\n\nYou'll then need to pre-tokenize the dataset using the command (please specify a filename as `PRETOKENIZED_DATA_OUTPUT_PATH` to store the pretokenized dataset):\n\n```bash\npython -m ochat.data.generate_dataset --model-type MODEL_TYPE --model-path BASE_REPO --in-files data.jsonl --out-prefix PRETOKENIZED_DATA_OUTPUT_PATH\n```\n\n### Launching the OpenChat Trainer\n\nYou can now launch the OpenChat trainer using the command below.\n- 13B model requires eight `A\u002FH100s` with 80GB VRAM\n- 7B model can be trained with four `A\u002FH100s` with 80GB VRAM or eight `A\u002FH100s` with 40GB VRAM.\n\nFor hyperparameters, we recommend first setting the batch size to the recommended batch size. If OOM occurs, try setting it to the exact maximum that VRAM can hold and as a multiple of `2048`.\nOther hyperparameters have been carefully selected as the default. Furthermore, the learning rate is automatically determined based on the [inverse square-root rule](https:\u002F\u002Farxiv.org\u002Fabs\u002F2006.09092).\n\n\u003Cdetails>\n\n\u003Csummary>Training Commands (click to expand)\u003C\u002Fsummary>\n\n```bash\nNUM_GPUS=8\n\ndeepspeed --num_gpus=$NUM_GPUS --module ochat.training_deepspeed.train \\\n          --model_path BASE_REPO \\\n          --data_prefix PRETOKENIZED_DATA_OUTPUT_PATH \\\n          --save_path PATH_TO_SAVE_MODEL \\\n          --batch_max_len BATCH_SIZE \\\n          --epochs 5 \\\n          --save_every 1 \\\n          --deepspeed \\\n          --deepspeed_config ochat\u002Ftraining_deepspeed\u002Fdeepspeed_config.json\n```\n\n\u003C\u002Fdetails>\n\nYou can find checkpoints of all epochs in `PATH_TO_SAVE_MODEL`. Then you may evaluate each epoch and choose the best one.\n\n# Limitations\n\n**Foundation Model Limitations**: Despite its advanced capabilities, OpenChat is still bound by the limitations inherent in its foundation models. These limitations may impact the model's performance in areas such as:\n\n - Complex reasoning\n - Mathematical and arithmetic tasks\n - Programming and coding challenges\n\n**Hallucination of Non-existent Information:** OpenChat may sometimes generate information that does not exist or is not accurate, also known as \"hallucination\". Users should be aware of this possibility and verify any critical information obtained the model.\n\n**Safety:** OpenChat may sometimes generate harmful, hate speech, biased responses, or answer unsafe questions. It's crucial to apply additional AI safety measures in use cases that require safe and moderated responses.\n\n# License\n\nCode is distributed under the **Apache License 2.0**.\n\n# Citation\n\n```\n@article{wang2023openchat,\n  title={OpenChat: Advancing Open-source Language Models with Mixed-Quality Data},\n  author={Wang, Guan and Cheng, Sijie and Zhan, Xianyuan and Li, Xiangang and Song, Sen and Liu, Yang},\n  journal={arXiv preprint arXiv:2309.11235},\n  year={2023}\n}\n```\n\n# 💌Contact\n\n**Project Lead:**\n- Guan Wang [imonenext at gmail dot com]\n- [Alpay Ariyak](https:\u002F\u002Fgithub.com\u002Falpayariyak) [aariyak at wpi dot edu]\n\n**Main Contributors:**\n- [Xianyuan Zhan](https:\u002F\u002Fscholar.google.com.hk\u002Fcitations?user=pDMnGloAAAAJ) (Tsinghua University)\n- Qiying Yu (Tsinghua University)\n- Changling Liu (GPT Desk Pte. Ltd.)\n- LDJ\n- AutoMeta (Alignment Lab AI)\n\n**Sponsors:**\n- [Sen Song](https:\u002F\u002Fscholar.google.com\u002Fcitations?user=cYgtRP4AAAAJ) (Tsinghua University)\n- [Yang Liu](https:\u002F\u002Fnlp.csai.tsinghua.edu.cn\u002F~ly\u002F) (Tsinghua University)\n- [01.AI Company](https:\u002F\u002Fwww.lingyiwanwu.com\u002Fen)\n- [RunPod](https:\u002F\u002Fwww.runpod.io\u002F)\n\n**Special Thanks:**\n - [Mistral](https:\u002F\u002Fmistral.ai\u002F)\n - [Chain-of-Thought Hub](https:\u002F\u002Fgithub.com\u002FFranxYao\u002Fchain-of-thought-hub)\n - [Llama 2](https:\u002F\u002Fai.meta.com\u002Fllama\u002F)\n - [Self-Instruct](https:\u002F\u002Farxiv.org\u002Fabs\u002F2212.10560)\n - [FastChat (Vicuna)](https:\u002F\u002Fgithub.com\u002Flm-sys\u002FFastChat)\n - [Alpaca](https:\u002F\u002Fgithub.com\u002Ftatsu-lab\u002Fstanford_alpaca.git)\n - [StarCoder](https:\u002F\u002Fgithub.com\u002Fbigcode-project\u002Fstarcoder)\n","# OpenChat：利用混合质量数据推进开源语言模型发展\n\n\u003Cdiv align=\"center\">\n  \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fimoneoi_openchat_readme_2508a546dfb3.png\" style=\"width: 65%\">\n\u003C\u002Fdiv>\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Fopenchat.team\">💻在线演示\u003C\u002Fa> |\n  \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fopenchat\">🤗Huggingface\u003C\u002Fa> |\n  \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F2309.11235.pdf\">📃论文\u003C\u002Fa> |\n  \u003Ca href=\"https:\u002F\u002Fdiscord.gg\u002FpQjnXvNKHY\">💭Discord\u003C\u002Fa> \n\u003C\u002Fp>\n\n- OpenChat 是一个创新的 **开源语言模型** 库，采用受离线强化学习启发的 [**C-RLFT**](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2309.11235.pdf) 策略进行微调。\n- 我们的模型无需偏好标签即可从混合质量的数据中学习，在性能上可与 `ChatGPT` 相媲美，即便是仅需 **消费级显卡（如 RTX 3090）** 即可运行的 `7B` 参数模型也能达到这一水平。\n- 尽管我们的方法简单，但我们致力于开发高性能、具备商业可行性的开源大型语言模型，并在此愿景上不断取得重大进展。\n\n[![DOI](https:\u002F\u002Fzenodo.org\u002Fbadge\u002F645397533.svg)](https:\u002F\u002Fzenodo.org\u002Fbadge\u002Flatestdoi\u002F645397533)\n\n# ✨ 新闻\n\n - [2024年5月22日] 我们发布了基于 Llama-3 的版本 [OpenChat 3.6 20240522](https:\u002F\u002Fhuggingface.co\u002Fopenchat\u002Fopenchat-3.6-8b-20240522)，其性能超越了官方 Llama 3 8B Instruct 以及各类开源微调和合并模型。\n\n- [2024年1月6日] 我们发布了第二次更新 [OpenChat 3.5 0106](openchat\u002Fopenchat-3.5-0106)，进一步提升了代码编写能力和整体性能 🏆。\n\n- [2023年12月10日] 我们发布了首次更新 [OpenChat 3.5 1210](openchat\u002Fopenchat-3.5-1210)，代码能力提升了15分 🚀。\n\n- [2023年11月1日] 我们发布了 [OpenChat-3.5-7B](https:\u002F\u002Fhuggingface.co\u002Fopenchat\u002Fopenchat_3.5) 模型，在多项基准测试中超越了 ChatGPT 🔥。\n\n- [2023年9月21日] 我们发表了论文 [OpenChat：利用混合质量数据推进开源语言模型发展](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2309.11235.pdf)。\n  \n\u003Cdetails>\n  \u003Csummary>阅读更多\u003C\u002Fsummary>\n  \n- [2023年9月3日] 我们发布了 [OpenChat V3.2 SUPER]([#models](https:\u002F\u002Fhuggingface.co\u002Fopenchat\u002Fopenchat_v3.2_super)) 模型。\n\n- [2023年8月4日] 我们推出了 [在线演示](https:\u002F\u002Fopenchat.team)，展示了最新版本 OpenChat 3.2。\n\n- [2023年7月30日] 我们很高兴地推出基于 Llama 2 的 [OpenChat V3 模型系列](#models)，现已免费开放用于商业用途！\n\n- [2023年7月7日] 我们发布了 [OpenChat V2 模型系列](#legacy-models)。\n\n- [2023年7月1日] 我们发布了 [OpenChat V1 模型系列](#legacy-models)。\n\u003C\u002Fdetails>\n\n# 🏷️ 基准测试 - OpenChat 3.6\n\n\u003Cdiv align=\"center\">\n  \u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002Fimoneoi\u002Fopenchat\u002Fmaster\u002Fassets\u002Fbenchmarks-openchat-3.6-20240522.svg\" style=\"width: 95%;\">\n\u003C\u002Fdiv>\n\n\u003Cdetails>\n  \u003Csummary>基准测试复现\u003C\u002Fsummary>\n\n注意：请在本仓库的根目录下运行以下命令。\n\n```bash\npython -m ochat.evaluation.run_eval --condition \"GPT4 Correct\" --model openchat\u002Fopenchat-3.6-8b-20240522 --eval_sets fs_cothub\u002Fmmlu fs_cothub\u002Fgsm8k fs_cothub\u002Fmath\npython -m ochat.evaluation.run_eval --condition \"GPT4\" --model openchat\u002Fopenchat-3.6-8b-20240522 --eval_sets zs\u002Fgpqa\n```\n\nHumanEval 使用官方的 [EvalPlus 仓库](https:\u002F\u002Fgithub.com\u002Fevalplus\u002Fevalplus) 进行运行。\n\u003C\u002Fdetails>\n\n# 🏷️ 基准测试 - OpenChat 3.5\n\n| 模型                 | 参数量 | 平均得分 | MT-Bench     | HumanEval       | BBH MC   | AGIEval  | TruthfulQA    | MMLU         | GSM8K        | BBH CoT     |\n|-----------------------|----------|----------|--------------|-----------------|----------|----------|---------------|--------------|--------------|-------------|\n| **OpenChat-3.5-0106** | **7B**   | **64.5** | 7.8          | **71.3**        | **51.5** | **49.1** | 61.0          | 65.8         | **77.4**     | 62.2        |\n| ChatGPT (三月)*      | ???B     | 61.5     | **7.94**     | 48.1            | 47.6     | 47.1     | 57.7          | **67.3**     | 74.9         | **70.1**    |\n|                       |          |          |              |                 |          |          |               |              |              |             |\n| OpenHermes 2.5        | 7B       | 59.3     | 7.54         | 48.2            | 49.4     | 46.5     | 57.5          | 63.8         | 73.5         | 59.9        |\n| OpenOrca Mistral      | 7B       | 52.7     | 6.86         | 38.4            | 49.4     | 42.9     | 45.9          | 59.3         | 59.1         | 58.1        |\n| Zephyr-β^             | 7B       | 34.6     | 7.34         | 22.0            | 40.6     | 39.0     | 40.8          | 39.8         | 5.1          | 16.0        |\n| Mistral               | 7B       | -        | 6.84         | 30.5            | 39.0     | 38.0     | -             | 60.1         | 52.2         | -           |\n| 开源SOTA**            | 13B-70B  | 61.4     | 7.71         | 73.2            | 49.7     | 41.7     | 62.3          | 63.7         | 82.3         | 41.4        |\n|                       |          |          | WizardLM 70B | WizardCoder 34B | Orca 13B | Orca 13B | Platypus2 70B | WizardLM 70B | MetaMath 70B | Flan-T5 11B |\n\n🔥 OpenChat-3.5-0106 (7B) 现在在**所有4项基准测试**上都超过了Grok-0 (33B)，并且在平均分和**3\u002F4的基准测试**上也超过了Grok-1 (314B)。\n\n|                       | 许可证     | 参数量 | 平均得分 | MMLU   | HumanEval | MATH     | GSM8k    |\n|-----------------------|-------------|---------|----------|--------|-----------|----------|----------|\n| **OpenChat-3.5-0106** | Apache-2.0  | **7B**  | **61.0** | 65.8   | **71.3**  | **29.3** | **77.4** |\n| Grok-0                | 专有       | 33B     | 44.5     | 65.7   | 39.7      | 15.7     | 56.8     |\n| Grok-1                | 专有       | 314B    | 55.8     | **73** | 63.2      | 23.9     | 62.9     |\n\n\u003Cdetails>\n  \u003Csummary>评估详情\u003C\u002Fsummary>\n*: ChatGPT (三月)的结果来自GPT-4技术报告、Chain-of-Thought Hub以及我们的评估。\n\n^: Zephyr-β经常无法遵循少样本CoT指令，这很可能是因为它只经过聊天数据对齐，而没有接受少样本数据的训练。\n\n **: Mistral和开源SOTA的结果取自指令微调模型论文及官方仓库中报告的数据。\n\n所有模型均以聊天模式进行评估（例如应用相应的对话模板）。所有零样本基准测试均采用与AGIEval论文和Orca论文相同的设置。CoT任务使用与Chain-of-Thought Hub相同的配置，HumanEval则通过EvalPlus进行评估，MT-bench则使用FastChat运行。要复现我们的结果，请按照以下说明操作。\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n  \u003Csummary>复现基准测试\u003C\u002Fsummary>\n\n推理与编码：\n\n注意：请在本仓库的根目录下运行以下命令。\n\n```bash\npython -m ochat.evaluation.run_eval --condition \"GPT4 Correct\" --model openchat\u002Fopenchat-3.5-0106 --eval_sets coding fs_cothub\u002Fbbh fs_cothub\u002Fmmlu zs\u002Fagieval zs\u002Fbbh_mc_orca zs\u002Ftruthfulqa_orca\npython ochat\u002Fevaluation\u002Fview_results.py\npython ochat\u002Fevaluation\u002Fconvert_to_evalplus.py\n```\n\n然后所有humaneval代码样本都会被放置在`ochat\u002Fevaluation\u002Fevalplus_codegen`中。使用以下命令，通过Docker作为沙箱来评估名为`samples.jsonl`的单个代码样本。\n\n```bash\ndocker run -v $(pwd):\u002Fapp ganler\u002Fevalplus:latest --dataset humaneval --samples samples.jsonl\n```\n\n数学推理：\n\n注意：请在本仓库的根目录下运行以下命令。\n\n```bash\npython -m ochat.evaluation.run_eval --condition \"Math Correct\" --model openchat\u002Fopenchat-3.5-0106 --eval_sets fs_cothub\u002Fgsm8k zs\u002Fmath\npython ochat\u002Fevaluation\u002Fview_results.py\n```\n\nMT-Bench：\n\n请先启动本地API服务器，然后下载FastChat并运行以下命令。\n\n注意：由于非零温度和GPT-4 API随时间的变化，结果可能会有所差异。\n\n```bash\ncd fastchat\u002Fllm_judge\npython gen_api_answer.py --model openchat-3.5-0106 --max-tokens 4096 --parallel 128 --openai-api-base http:\u002F\u002Flocalhost:18888\u002Fv1\npython gen_judgment.py --model-list openchat-3.5-0106 --parallel 8 --mode single\n```\n\n\u003C\u002Fdetails>\n\n# ⬇️ 安装\n> [!NOTE]\n> 运行OpenChat需要[`pytorch`](https:\u002F\u002Fpytorch.org\u002Fget-started\u002Flocally\u002F#start-locally)和[CUDA](https:\u002F\u002Fdeveloper.nvidia.com\u002Fcuda-toolkit-archive)\n\n## pip\n\n```bash\npip3 install ochat\n```\n> [!IMPORTANT]\n> 如果您在使用pip时遇到包兼容性问题，请尝试下面的conda方法，或查看[此问题](https:\u002F\u002Fgithub.com\u002Fimoneoi\u002Fopenchat\u002Fissues\u002F41)\n\n## conda\n\n```bash\nconda create -y --name openchat python=3.11\nconda activate openchat\n\npip3 install ochat\n```\n\n## Windows (WSL 1.x, Ubuntu-22.04)\n\n```bash\nsudo apt update\nsudo apt install build-essential\n\nsudo apt install -y curl\ncurl -o miniconda.sh https:\u002F\u002Frepo.anaconda.com\u002Fminiconda\u002FMiniconda3-latest-Linux-x86_64.sh\nbash miniconda.sh\n\n# 如果以下conda命令不起作用，请重启WSL终端\n\nconda create -y --name openchat python=3.11\nconda activate openchat\n\npip3 install ochat\n```\n\n## 从源码安装\n\n\u003Cdetails>\n  \u003Csummary>克隆本仓库，并以可编辑模式从源码安装openchat\u003C\u002Fsummary>\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002Fimoneoi\u002Fopenchat\ncd openchat\n\npip3 install --upgrade pip  # 启用PEP 660支持\npip3 install -e .  # 可编辑模式，您可以在此克隆的仓库中进行修改\n```\n\u003C\u002Fdetails>\n\n# 🚀 部署API服务器\n\n⚡ 我们的API服务器已准备好用于生产环境，并且兼容OpenAI API协议。它经过vLLM的高度优化，可以动态批处理请求。\n\n📎 注意：对于不支持`bfloat16`的20系或更早的GPU，请在服务器参数中添加`--dtype float16`。\n\n### 目前支持的模型列表\n\n| 模型类型   | 模型仓库                                                                                    | 许可证    |\n|--------------|-----------------------------------------------------------------------------------------------|------------|\n| openchat_3.6 | [openchat\u002Fopenchat-3.6-8b-20240522](https:\u002F\u002Fhuggingface.co\u002Fopenchat\u002Fopenchat-3.6-8b-20240522) | Llama 3    |\n| openchat_3.5 | [openchat\u002Fopenchat-3.5-0106](https:\u002F\u002Fhuggingface.co\u002Fopenchat\u002Fopenchat-3.5-0106)               | Apache 2.0 |\n\n### 对于单个 GPU（例如 RTX 3090、4090）\n\n```bash\npython -m ochat.serving.openai_api_server --model MODEL_REPO\n```\n\n### 对于多个 GPU（张量并行）\n\n```bash\n# N 是张量并行的 GPU 数量\npython -m ochat.serving.openai_api_server --model MODEL_REPO --engine-use-ray --worker-use-ray --tensor-parallel-size N\n```\n\n使用 `-h` 查看更多设置\n```bash\npython -m ochat.serving.openai_api_server --model MODEL_REPO -h\n```\n\n\u003Cdetails>\n  \u003Csummary>部署为在线服务\u003C\u002Fsummary>\n\n如果想要将服务器部署为在线服务，可以使用 `--api-keys sk-KEY1 sk-KEY2 ...` 来指定允许的 API 密钥，并使用 `--disable-log-requests --disable-log-stats --log-file openchat.log` 仅将日志记录到文件中。出于安全考虑，我们建议在服务器前面使用一个 [HTTPS 网关](https:\u002F\u002Ffastapi.tiangolo.com\u002Fes\u002Fdeployment\u002Fconcepts\u002F#security-https)。\n\n\u003C\u002Fdetails>\n\n## 请求示例\n\n启动后，服务器会在 `localhost:18888` 监听请求，并且兼容 [OpenAI ChatCompletion API 规范](https:\u002F\u002Fplatform.openai.com\u002Fdocs\u002Fapi-reference\u002Fchat)。\n\n💡 **默认模式（GPT4 Correct）**：最适合编码、聊天和一般任务\n\n```bash\ncurl http:\u002F\u002Flocalhost:18888\u002Fv1\u002Fchat\u002Fcompletions \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\n    \"model\": \"MODEL_TYPE\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"你是一个名为 OpenChat 的大型语言模型。请写一首诗来描述你自己\"}]\n  }'\n```\n\n🧮 **数学推理模式**：专为解决数学问题而设计\n\n```bash\ncurl http:\u002F\u002Flocalhost:18888\u002Fv1\u002Fchat\u002Fcompletions \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\n    \"model\": \"MODEL_TYPE\",\n    \"condition\": \"Math Correct\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"10.3 − 7988.8133 = \"}]\n  }'\n```\n\n# \u003Ca id=\"web-ui\">\u003C\u002Fa> 🌐 Web UI - [OpenChat-UI](https:\u002F\u002Fgithub.com\u002Fimoneoi\u002Fopenchat-ui)\n\n启动 API 服务器后，OpenChat 提供了一个易于交互的用户界面。[点击此处查看 Web UI](https:\u002F\u002Fgithub.com\u002Fimoneoi\u002Fopenchat-ui)\n\n# 🤗 使用 Transformers 进行推理\n\n> [!WARNING]\n> 建议使用我们优化后的 API 服务器进行部署。使用 Transformers 进行推理会更慢。\n\n💡 **默认模式（GPT4 Correct）**：最适合编码、聊天和一般任务\n\n```\nGPT4 Correct 用户: 你好\u003C|end_of_turn|>GPT4 Correct 助手: 你好\u003C|end_of_turn|>GPT4 Correct 用户: 你今天怎么样?\u003C|end_of_turn|>GPT4 Correct 助手:\n```\n\n🧮 **数学推理模式**：专为解决数学问题而设计\n\n```\nMath Correct 用户: 10.3 − 7988.8133=\u003C|end_of_turn|>Math Correct 助手:\n```\n\n⚠️ **注意**：请记住将 `\u003C|end_of_turn|>` 设置为生成结束标记。\n\n默认的（GPT4 Correct）模板也可以作为集成的 `tokenizer.chat_template` 使用，这样就不需要手动指定模板了。\n\n# \u003Ca id=\"training\">\u003C\u002Fa> 🛠️ 训练\n\nOpenChat 的训练系统采用无填充训练和 [Multipack Sampler](https:\u002F\u002Fgithub.com\u002Fimoneoi\u002Fmultipack_sampler)，相比传统的填充训练，速度提升了 **3~10倍**。\n\n## 选择基础模型\n\nOpenChat 支持 Llama 3 和 Mistral 模型。请先根据需求选择一个基础模型。每个基础模型都有对应的权重仓库、模型类型和推荐的每 GPU 批量大小，如下表所示，这些信息应填写到后续说明中的 `BASE_REPO`、`MODEL_TYPE` 和 `BATCH_SIZE` 中。\n\n| 基础模型 | 大小 | 权重（含 EOT 标记）                   | 模型类型              | 推荐的每 GPU 批量大小（8xA100 80GB） |\n|------------|------|--------------------------------------------|-------------------------|----------------------------------------------|\n| Llama 3    | 8B   | `imone\u002FLlama-3-8B-fixed-special-embedding` | `openchat_3.6`          | 40960                                        |\n| Mistral    | 7B   | `imone\u002FMistral_7B_with_EOT_token`          | `openchat_v3.2_mistral` | 77824                                        |\n\n注意：OpenChat 的对话模板需要 `\u003C|eot_id|>, \u003C|start_header_id|>, \u003C|end_header_id|>`（Llama 3）或 `\u003C|end_of_turn|>`（Mistral）特殊标记。指定的基础模型必须包含这些标记，并已初始化嵌入。我们提供的权重是在原始基础权重基础上添加了这些标记并进行了嵌入初始化。如果您想手动添加，可以使用 `scripts` 目录下的 `init_special_embedding_llama3.py` 或 `mistral_add_tokens.py`。\n\n## 安装 DeepSpeed 和 Flash Attention\n\n首先，请确保您的环境中可用 CUDA 的 `nvcc` 编译器。如果不可用，请安装与 PyTorch 版本匹配的 CUDA 工具包。\n\n接下来，安装构建依赖项：\n\n```bash\npip install packaging ninja\n```\n\n最后，安装相关包：\n\n```bash\npip install deepspeed flash-attn\n```\n\n### 准备数据\n\n要使用 OpenChat 训练器，需将 SFT 数据准备成 JSON Lines 格式，其中每行对应一个 `Conversation` 对象：\n\n```python\nclass Message(BaseModel):\n    role: str     # 必须是 \"user\" 或 \"assistant\"\n    content: str  # 消息内容\n    weight: Optional[float] = None  # 此消息的损失权重。通常用户为 0，助手为 1，以仅监督助手的回答\n\n\nclass Conversation(BaseModel):\n    items: List[Message]  # 对话中的所有消息\n    condition: str = \"\"  # C-RLFT 条件，可以是任意字符串或空。\n    system: str = \"\"  # 此对话的系统提示\n```\n\n对于基本的 SFT，将人类消息的 `weight` 设为 `0`，助手回复的 `weight` 设为 `1`。\n\nSFT 示例：\n\n```json\n{\"items\":[{\"role\":\"user\",\"content\":\"你好\",\"weight\":0.0},{\"role\":\"assistant\",\"content\":\"嗨\",\"weight\":1.0},{\"role\":\"user\",\"content\":\"你今天怎么样？\",\"weight\":0.0},{\"role\":\"assistant\",\"content\":\"我很好。\",\"weight\":1.0}],\"system\":\"\"}\n{\"items\":[{\"role\":\"user\",\"content\":\"你是谁？\",\"weight\":0.0},{\"role\":\"assistant\",\"content\":\"我是 OpenChat。\",\"weight\":1.0}],\"system\":\"你是一个名为 OpenChat 的 helpful 助手。\"}\n```\n\n对于 C-RLFT，`condition` 应设置为对话所属的类别（例如 `GPT3` 或 `GPT4`）。人类消息的 `weight` 设为 `0`，助手回复的 `weight` 设为 `w`，其中 `w` 是该类别的权重（例如，`GPT3` 为 `0.1`，`GPT4` 为 `1`，如我们在 C-RLFT 论文中所述）。\n\nC-RLFT 示例：\n\n```json\n{\"items\":[{\"role\":\"user\",\"content\":\"什么是 C-RLFT？\",\"weight\":0.0},{\"role\":\"assistant\",\"content\":\"C-RLFT 是一种利用混合质量数据改进开源 LLM 的方法。\",\"weight\":1.0}],\"condition\":\"GPT4\",\"system\":\"\"}\n{\"items\":[{\"role\":\"user\",\"content\":\"什么是 C-RLFT？\",\"weight\":0.0},{\"role\":\"assistant\",\"content\":\"我不知道。\",\"weight\":0.1}],\"condition\":\"GPT3\",\"system\":\"\"}\n```\n\n### 数据集的预分词\n\n接下来，你需要使用以下命令对数据集进行预分词（请指定一个文件名作为 `PRETOKENIZED_DATA_OUTPUT_PATH` 来存储预分词后的数据集）：\n\n```bash\npython -m ochat.data.generate_dataset --model-type MODEL_TYPE --model-path BASE_REPO --in-files data.jsonl --out-prefix PRETOKENIZED_DATA_OUTPUT_PATH\n```\n\n### 启动 OpenChat 训练器\n\n现在你可以使用下面的命令启动 OpenChat 训练器。\n- 13B 模型需要八张配备 80GB 显存的 A\u002FH100 显卡。\n- 7B 模型可以使用四张配备 80GB 显存的 A\u002FH100 显卡，或者八张配备 40GB 显存的 A\u002FH100 显卡来训练。\n\n对于超参数，我们建议首先将批大小设置为推荐值。如果出现显存不足（OOM）的情况，可以尝试将其调整为显存所能容纳的最大值，并且确保该值是 `2048` 的倍数。其他超参数已作为默认值精心选择。此外，学习率会根据 [反平方根法则](https:\u002F\u002Farxiv.org\u002Fabs\u002F2006.09092) 自动确定。\n\n\u003Cdetails>\n\n\u003Csummary>训练命令（点击展开）\u003C\u002Fsummary>\n\n```bash\nNUM_GPUS=8\n\ndeepspeed --num_gpus=$NUM_GPUS --module ochat.training_deepspeed.train \\\n          --model_path BASE_REPO \\\n          --data_prefix PRETOKENIZED_DATA_OUTPUT_PATH \\\n          --save_path PATH_TO_SAVE_MODEL \\\n          --batch_max_len BATCH_SIZE \\\n          --epochs 5 \\\n          --save_every 1 \\\n          --deepspeed \\\n          --deepspeed_config ochat\u002Ftraining_deepspeed\u002Fdeepspeed_config.json\n```\n\n\u003C\u002Fdetails>\n\n你可以在 `PATH_TO_SAVE_MODEL` 中找到所有 epoch 的检查点。然后你可以评估每个 epoch，并选择表现最好的模型。\n\n# 局限性\n\n**基础模型的局限性**：尽管 OpenChat 具备先进的能力，但它仍然受到其基础模型固有局限性的约束。这些局限性可能会影响模型在以下方面的表现：\n\n- 复杂推理\n- 数学和算术任务\n- 编程挑战\n\n**编造不存在的信息**：OpenChat 有时可能会生成并不存在或不准确的信息，这也被称为“幻觉”。用户应意识到这种可能性，并对从模型获得的任何关键信息进行核实。\n\n**安全性**：OpenChat 有时可能会生成有害、仇恨言论、带有偏见的回复，或者回答不安全的问题。在需要安全且经过审核的响应的应用场景中，务必采取额外的人工智能安全措施。\n\n# 许可证\n\n代码以 **Apache License 2.0** 协议发布。\n\n# 引用\n\n```\n@article{wang2023openchat,\n  title={OpenChat: Advancing Open-source Language Models with Mixed-Quality Data},\n  author={Wang, Guan and Cheng, Sijie and Zhan, Xianyuan and Li, Xiangang and Song, Sen and Liu, Yang},\n  journal={arXiv preprint arXiv:2309.11235},\n  year={2023}\n}\n```\n\n# 💌联系方式\n\n**项目负责人：**\n- Guan Wang [imonenext at gmail dot com]\n- [Alpay Ariyak](https:\u002F\u002Fgithub.com\u002Falpayariyak) [aariyak at wpi dot edu]\n\n**主要贡献者：**\n- [Xianyuan Zhan](https:\u002F\u002Fscholar.google.com.hk\u002Fcitations?user=pDMnGloAAAAJ)（清华大学）\n- Qiying Yu（清华大学）\n- Changling Liu（GPT Desk Pte. Ltd.）\n- LDJ\n- AutoMeta（Alignment Lab AI）\n\n**赞助方：**\n- [Sen Song](https:\u002F\u002Fscholar.google.com\u002Fcitations?user=cYgtRP4AAAAJ)（清华大学）\n- [Yang Liu](https:\u002F\u002Fnlp.csai.tsinghua.edu.cn\u002F~ly\u002F)（清华大学）\n- [01.AI Company](https:\u002F\u002Fwww.lingyiwanwu.com\u002Fen)\n- [RunPod](https:\u002F\u002Fwww.runpod.io\u002F)\n\n**特别感谢：**\n- [Mistral](https:\u002F\u002Fmistral.ai\u002F)\n- [Chain-of-Thought Hub](https:\u002F\u002Fgithub.com\u002FFranxYao\u002Fchain-of-thought-hub)\n- [Llama 2](https:\u002F\u002Fai.meta.com\u002Fllama\u002F)\n- [Self-Instruct](https:\u002F\u002Farxiv.org\u002Fabs\u002F2212.10560)\n- [FastChat (Vicuna)](https:\u002F\u002Fgithub.com\u002Flm-sys\u002FFastChat)\n- [Alpaca](https:\u002F\u002Fgithub.com\u002Ftatsu-lab\u002Fstanford_alpaca.git)\n- [StarCoder](https:\u002F\u002Fgithub.com\u002Fbigcode-project\u002Fstarcoder)","# OpenChat 快速上手指南\n\nOpenChat 是一个创新的开源大语言模型库，采用 **C-RLFT**（受离线强化学习启发的策略）进行微调。它能够从混合质量数据中学习，无需偏好标签，即可在消费级显卡（如 RTX 3090）上运行，并达到媲美 ChatGPT 的性能。\n\n## 1. 环境准备\n\n在开始之前，请确保您的系统满足以下要求：\n\n*   **操作系统**: Linux (推荐), macOS, 或 Windows (需通过 WSL2 运行)。\n*   **硬件**: \n    *   NVIDIA GPU (支持 CUDA)。\n    *   显存建议：运行 7B\u002F8B 模型建议至少 16GB 显存 (如 RTX 3090\u002F4090)。\n    *   *注：对于不支持 `bfloat16` 的旧款 GPU (如 20 系列)，启动时需指定 `--dtype float16`。*\n*   **前置依赖**:\n    *   [PyTorch](https:\u002F\u002Fpytorch.org\u002Fget-started\u002Flocally\u002F#start-locally) (需匹配您的 CUDA 版本)\n    *   [CUDA Toolkit](https:\u002F\u002Fdeveloper.nvidia.com\u002Fcuda-toolkit-archive)\n    *   Python 3.11 (推荐)\n\n> **提示**: 国内用户若遇到 PyTorch 下载慢的问题，建议使用清华源或阿里源安装 PyTorch。\n\n## 2. 安装步骤\n\n您可以选择使用 `pip` 直接安装，或使用 `conda` 创建隔离环境（推荐）。\n\n### 方法一：使用 Conda (推荐)\n\n```bash\n# 创建名为 openchat 的虚拟环境，指定 Python 版本为 3.11\nconda create -y --name openchat python=3.11\nconda activate openchat\n\n# 安装 ochat 包\n# 国内用户如遇网络问题，可添加镜像源：pip3 install ochat -i https:\u002F\u002Fpypi.tuna.tsinghua.edu.cn\u002Fsimple\npip3 install ochat\n```\n\n### 方法二：直接使用 Pip\n\n```bash\npip3 install ochat\n```\n\n> **注意**: 如果遇到包兼容性问题，请优先尝试上述 Conda 方法。\n\n### 方法三：Windows (WSL) 用户\n\n如果您在 Windows 上使用 WSL (Ubuntu 22.04)，请先安装基础构建工具：\n\n```bash\nsudo apt update\nsudo apt install build-essential curl\n\n# 下载并安装 Miniconda (如果尚未安装)\ncurl -o miniconda.sh https:\u002F\u002Frepo.anaconda.com\u002Fminiconda\u002FMiniconda3-latest-Linux-x86_64.sh\nbash miniconda.sh\n\n# 重启终端后执行以下命令\nconda create -y --name openchat python=3.11\nconda activate openchat\npip3 install ochat\n```\n\n## 3. 基本使用\n\nOpenChat 提供了一个兼容 OpenAI API 协议的高性能 API 服务器，基于 vLLM 优化，支持动态批处理。\n\n### 启动 API 服务器\n\n**单卡模式 (如 RTX 3090, 4090):**\n\n```bash\n# 将 MODEL_REPO 替换为您想使用的模型仓库路径\n# 例如：openchat\u002Fopenchat-3.6-8b-20240522 或 openchat\u002Fopenchat-3.5-0106\npython -m ochat.serving.openai_api_server --model openchat\u002Fopenchat-3.6-8b-20240522\n```\n\n**多卡模式 (张量并行):**\n\n```bash\n# N 为使用的 GPU 数量\npython -m ochat.serving.openai_api_server --model openchat\u002Fopenchat-3.6-8b-20240522 --engine-use-ray --worker-use-ray --tensor-parallel-size N\n```\n\n*服务器默认监听 `localhost:18888`。*\n\n### 发送请求示例\n\n服务器启动后，您可以使用 `curl` 或任何兼容 OpenAI SDK 的客户端进行调用。\n\n#### 场景 A：通用对话与代码生成 (默认模式)\n此模式 (`GPT4 Correct`) 适用于聊天、代码编写及一般任务。\n\n```bash\ncurl http:\u002F\u002Flocalhost:18888\u002Fv1\u002Fchat\u002Fcompletions \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\n    \"model\": \"openchat_3.6\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"You are a large language model named OpenChat. Write a poem to describe yourself\"}]\n  }'\n```\n\n#### 场景 B：数学推理模式\n此模式 (`Math Correct`) 专为解决数学问题优化。\n\n```bash\ncurl http:\u002F\u002Flocalhost:18888\u002Fv1\u002Fchat\u002Fcompletions \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\n    \"model\": \"openchat_3.6\",\n    \"condition\": \"Math Correct\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"10.3 − 7988.8133 = \"}]\n  }'\n```\n\n### 当前支持的模型\n\n| 模型类型 | 模型仓库 (MODEL_REPO) | 许可证 | 特点 |\n| :--- | :--- | :--- | :--- |\n| **OpenChat 3.6** | `openchat\u002Fopenchat-3.6-8b-20240522` | Llama 3 | 基于 Llama-3，性能最强 |\n| **OpenChat 3.5** | `openchat\u002Fopenchat-3.5-0106` | Apache 2.0 | 编码与综合能力优秀，商业友好 |\n\n只需将启动命令和请求中的 `--model` 及 `\"model\"` 字段替换为对应的仓库名称即可切换模型。","一家初创公司的后端团队需要在单张消费级显卡（如 RTX 3090）上部署私有化代码助手，以辅助开发人员快速生成和修复业务逻辑代码，同时严格保障数据不出域。\n\n### 没有 openchat 时\n- **硬件门槛高**：想要获得接近 ChatGPT 的代码能力，通常必须租用昂贵的云端 A100\u002FH800 集群或购买多卡服务器，初创团队预算难以承受。\n- **数据利用难**：团队内部积累了大量未经人工偏好标注的混合质量历史代码日志，因缺乏精细清洗和打标资源，这些宝贵数据只能被闲置浪费。\n- **响应延迟大**：调用外部公有云 API 存在网络波动和高延迟问题，且敏感业务代码上传至第三方模型存在严重的数据泄露合规风险。\n- **微调效果差**：尝试使用传统指令微调（SFT）处理混合质量数据时，模型容易“学坏”，导致输出不稳定，无法在复杂逻辑任务中达到生产级可用标准。\n\n### 使用 openchat 后\n- **单机即可部署**：借助 openchat 基于 C-RLFT 策略优化的 7B\u002F8B 模型，团队直接在单张 RTX 3090 上跑通了媲美 ChatGPT 的代码助手，大幅降低硬件成本。\n- **变废为宝**：openchat 能够直接从无偏好标签的混合质量数据中学习，团队将原本闲置的历史代码库转化为训练燃料，显著提升了模型对自家业务场景的理解。\n- **本地实时响应**：模型完全本地化运行，消除了网络延迟，实现了毫秒级的代码补全与纠错反馈，同时确保核心代码资产永远留在内网。\n- **性能越级表现**：得益于独特的离线强化学习策略，openchat 在 HumanEval 等代码基准测试中得分超越众多大参数模型，即使面对复杂算法题也能给出高质量解答。\n\nopenchat 让中小企业得以用消费级硬件和原始数据，构建出具备商业级竞争力的私有化大语言模型应用。","https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fimoneoi_openchat_2508a546.png","imoneoi","One","https:\u002F\u002Foss.gittoolsai.com\u002Favatars\u002Fimoneoi_2c5541ff.png","Professional RL(HF) hyperparameter tuner",null,"http:\u002F\u002Fimone.me","https:\u002F\u002Fgithub.com\u002Fimoneoi",[83,87,91,95],{"name":84,"color":85,"percentage":86},"Python","#3572A5",57.4,{"name":88,"color":89,"percentage":90},"Jupyter Notebook","#DA5B0B",42,{"name":92,"color":93,"percentage":94},"Dockerfile","#384d54",0.4,{"name":96,"color":97,"percentage":98},"Shell","#89e051",0.2,5477,435,"2026-04-06T02:45:17","Apache-2.0","Linux, Windows (via WSL)","必需 NVIDIA GPU。支持消费级显卡（如 RTX 3090, 4090）。需安装 CUDA。20 系列或更旧的显卡需指定 --dtype float16（不支持 bfloat16）。","未说明",{"notes":107,"python":108,"dependencies":109},"1. Windows 用户需通过 WSL (Ubuntu 22.04) 运行。\n2. 推荐使用 conda 创建 Python 3.11 环境以避免包兼容性问题。\n3. API 服务器基于 vLLM 优化，兼容 OpenAI API 协议。\n4. 多卡部署需使用 Ray 并设置 tensor-parallel-size 参数。","3.11",[110,111,112],"pytorch","vLLM","ochat",[15],[115,116,117],"large-language-models","open-source","transformers","2026-03-27T02:49:30.150509","2026-04-07T07:19:34.410523",[121,126,131,136,141,146],{"id":122,"question_zh":123,"answer_zh":124,"source_url":125},21240,"如何复现 OpenChat V3.1 13B 在 alpaca_eval 上的评估结果？","请使用最新的 `alpaca_eval` 版本和 `gpt-4` (latest) 评估器。具体命令如下：\n```bash\nOPENAI_API_KEYS=sk-XXX PYTHONPATH=\"$PYTHONPATH:$(pwd)\u002Fsrc\" python -m alpaca_eval.main evaluate --model_outputs results\u002Fopenchat-v3.1-13b\u002Fmodel_outputs.json --annotators_config alpaca_eval_gpt4\n```\n注意：确保使用的是最新的 GPT-4 API 版本，旧版本（如 gpt-4-0314）可能导致结果差异。","https:\u002F\u002Fgithub.com\u002Fimoneoi\u002Fopenchat\u002Fissues\u002F42",{"id":127,"question_zh":128,"answer_zh":129,"source_url":130},21241,"在线演示版是在与 GPT-4 对话还是 OpenChat 模型？","在线演示使用的是 OpenChat 服务器，但为了兼容性，它采用了 OpenAI API 的规范接口。这意味着虽然底层运行的是 OpenChat 模型，但其交互方式模拟了 OpenAI 的 API 标准，因此某些界面提示可能会指向 OpenAI 相关服务或需要配置 API Key 以兼容特定功能。","https:\u002F\u002Fgithub.com\u002Fimoneoi\u002Fopenchat\u002Fissues\u002F78",{"id":132,"question_zh":133,"answer_zh":134,"source_url":135},21242,"为什么 openchat-3.5-1210 版本在启动 API 服务器时非常慢或卡住？","启动速度慢通常与磁盘读取速度和缓存有关，特别是在 Windows WSL 环境下内存不足时，系统可能使用共享显存导致加载缓慢。\n解决方案：\n1. 尝试重新加载模型，观察速度是否有变化。\n2. 确保有足够的物理内存（例如 24GB 以上），避免系统使用慢速的交换空间。\n3. 耐心等待首次加载完成（可能需要 15 分钟），后续启动通常会快很多。","https:\u002F\u002Fgithub.com\u002Fimoneoi\u002Fopenchat\u002Fissues\u002F128",{"id":137,"question_zh":138,"answer_zh":139,"source_url":140},21243,"安装 requirements.txt 时遇到 'ModuleNotFoundError: No module named torch' 错误怎么办？","该错误通常发生在安装 `flash-attn` 依赖时，因为构建过程需要预先安装 `torch`。\n解决方案：\n1. 建议直接从 Git 源码安装 flash-attn，而不是通过 requirements.txt 自动安装：\n   ```bash\n   pip install git+https:\u002F\u002Fgithub.com\u002FHazyResearch\u002Fflash-attention.git\n   ```\n2. 或者先手动确保 `torch` 已正确安装且版本兼容，然后再运行 `pip install -r requirements.txt`。\n3. 推荐使用 `conda` 创建独立环境来管理依赖，避免版本冲突。","https:\u002F\u002Fgithub.com\u002Fimoneoi\u002Fopenchat\u002Fissues\u002F8",{"id":142,"question_zh":143,"answer_zh":144,"source_url":145},21244,"在 transformers 库中使用 OpenChat 的训练技巧时，Loss 值异常高（如 100.x）是什么原因？","Loss 异常高可能是因为加权损失计算方式与 HuggingFace Trainer 的默认行为不一致。\n关键点：\n1. 当 batch size > 1 且混合不同来源数据（如 GPT-3 和 GPT-4）时，权重会影响最终 Loss。\n2. HuggingFace Trainer 会将每个 GPU 的 Loss 简单除以 GPU 数量，而 OpenChat 原仓库是将总加权 Loss 除以权重总和。\n3. 如果使用梯度累积，也会导致计算差异。\n建议：检查是否正确实现了 `weighted_cross_entropy`，并确保在分布式训练或梯度累积时，Loss 归一化逻辑与原仓库一致。","https:\u002F\u002Fgithub.com\u002Fimoneoi\u002Fopenchat\u002Fissues\u002F74",{"id":147,"question_zh":148,"answer_zh":149,"source_url":130},21245,"如何在本地部署支持 OpenAI API 格式的 OpenChat 服务？","可以使用项目提供的 `openai_api_server` 模块来启动服务，命令如下：\n```bash\npython -m ochat.serving.openai_api_server --model \u003C模型路径>\n```\n该服务兼容 OpenAI API 规范，允许你使用标准的 OpenAI 客户端库进行调用。注意：如果遇到启动缓慢问题，请参考磁盘速度和内存相关的优化建议。",[151,155,159,163,167,171,176,181,185,190,194,198,203],{"id":152,"version":153,"summary_zh":79,"released_at":154},127267,"3.6.1","2024-05-24T18:26:23",{"id":156,"version":157,"summary_zh":79,"released_at":158},127268,"3.6.0","2024-05-24T08:04:16",{"id":160,"version":161,"summary_zh":79,"released_at":162},127269,"3.5.2","2024-03-09T17:37:28",{"id":164,"version":165,"summary_zh":79,"released_at":166},127270,"3.5.1","2023-12-13T18:00:16",{"id":168,"version":169,"summary_zh":79,"released_at":170},127271,"3.5.0","2023-11-02T15:29:13",{"id":172,"version":173,"summary_zh":174,"released_at":175},127272,"3.2.2","## 变更内容\n* 由 @imoneoi 在 https:\u002F\u002Fgithub.com\u002Fimoneoi\u002Fopenchat\u002Fpull\u002F53 中修复了元数据问题\n\n**完整变更日志**: https:\u002F\u002Fgithub.com\u002Fimoneoi\u002Fopenchat\u002Fcompare\u002F3.2.1...3.2.2","2023-09-18T15:55:25",{"id":177,"version":178,"summary_zh":179,"released_at":180},127273,"3.2.1","添加 flash-attn 作为依赖项","2023-09-14T10:44:59",{"id":182,"version":183,"summary_zh":79,"released_at":184},127274,"3.2.0","2023-09-04T15:12:39",{"id":186,"version":187,"summary_zh":188,"released_at":189},127275,"3.1.2","修复 vLLM 1.1.3 的停止令牌","2023-08-03T11:12:16",{"id":191,"version":192,"summary_zh":79,"released_at":193},127276,"3.1.1","2023-08-01T17:52:31",{"id":195,"version":196,"summary_zh":79,"released_at":197},127277,"3.1.0","2023-07-31T06:50:14",{"id":199,"version":200,"summary_zh":201,"released_at":202},127278,"OpenChat-v1","这是我们OpenChat系列模型的v1版本。","2023-07-07T00:50:30",{"id":204,"version":205,"summary_zh":79,"released_at":206},127279,"pre-release","2023-07-02T10:19:05"]