[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"similar-Curated-Awesome-Lists--awesome-llms-fine-tuning":3,"tool-Curated-Awesome-Lists--awesome-llms-fine-tuning":64},[4,17,27,35,43,56],{"id":5,"name":6,"github_repo":7,"description_zh":8,"stars":9,"difficulty_score":10,"last_commit_at":11,"category_tags":12,"status":16},3808,"stable-diffusion-webui","AUTOMATIC1111\u002Fstable-diffusion-webui","stable-diffusion-webui 是一个基于 Gradio 构建的网页版操作界面，旨在让用户能够轻松地在本地运行和使用强大的 Stable Diffusion 图像生成模型。它解决了原始模型依赖命令行、操作门槛高且功能分散的痛点，将复杂的 AI 绘图流程整合进一个直观易用的图形化平台。\n\n无论是希望快速上手的普通创作者、需要精细控制画面细节的设计师，还是想要深入探索模型潜力的开发者与研究人员，都能从中获益。其核心亮点在于极高的功能丰富度：不仅支持文生图、图生图、局部重绘（Inpainting）和外绘（Outpainting）等基础模式，还独创了注意力机制调整、提示词矩阵、负向提示词以及“高清修复”等高级功能。此外，它内置了 GFPGAN 和 CodeFormer 等人脸修复工具，支持多种神经网络放大算法，并允许用户通过插件系统无限扩展能力。即使是显存有限的设备，stable-diffusion-webui 也提供了相应的优化选项，让高质量的 AI 艺术创作变得触手可及。",162132,3,"2026-04-05T11:01:52",[13,14,15],"开发框架","图像","Agent","ready",{"id":18,"name":19,"github_repo":20,"description_zh":21,"stars":22,"difficulty_score":23,"last_commit_at":24,"category_tags":25,"status":16},1381,"everything-claude-code","affaan-m\u002Feverything-claude-code","everything-claude-code 是一套专为 AI 编程助手（如 Claude Code、Codex、Cursor 等）打造的高性能优化系统。它不仅仅是一组配置文件，而是一个经过长期实战打磨的完整框架，旨在解决 AI 代理在实际开发中面临的效率低下、记忆丢失、安全隐患及缺乏持续学习能力等核心痛点。\n\n通过引入技能模块化、直觉增强、记忆持久化机制以及内置的安全扫描功能，everything-claude-code 能显著提升 AI 在复杂任务中的表现，帮助开发者构建更稳定、更智能的生产级 AI 代理。其独特的“研究优先”开发理念和针对 Token 消耗的优化策略，使得模型响应更快、成本更低，同时有效防御潜在的攻击向量。\n\n这套工具特别适合软件开发者、AI 研究人员以及希望深度定制 AI 工作流的技术团队使用。无论您是在构建大型代码库，还是需要 AI 协助进行安全审计与自动化测试，everything-claude-code 都能提供强大的底层支持。作为一个曾荣获 Anthropic 黑客大奖的开源项目，它融合了多语言支持与丰富的实战钩子（hooks），让 AI 真正成长为懂上",140436,2,"2026-04-05T23:32:43",[13,15,26],"语言模型",{"id":28,"name":29,"github_repo":30,"description_zh":31,"stars":32,"difficulty_score":23,"last_commit_at":33,"category_tags":34,"status":16},2271,"ComfyUI","Comfy-Org\u002FComfyUI","ComfyUI 是一款功能强大且高度模块化的视觉 AI 引擎，专为设计和执行复杂的 Stable Diffusion 图像生成流程而打造。它摒弃了传统的代码编写模式，采用直观的节点式流程图界面，让用户通过连接不同的功能模块即可构建个性化的生成管线。\n\n这一设计巧妙解决了高级 AI 绘图工作流配置复杂、灵活性不足的痛点。用户无需具备编程背景，也能自由组合模型、调整参数并实时预览效果，轻松实现从基础文生图到多步骤高清修复等各类复杂任务。ComfyUI 拥有极佳的兼容性，不仅支持 Windows、macOS 和 Linux 全平台，还广泛适配 NVIDIA、AMD、Intel 及苹果 Silicon 等多种硬件架构，并率先支持 SDXL、Flux、SD3 等前沿模型。\n\n无论是希望深入探索算法潜力的研究人员和开发者，还是追求极致创作自由度的设计师与资深 AI 绘画爱好者，ComfyUI 都能提供强大的支持。其独特的模块化架构允许社区不断扩展新功能，使其成为当前最灵活、生态最丰富的开源扩散模型工具之一，帮助用户将创意高效转化为现实。",107662,"2026-04-03T11:11:01",[13,14,15],{"id":36,"name":37,"github_repo":38,"description_zh":39,"stars":40,"difficulty_score":23,"last_commit_at":41,"category_tags":42,"status":16},3704,"NextChat","ChatGPTNextWeb\u002FNextChat","NextChat 是一款轻量且极速的 AI 助手，旨在为用户提供流畅、跨平台的大模型交互体验。它完美解决了用户在多设备间切换时难以保持对话连续性，以及面对众多 AI 模型不知如何统一管理的痛点。无论是日常办公、学习辅助还是创意激发，NextChat 都能让用户随时随地通过网页、iOS、Android、Windows、MacOS 或 Linux 端无缝接入智能服务。\n\n这款工具非常适合普通用户、学生、职场人士以及需要私有化部署的企业团队使用。对于开发者而言，它也提供了便捷的自托管方案，支持一键部署到 Vercel 或 Zeabur 等平台。\n\nNextChat 的核心亮点在于其广泛的模型兼容性，原生支持 Claude、DeepSeek、GPT-4 及 Gemini Pro 等主流大模型，让用户在一个界面即可自由切换不同 AI 能力。此外，它还率先支持 MCP（Model Context Protocol）协议，增强了上下文处理能力。针对企业用户，NextChat 提供专业版解决方案，具备品牌定制、细粒度权限控制、内部知识库整合及安全审计等功能，满足公司对数据隐私和个性化管理的高标准要求。",87618,"2026-04-05T07:20:52",[13,26],{"id":44,"name":45,"github_repo":46,"description_zh":47,"stars":48,"difficulty_score":23,"last_commit_at":49,"category_tags":50,"status":16},2268,"ML-For-Beginners","microsoft\u002FML-For-Beginners","ML-For-Beginners 是由微软推出的一套系统化机器学习入门课程，旨在帮助零基础用户轻松掌握经典机器学习知识。这套课程将学习路径规划为 12 周，包含 26 节精炼课程和 52 道配套测验，内容涵盖从基础概念到实际应用的完整流程，有效解决了初学者面对庞大知识体系时无从下手、缺乏结构化指导的痛点。\n\n无论是希望转型的开发者、需要补充算法背景的研究人员，还是对人工智能充满好奇的普通爱好者，都能从中受益。课程不仅提供了清晰的理论讲解，还强调动手实践，让用户在循序渐进中建立扎实的技能基础。其独特的亮点在于强大的多语言支持，通过自动化机制提供了包括简体中文在内的 50 多种语言版本，极大地降低了全球不同背景用户的学习门槛。此外，项目采用开源协作模式，社区活跃且内容持续更新，确保学习者能获取前沿且准确的技术资讯。如果你正寻找一条清晰、友好且专业的机器学习入门之路，ML-For-Beginners 将是理想的起点。",84991,"2026-04-05T10:45:23",[14,51,52,53,15,54,26,13,55],"数据工具","视频","插件","其他","音频",{"id":57,"name":58,"github_repo":59,"description_zh":60,"stars":61,"difficulty_score":10,"last_commit_at":62,"category_tags":63,"status":16},3128,"ragflow","infiniflow\u002Fragflow","RAGFlow 是一款领先的开源检索增强生成（RAG）引擎，旨在为大语言模型构建更精准、可靠的上下文层。它巧妙地将前沿的 RAG 技术与智能体（Agent）能力相结合，不仅支持从各类文档中高效提取知识，还能让模型基于这些知识进行逻辑推理和任务执行。\n\n在大模型应用中，幻觉问题和知识滞后是常见痛点。RAGFlow 通过深度解析复杂文档结构（如表格、图表及混合排版），显著提升了信息检索的准确度，从而有效减少模型“胡编乱造”的现象，确保回答既有据可依又具备时效性。其内置的智能体机制更进一步，使系统不仅能回答问题，还能自主规划步骤解决复杂问题。\n\n这款工具特别适合开发者、企业技术团队以及 AI 研究人员使用。无论是希望快速搭建私有知识库问答系统，还是致力于探索大模型在垂直领域落地的创新者，都能从中受益。RAGFlow 提供了可视化的工作流编排界面和灵活的 API 接口，既降低了非算法背景用户的上手门槛，也满足了专业开发者对系统深度定制的需求。作为基于 Apache 2.0 协议开源的项目，它正成为连接通用大模型与行业专有知识之间的重要桥梁。",77062,"2026-04-04T04:44:48",[15,14,13,26,54],{"id":65,"github_repo":66,"name":67,"description_en":68,"description_zh":69,"ai_summary_zh":70,"readme_en":71,"readme_zh":72,"quickstart_zh":73,"use_case_zh":74,"hero_image_url":75,"owner_login":76,"owner_name":77,"owner_avatar_url":78,"owner_bio":79,"owner_company":80,"owner_location":80,"owner_email":80,"owner_twitter":80,"owner_website":80,"owner_url":81,"languages":80,"stars":82,"forks":83,"last_commit_at":84,"license":80,"difficulty_score":23,"env_os":85,"env_gpu":86,"env_ram":85,"env_deps":87,"category_tags":90,"github_topics":91,"view_count":23,"oss_zip_url":80,"oss_zip_packed_at":80,"status":16,"created_at":102,"updated_at":103,"faqs":104,"releases":105},2303,"Curated-Awesome-Lists\u002Fawesome-llms-fine-tuning","awesome-llms-fine-tuning","Explore a comprehensive collection of resources, tutorials, papers, tools, and best practices for fine-tuning Large Language Models (LLMs). Perfect for ML practitioners and researchers!","awesome-llms-fine-tuning 是一个专为大型语言模型（LLM）微调打造的精选资源库。它汇聚了从基础教程、前沿论文到实用工具和最佳实践的全方位内容，旨在帮助用户将 GPT、BERT、Llama 等预训练模型高效适配到特定任务与领域。\n\n在人工智能快速发展的今天，通用大模型虽知识渊博，但往往难以直接满足垂直场景的精准需求。awesome-llms-fine-tuning 正是为了解决这一痛点而生，它通过整理高质量的学习路径和技术方案，让用户能够轻松掌握如何让模型理解专业术语、适应特定语境，从而释放模型的最大潜力。\n\n这份资源库特别适合机器学习从业者、数据科学家以及希望深入探索 LLM 的研究人员。无论你是刚入门的新手，还是寻求技术突破的资深专家，都能在这里找到有价值的指引。其独特亮点在于不仅涵盖了 AutoTrain、LLaMA-Factory 等支持无代码或高效微调的热门开源项目，还系统性地分类了课程、书籍、会议演讲甚至播客等多种形式的学习材料。通过整合这些分散的技术资源，awesome-llms-fine-tuning 为用户构建了一条清晰、平滑的微调学习与实践之路","awesome-llms-fine-tuning 是一个专为大型语言模型（LLM）微调打造的精选资源库。它汇聚了从基础教程、前沿论文到实用工具和最佳实践的全方位内容，旨在帮助用户将 GPT、BERT、Llama 等预训练模型高效适配到特定任务与领域。\n\n在人工智能快速发展的今天，通用大模型虽知识渊博，但往往难以直接满足垂直场景的精准需求。awesome-llms-fine-tuning 正是为了解决这一痛点而生，它通过整理高质量的学习路径和技术方案，让用户能够轻松掌握如何让模型理解专业术语、适应特定语境，从而释放模型的最大潜力。\n\n这份资源库特别适合机器学习从业者、数据科学家以及希望深入探索 LLM 的研究人员。无论你是刚入门的新手，还是寻求技术突破的资深专家，都能在这里找到有价值的指引。其独特亮点在于不仅涵盖了 AutoTrain、LLaMA-Factory 等支持无代码或高效微调的热门开源项目，还系统性地分类了课程、书籍、会议演讲甚至播客等多种形式的学习材料。通过整合这些分散的技术资源，awesome-llms-fine-tuning 为用户构建了一条清晰、平滑的微调学习与实践之路，让复杂的模型定制工作变得更加井然有序。","# Awesome LLMs Fine-Tuning\n\nWelcome to the curated collection of resources for fine-tuning Large Language Models (LLMs) like GPT, BERT, RoBERTa, and their numerous variants! In this era of artificial intelligence, the ability to adapt pre-trained models to specific tasks and domains has become an indispensable skill for researchers, data scientists, and machine learning practitioners.\n\nLarge Language Models, trained on massive datasets, capture an extensive range of knowledge and linguistic nuances. However, to unleash their full potential in specific applications, fine-tuning them on targeted datasets is paramount. This process not only enhances the models’ performance but also ensures that they align with the particular context, terminology, and requirements of the task at hand.\n\nIn this awesome list, we have meticulously compiled a range of resources, including tutorials, papers, tools, frameworks, and best practices, to aid you in your fine-tuning journey. Whether you are a seasoned practitioner looking to expand your expertise or a beginner eager to step into the world of LLMs, this repository is designed to provide valuable insights and guidelines to streamline your endeavors.\n\n## Table of Contents\n\n- [GitHub projects](#github-projects)\n- [Articles & Blogs](#articles--blogs)\n- [Online Courses](#online-courses)\n- [Books](#books)\n- [Research Papers](#research-papers)\n- [Videos](#videos)\n- [Tools & Software](#tools-&-software)\n- [Conferences & Events](#conferences-&-events)\n- [Slides & Presentations](#slides-&-presentations)\n- [Podcasts](#podcasts)\n\n## GitHub projects\n- [AutoTrain](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fautotrain-advanced) - No code fine-tuning of LLMs and other machine learning tasks.\n- [LlamaIndex](https:\u002F\u002Fgithub.com\u002Frun-llama\u002Fllama_index) 🦙: A data framework for your LLM applications. (23010 stars)\n- [Petals](https:\u002F\u002Fgithub.com\u002Fbigscience-workshop\u002Fpetals) 🌸: Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading. (7768 stars)\n- [LLaMA-Factory](https:\u002F\u002Fgithub.com\u002Fhiyouga\u002FLLaMA-Factory): An easy-to-use LLM fine-tuning framework (LLaMA-2, BLOOM, Falcon, Baichuan, Qwen, ChatGLM3). (5532 stars)\n- [lit-gpt](https:\u002F\u002Fgithub.com\u002FLightning-AI\u002Flit-gpt): Hackable implementation of state-of-the-art open-source LLMs based on nanoGPT. Supports flash attention, 4-bit and 8-bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed. (3469 stars)\n- [H2O LLM Studio](https:\u002F\u002Fgithub.com\u002Fh2oai\u002Fh2o-llmstudio): A framework and no-code GUI for fine-tuning LLMs. Documentation: [https:\u002F\u002Fh2oai.github.io\u002Fh2o-llmstudio\u002F](https:\u002F\u002Fh2oai.github.io\u002Fh2o-llmstudio\u002F) (2880 stars)\n- [Phoenix](https:\u002F\u002Fgithub.com\u002FArize-ai\u002Fphoenix): AI Observability & Evaluation - Evaluate, troubleshoot, and fine tune your LLM, CV, and NLP models in a notebook. (1596 stars)\n- [LLM-Adapters](https:\u002F\u002Fgithub.com\u002FAGI-Edgerunners\u002FLLM-Adapters): Code for the EMNLP 2023 Paper: \"LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models\". (769 stars)\n- [Platypus](https:\u002F\u002Fgithub.com\u002Farielnlee\u002FPlatypus): Code for fine-tuning Platypus fam LLMs using LoRA. (589 stars)\n- [xtuner](https:\u002F\u002Fgithub.com\u002FInternLM\u002Fxtuner): A toolkit for efficiently fine-tuning LLM (InternLM, Llama, Baichuan, QWen, ChatGLM2). (540 stars)\n- [DB-GPT-Hub](https:\u002F\u002Fgithub.com\u002Feosphoros-ai\u002FDB-GPT-Hub): A repository that contains models, datasets, and fine-tuning techniques for DB-GPT, with the purpose of enhancing model performance, especially in Text-to-SQL, and achieved higher exec acc than GPT-4 in spider eval with 13B LLM used this project. (422 stars)\n- [LLM-Finetuning-Hub](https:\u002F\u002Fgithub.com\u002Fgeorgian-io\u002FLLM-Finetuning-Hub) : Repository that contains LLM fine-tuning and deployment scripts along with our research findings. :star: 416\n- [Finetune_LLMs](https:\u002F\u002Fgithub.com\u002Fmallorbc\u002FFinetune_LLMs) : Repo for fine-tuning Casual LLMs. :star: 391\n- [MFTCoder](https:\u002F\u002Fgithub.com\u002Fcodefuse-ai\u002FMFTCoder) : High Accuracy and efficiency multi-task fine-tuning framework for Code LLMs; 业内首个高精度、高效率、多任务、多模型支持、多训练算法，大模型代码能力微调框架. :star: 337\n- [llmware](https:\u002F\u002Fgithub.com\u002Fllmware-ai\u002Fllmware) : Providing enterprise-grade LLM-based development framework, tools, and fine-tuned models. :star: 289\n- [LLM-Kit](https:\u002F\u002Fgithub.com\u002Fwpydcr\u002FLLM-Kit) : 🚀WebUI integrated platform for latest LLMs | 各大语言模型的全流程工具 WebUI 整合包。支持主流大模型API接口和开源模型。支持知识库，数据库，角色扮演，mj文生图，LoRA和全参数微调，数据集制作，live2d等全流程应用工具. :star: 232\n- [h2o-wizardlm](https:\u002F\u002Fgithub.com\u002Fh2oai\u002Fh2o-wizardlm) : Open-Source Implementation of WizardLM to turn documents into Q:A pairs for LLM fine-tuning. :star: 228\n- [hcgf](https:\u002F\u002Fgithub.com\u002Fhscspring\u002Fhcgf) : Humanable Chat Generative-model Fine-tuning | LLM微调. :star: 196\n- [llm_qlora](https:\u002F\u002Fgithub.com\u002Fgeorgesung\u002Fllm_qlora) : Fine-tuning LLMs using QLoRA. :star: 136\n- [awesome-llm-human-preference-datasets](https:\u002F\u002Fgithub.com\u002Fglgh\u002Fawesome-llm-human-preference-datasets) : A curated list of Human Preference Datasets for LLM fine-tuning, RLHF, and eval. :star: 124\n- [llm_finetuning](https:\u002F\u002Fgithub.com\u002Ftaprosoft\u002Fllm_finetuning) : Convenient wrapper for fine-tuning and inference of Large Language Models (LLMs) with several quantization techniques (GTPQ, bitsandbytes). :star: 114\n\n## Articles & Blogs\n- [Fine-Tune LLMs in 2024 with Hugging Face: TRL and Flash Attention](https:\u002F\u002Fwww.philschmid.de\u002Ffine-tune-llms-in-2024-with-trl) 🤗: This blog post provides a comprehensive guide to fine-tune LLMs (e.g., Llama 2), using hugging face trl and flash attention on consumer size GPUs (24GB).\n- [Complete Guide to LLM Fine Tuning for Beginners](https:\u002F\u002Fmedium.com\u002F@mayaakim\u002Fcomplete-guide-to-llm-fine-tuning-for-beginners-c2c38a3252be) 📚: A comprehensive guide that explains the process of fine-tuning a pre-trained model for new tasks, covering key concepts and providing a concrete example.\n- [Fine-Tuning Large Language Models (LLMs)](https:\u002F\u002Ftowardsdatascience.com\u002Ffine-tuning-large-language-models-llms-23473d763b91) 📖: This blog post presents an overview of fine-tuning pre-trained LLMs, discussing important concepts and providing a practical example with Python code.\n- [Creating a Domain Expert LLM: A Guide to Fine-Tuning](https:\u002F\u002Fhackernoon.com\u002Fcreating-a-domain-expert-llm-a-guide-to-fine-tuning) 📝: An article that dives into the concept of fine-tuning using OpenAI's API, showcasing an example of fine-tuning a large language model for understanding the plot of a Handel opera.\n- [A Beginner's Guide to LLM Fine-Tuning](https:\u002F\u002Ftowardsdatascience.com\u002Fa-beginners-guide-to-llm-fine-tuning-4bae7d4da672) 🌱: A guide that covers the process of fine-tuning LLMs, including the use of tools like QLoRA for configuring and fine-tuning models.\n- [Knowledge Graphs & LLMs: Fine-Tuning Vs. Retrieval-Augmented Generation](https:\u002F\u002Fmedium.com\u002Fneo4j\u002Fknowledge-graphs-llms-fine-tuning-vs-retrieval-augmented-generation-30e875d63a35) 📖: This blog post explores the limitations of LLMs and provides insights into fine-tuning them in conjunction with knowledge graphs.\n- [Fine-tune an LLM on your personal data: create a “The Lord of the Rings” storyteller](https:\u002F\u002Fmedium.com\u002F@jeremyarancio\u002Ffine-tune-an-llm-on-your-personal-data-create-a-the-lord-of-the-rings-storyteller-6826dd614fa9) ✏️: An article that demonstrates how to train your own LLM on personal data, offering control over personal information without relying on OpenAI's GPT-4.\n- [Fine-tuning an LLM model with H2O LLM Studio to generate Cypher statements](https:\u002F\u002Ftowardsdatascience.com\u002Ffine-tuning-an-llm-model-with-h2o-llm-studio-to-generate-cypher-statements-3f34822ad5) 🧪: This blog post provides an example of fine-tuning an LLM model using H2O LLM Studio for generating Cypher statements, enabling chatbot applications with knowledge graphs.\n- [Fine-Tune Your Own Llama 2 Model in a Colab Notebook](https:\u002F\u002Ftowardsdatascience.com\u002Ffine-tune-your-own-llama-2-model-in-a-colab-notebook-df9823a04a32) 📝: A practical introduction to LLM fine-tuning, demonstrating how to implement it in a Google Colab notebook to create your own Llama 2 model.\n- [Thinking about fine-tuning a LLM? Here's 3 considerations before you get started](https:\u002F\u002Ftowardsdatascience.com\u002Fthinking-about-fine-tuning-an-llm-heres-3-considerations-before-you-get-started-c1f483f293) 💡: This article discusses three ideas to consider when fine-tuning LLMs, including ways to improve GPT beyond PEFT and LoRA, and the importance of investing resources wisely.\n- [Introduction to LLMs and the generative AI : Part 3—Fine Tuning LLM with instruction](https:\u002F\u002Fmedium.com\u002F@yash9439\u002Fintroduction-to-llms-and-the-generative-ai-part-3-fine-tuning-llm-with-instruction-and-326bc95e07ae) 📚: This article explores the role of LLMs in artificial intelligence applications and provides an overview of fine-tuning them.\n- [RAG vs Finetuning — Which Is the Best Tool to Boost Your LLM Application](https:\u002F\u002Ftowardsdatascience.com\u002Frag-vs-finetuning-which-is-the-best-tool-to-boost-your-llm-application-94654b1eaba7) - A blog post discussing the aspects to consider when building LLM applications and choosing the right method for your use case. 👨‍💻\n- [Finetuning an LLM: RLHF and alternatives (Part I)](https:\u002F\u002Fmedium.com\u002Fmantisnlp\u002Ffinetuning-an-llm-rlhf-and-alternatives-part-i-2106b95c8087) - An article showcasing alternative methods to RLHF, specifically Direct Preference Optimization (DPO). 🔄\n- [When Should You Fine-Tune LLMs?](https:\u002F\u002Ftowardsdatascience.com\u002Fwhen-should-you-fine-tune-llms-2dddc09a404a) - Exploring the comparison between fine-tuning open-source LLMs and using a closed API for LLM Queries at Scale. 🤔\n- [Fine-Tuning Large Language Models](https:\u002F\u002Fcobusgreyling.medium.com\u002Ffine-tuning-large-language-models-f937869cef17) - Considering the fine-tuning of large language models and comparing it to zero and few shot approaches. 🎯\n- [Private GPT: Fine-Tune LLM on Enterprise Data](https:\u002F\u002Ftowardsdatascience.com\u002Fprivate-gpt-fine-tune-llm-on-enterprise-data-7e663d808e6a) - Exploring training techniques that allow fine-tuning LLMs on smaller GPUs. 🖥️\n- [Fine-tune Google PaLM 2 with Scikit-LLM](https:\u002F\u002Fmedium.com\u002F@iryna230520\u002Ffine-tune-google-palm-2-with-scikit-llm-d41b0aa673a5) - Demonstrating how to fine-tune Google PaLM 2, the most advanced LLM from Google, using Scikit-LLM. 📈\n- [A Deep-Dive into Fine-Tuning of Large Language Models](https:\u002F\u002Frpradeepmenon.medium.com\u002Fa-deep-dive-into-fine-tuning-of-large-language-models-96f7029ac0e1) - A comprehensive blog on fine-tuning LLMs like GPT-4 & BERT, providing insights, trends, and benefits. 🚀\n- [Pre-training, fine-tuning and in-context learning in Large Language Models](https:\u002F\u002Fmedium.com\u002F@atmabodha\u002Fpre-training-fine-tuning-and-in-context-learning-in-large-language-models-llms-dd483707b122) - Discussing the concepts of pre-training, fine-tuning, and in-context learning in LLMs. 📚\n- [List of Open Sourced Fine-Tuned Large Language Models](https:\u002F\u002Fsungkim11.medium.com\u002Flist-of-open-sourced-fine-tuned-large-language-models-llm-8d95a2e0dc76) - A curated list of open-sourced fine-tuned LLMs that can be run locally on your computer. 📋\n- [Practitioners guide to fine-tune LLMs for domain-specific use case](https:\u002F\u002Fcismography.medium.com\u002Fpractitioners-guide-to-fine-tune-llms-for-domain-specific-use-case-part-1-4561714d874f) - A guide covering key learnings and conclusions on fine-tuning LLMs for domain-specific use cases. 📝\n- [Finetune Llama 3.1 with a production stack on AWS, GCP or Azure](https:\u002F\u002Fwww.zenml.io\u002Fblog\u002Fhow-to-finetune-llama-3-1-with-zenml) - A guide and tutorial on finetuning Llama 3.1 ([or Phi 3.5](https:\u002F\u002Fwww.zenml.io\u002Fblog\u002Fhow-to-finetune-phi-3-5-with-zenml)) in a production setup designed for MLOps best practices. 📓\n\n## Online Courses\n\n- [Fine-Tuning Fundamentals: Unlocking the Potential of LLMs | Udemy](https:\u002F\u002Fwww.udemy.com\u002Fcourse\u002Fbuilding-chatgpt-style-agents\u002F): A practical course for beginners on building chatGPT-style models and adapting them for specific use cases.\n- [Generative AI with Large Language Models | Coursera](https:\u002F\u002Fwww.coursera.org\u002Flearn\u002Fgenerative-ai-with-llms): Learn the fundamentals of generative AI with LLMs and how to deploy them in practical applications. Enroll for free.\n- [Large Language Models: Application through Production | edX](https:\u002F\u002Fwww.edx.org\u002Flearn\u002Fcomputer-science\u002Fdatabricks-large-language-models-application-through-production): An advanced course for developers, data scientists, and engineers to build LLM-centric applications using popular frameworks and achieve end-to-end production readiness.\n- [Finetuning Large Language Models | Coursera Guided Project](https:\u002F\u002Fwww.coursera.org\u002Fprojects\u002Ffinetuning-large-language-models-project): A short guided project that covers essential finetuning concepts and training of large language models.\n- [OpenAI & ChatGPT API's: Expert Fine-tuning for Developers | Udemy](https:\u002F\u002Fwww.udemy.com\u002Fcourse\u002Fmastering-chatgpt-models-from-fine-tuning-to-deployment-openai\u002F): Discover the power of GPT-3 in creating conversational AI solutions, including topics like prompt engineering, fine-tuning, integration, and deploying ChatGPT models.\n- [Large Language Models Professional Certificate | edX](https:\u002F\u002Fwww.edx.org\u002Fcertificates\u002Fprofessional-certificate\u002Fdatabricks-large-language-models): Learn how to build and productionize Large Language Model (LLM) based applications using the latest frameworks, techniques, and theory behind foundation models.\n- [Improving the Performance of Your LLM Beyond Fine Tuning | Udemy](https:\u002F\u002Fwww.udemy.com\u002Fcourse\u002Fimproving-the-performance-of-your-llm-beyond-fine-tuning\u002F): A course designed for business leaders and developers interested in fine-tuning LLM models and exploring techniques for improving their performance.\n- [Introduction to Large Language Models | Coursera](https:\u002F\u002Fwww.coursera.org\u002Flearn\u002Fintroduction-to-large-language-models): An introductory level micro-learning course offered by Google Cloud, explaining the basics of Large Language Models (LLMs) and their use cases. Enroll for free.\n- [Syllabus | LLM101x | edX](https:\u002F\u002Fcourses.edx.org\u002Fcourses\u002Fcourse-v1:Databricks+LLM101x+2T2023\u002Fc861b0726ce24e099ad80111145f4217\u002F): Learn how to use data embeddings, vector databases, and fine-tune LLMs with domain-specific data to augment LLM pipelines.\n- [Performance Tuning Deep Learning Models Master Class | Udemy](https:\u002F\u002Fwww.udemy.com\u002Fcourse\u002Fperformance-tuning-deep-learning-models-master-class\u002F): A master class on tuning deep learning models, covering techniques to accelerate learning and optimize performance.\n- [Best Large Language Models (LLMs) Courses & Certifications](https:\u002F\u002Fwww.coursera.org\u002Fcourses?query=large%20language%20models): Curated from top educational institutions and industry leaders, this selection of LLMs courses aims to provide quality training for individuals and corporate teams looking to learn or improve their skills in fine-tuning LLMs.\n- [Mastering Language Models: Unleashing the Power of LLMs](https:\u002F\u002Fwww.udemy.com\u002Fcourse\u002Fmastering-language-models-unleashing-the-power-of-llms\u002F): In this comprehensive course, you'll delve into the fundamental principles of NLP and explore how LLMs have reshaped the landscape of AI applications. A comprehensive guide to advanced NLP and LLMs.\n- [LLMs Mastery: Complete Guide to Transformers & Generative AI](https:\u002F\u002Fwww.udemy.com\u002Fcourse\u002Fllms-mastery-complete-guide-to-transformers-generative-ai\u002F): This course provides a great overview of AI history and covers fine-tuning the three major LLM models: BERT, GPT, and T5. Suitable for those interested in generative AI, LLMs, and production-level applications.\n- [Exploring The Technologies Behind ChatGPT, GPT4 & LLMs](https:\u002F\u002Fwww.udemy.com\u002Fcourse\u002Fexploring-the-technologies-behind-chatgpt-openai\u002F): The only course you need to learn about large language models like ChatGPT, GPT4, BERT, and more. Gain insights into the technologies behind these LLMs.\n- [Non-technical Introduction to Large Language Models](https:\u002F\u002Fwww.udemy.com\u002Fcourse\u002Fnon-technical-introduction-to-large-language-models\u002F): An overview of large language models for non-technical individuals, explaining the existing challenges and providing simple explanations without complex jargon.\n- [Large Language Models: Foundation Models from the Ground Up](https:\u002F\u002Fwww.edx.org\u002Flearn\u002Fcomputer-science\u002Fdatabricks-large-language-models-foundation-models-from-the-ground-up): Delve into the details of foundation models in LLMs, such as BERT, GPT, and T5. Gain an understanding of the latest advances that enhance LLM functionality.\n\n## Books\n\n- [Generative AI with Large Language Models — New Hands-on Course by Deeplearning.ai and AWS](https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Faws\u002Fgenerative-ai-with-large-language-models-new-hands-on-course-by-deeplearning-ai-and-aws\u002F)\n- A hands-on course that teaches how to fine-tune Large Language Models (LLMs) using reward models and reinforcement learning, with a focus on generative AI.\n- [From Data Selection To Fine Tuning: The Technical Guide To Constructing LLM Models](https:\u002F\u002Fwww.amazon.com\u002FData-Selection-Fine-Tuning-Constructing\u002Fdp\u002FB0CH2FLV2Q)\n- A technical guide that covers the process of constructing LLM models, from data selection to fine-tuning.\n- [The LLM Knowledge Cookbook: From, RAG, to QLoRA, to Fine Tuning, and all the Recipes In Between!](https:\u002F\u002Fwww.amazon.com\u002FLLM-Knowledge-Cookbook-Recipes-Between-ebook\u002Fdp\u002FB0CKH9B58N)\n- A comprehensive cookbook that explores various LLM models, including techniques like Retrieve and Generate (RAG) and Query Language Representation (QLoRA), as well as the fine-tuning process.\n- [Principles for Fine-tuning LLMs](https:\u002F\u002Fwww.packtpub.com\u002Farticle-hub\u002Fprinciples-for-fine-tuning-llms)\n- An article that demystifies the process of fine-tuning LLMs and explores different techniques, such as in-context learning, classic fine-tuning methods, parameter-efficient fine-tuning, and Reinforcement Learning with Human Feedback (RLHF).\n- [From Data Selection To Fine Tuning: The Technical Guide To Constructing LLM Models](https:\u002F\u002Fwww.goodreads.com\u002Fbook\u002Fshow\u002F198555505-from-data-selection-to-fine-tuning)\n- A technical guide that provides insights into building and training large language models (LLMs).\n- [Hands-On Large Language Models](https:\u002F\u002Fwww.oreilly.com\u002Flibrary\u002Fview\u002Fhands-on-large-language\u002F9781098150952\u002F)\n- A book that covers the advancements in language AI systems driven by deep learning, focusing on large language models.\n- [Fine-tune Llama 2 for text generation on Amazon SageMaker JumpStart](https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fmachine-learning\u002Ffine-tune-llama-2-for-text-generation-on-amazon-sagemaker-jumpstart\u002F)\n- Learn how to fine-tune Llama 2 models using Amazon SageMaker JumpStart for optimized dialogue generation.\n- [Fast and cost-effective LLaMA 2 fine-tuning with AWS Trainium](https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fmachine-learning\u002Ffast-and-cost-effective-llama-2-fine-tuning-with-aws-trainium\u002F)\n- A blog post that explains how to achieve fast and cost-effective fine-tuning of LLaMA 2 models using AWS Trainium.\n- [Fine-tuning - Advanced Deep Learning with Python [Book]](https:\u002F\u002Fwww.oreilly.com\u002Flibrary\u002Fview\u002Fadvanced-deep-learning\u002F9781789956177\u002Fbd91eb1d-87cf-463d-bbd3-4177a42f3da7.xhtml) 💡: A book that explores the fine-tuning task following the pretraining task in advanced deep learning with Python.\n- [The LLM Knowledge Cookbook: From, RAG, to QLoRA, to Fine ...](https:\u002F\u002Fwww.barnesandnoble.com\u002Fw\u002Fthe-llm-knowledge-cookbook-richard-anthony-aragon\u002F1144180729) 💡: A comprehensive guide to using large language models (LLMs) for various tasks, covering everything from the basics to advanced fine-tuning techniques.\n- [Quick Start Guide to Large Language Models: Strategies and Best ...](https:\u002F\u002Fwww.oreilly.com\u002Flibrary\u002Fview\u002Fquick-start-guide\u002F9780138199425\u002F) 💡: A guide focusing on strategies and best practices for large language models (LLMs) like BERT, T5, and ChatGPT, showcasing their unprecedented performance in various NLP tasks.\n- [4. Advanced GPT-4 and ChatGPT Techniques - Developing Apps ...](https:\u002F\u002Fwww.oreilly.com\u002Flibrary\u002Fview\u002Fdeveloping-apps-with\u002F9781098152475\u002Fch04.html) 💡: A chapter that dives into advanced techniques for GPT-4 and ChatGPT, including prompt engineering, zero-shot learning, few-shot learning, and task-specific fine-tuning.\n- [What are Large Language Models? - LLM AI Explained - AWS](https:\u002F\u002Faws.amazon.com\u002Fwhat-is\u002Flarge-language-model\u002F) 💡: An explanation of large language models (LLMs), discussing the concepts of few-shot learning and fine-tuning to improve model performance.\n\n## Research Papers\n\n- [LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning](https:\u002F\u002Farxiv.org\u002Fabs\u002F2304.01933) 📄: This paper presents LLM-Adapters, an easy-to-use framework that integrates various adapters into LLMs for parameter-efficient fine-tuning (PEFT) on different tasks.\n- [Two-stage LLM Fine-tuning with Less Specialization](https:\u002F\u002Farxiv.org\u002Fabs\u002F2211.00635) 📄: ProMoT, a two-stage fine-tuning framework, addresses the issue of format specialization in LLMs through Prompt Tuning with MOdel Tuning, improving their general in-context learning performance.\n- [Fine-tuning Large Enterprise Language Models via Ontological Reasoning](https:\u002F\u002Farxiv.org\u002Fabs\u002F2306.10723) 📄: This paper proposes a neurosymbolic architecture that combines Large Language Models (LLMs) with Enterprise Knowledge Graphs (EKGs) to achieve domain-specific fine-tuning of LLMs.\n- [QLoRA: Efficient Finetuning of Quantized LLMs](https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.14314) 📄: QLoRA is an efficient finetuning approach that reduces memory usage while preserving task performance, offering insights on quantized pretrained language models.\n- [Full Parameter Fine-tuning for Large Language Models with Limited Resources](https:\u002F\u002Farxiv.org\u002Fabs\u002F2306.09782) 📄: This work introduces LOMO, a low-memory optimization technique, enabling the full parameter fine-tuning of large LLMs with limited GPU resources.\n- [LoRA: Low-Rank Adaptation of Large Language Models](https:\u002F\u002Farxiv.org\u002Fabs\u002F2106.09685) 📄: LoRA proposes a methodology to adapt large pre-trained models to specific tasks by injecting trainable rank decomposition matrices into each layer, reducing the number of trainable parameters while maintaining model quality.\n- [Enhancing LLM with Evolutionary Fine Tuning for News Summary Generation](https:\u002F\u002Farxiv.org\u002Fabs\u002F2307.02839) 📄: This paper presents a new paradigm for news summary generation using LLMs, incorporating genetic algorithms and powerful natural language understanding capabilities.\n- [How do languages influence each other? Studying cross-lingual data sharing during LLM fine-tuning](https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.13286) 📄: This study investigates cross-lingual data sharing during fine-tuning of multilingual large language models (MLLMs) and analyzes the influence of different languages on model performance.\n- [Fine-Tuning Language Models with Just Forward Passes](https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.17333) 📄: MeZO, a memory-efficient zeroth-order optimizer, enables fine-tuning of large language models while significantly reducing the memory requirements.\n- [Learning to Reason over Scene Graphs: A Case Study of Finetuning LLMs](https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.07716) 📄: This work explores the applicability of GPT-2 LLMs in robotic task planning, demonstrating the potential for using LLMs in long-horizon task planning scenarios.\n- [Privately Fine-Tuning Large Language Models with](https:\u002F\u002Farxiv.org\u002Fabs\u002F2210.15042): This paper explores the application of differential privacy to add privacy guarantees to fine-tuning large language models (LLMs).\n- [DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Systems](http:\u002F\u002Farxiv.org\u002Fabs\u002F2309.11325): This paper presents DISC-LawLLM, an intelligent legal system that utilizes fine-tuned LLMs with legal reasoning capability to provide a wide range of legal services.\n- [Multi-Task Instruction Tuning of LLaMa for Specific Scenarios: A](https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.13225): The paper investigates the effectiveness of fine-tuning LLaMa, a foundational LLM, on specific writing tasks, demonstrating significant improvement in writing abilities.\n- [Training language models to follow instructions with human feedback](https:\u002F\u002Farxiv.org\u002Fabs\u002F2203.02155): This paper proposes a method to align language models with user intent by fine-tuning them using human feedback, resulting in models preferred over larger models in human evaluations.\n- [Large Language Models Can Self-Improve](https:\u002F\u002Farxiv.org\u002Fabs\u002F2210.11610): The paper demonstrates that LLMs can self-improve their reasoning abilities by fine-tuning using self-generated solutions, achieving state-of-the-art performance without ground truth labels.\n- [Embracing Large Language Models for Medical Applications](https:\u002F\u002Fwww.ncbi.nlm.nih.gov\u002Fpmc\u002Farticles\u002FPMC10292051\u002F): This paper highlights the potential of fine-tuned LLMs in medical applications, improving diagnostic accuracy and supporting clinical decision-making.\n- [Scaling Instruction-Finetuned Language Models](https:\u002F\u002Farxiv.org\u002Fabs\u002F2210.11416): The paper explores instruction fine-tuning on LLMs, demonstrating significant improvements in performance and generalization to unseen tasks.\n- [Federated Fine-tuning of Billion-Sized Language Models across](https:\u002F\u002Farxiv.org\u002Fabs\u002F2308.13894): This work introduces FwdLLM, a federated learning protocol designed to enhance the efficiency of fine-tuning large LLMs on mobile devices, improving memory and time efficiency.\n- [A Comprehensive Overview of Large Language Models](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2307.06435): This paper provides an overview of the development and applications of large language models and their transfer learning capabilities.\n- [Fine-tuning language models to find agreement among humans with](https:\u002F\u002Farxiv.org\u002Fabs\u002F2211.15006): The paper explores the fine-tuning of a large LLM to generate consensus statements that maximize approval for a group of people with diverse opinions.\n\n## Videos\n- [Intro to Large Language Models by Andrej Karpathy](https:\u002F\u002Fyoutu.be\u002FzjkBMFhNj_g?si=5S9tI-G2AD7xUuhf): This is a 1 hour introduction to Large Language Models. What they are, where they are headed, comparisons and analogies to present-day operating systems, and some of the security-related challenges of this new computing paradigm.\n- [Fine-tuning Llama 2 on Your Own Dataset | Train an LLM for Your ...](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=MDA3LUKNl1E): Learn how to fine-tune Llama 2 model on a custom dataset.\n- [Fine-tuning LLM with QLoRA on Single GPU: Training Falcon-7b on ...](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=DcBC4yGHV4Q): This video demonstrates the fine-tuning process of the Falcon 7b LLM using QLoRA.\n- [Fine-tuning an LLM using PEFT | Introduction to Large Language ...](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=6SpZkwWuldU): Discover how to fine-tune an LLM using PEFT, a technique that requires fewer resources.\n- [LLAMA-2 Open-Source LLM: Custom Fine-tuning Made Easy on a ...](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=8cc4bJtycOA): A step-by-step guide on how to fine-tune the LLama 2 LLM model on your custom dataset.\n- [New Course: Finetuning Large Language Models - YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=9PxhCekQYNI): This video introduces a course on fine-tuning LLMs, covering model selection, data preparation, training, and evaluation.\n- [Q: How to create an Instruction Dataset for Fine-tuning my LLM ...](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=BJQrQT2Xfyo): In this tutorial, beginners learn about fine-tuning LLMs, including when, how, and why to do it.\n- [LLM Module 4: Fine-tuning and Evaluating LLMs | 4.13.1 Notebook ...](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=2vEpMb4ofVU): A notebook demo on fine-tuning and evaluating LLMs.\n- [Google LLM Fine-Tuning\u002FAdapting\u002FCustomizing - Getting Started ...](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=cJ96rqW8L84): Get started with fine-tuning Google's PaLM 2 large language model through a step-by-step guide.\n- [Pretraining vs Fine-tuning vs In-context Learning of LLM (GPT-x ...](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=_FYwnO_g-4E): An ultimate guide explaining pretraining, fine-tuning, and in-context learning of LLMs like GPT-x.\n- [How to Fine-Tune an LLM with a PDF - Langchain Tutorial - YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=bOS929yCkGE): Learn how to fine-tune OpenAI's GPT LLM to process PDF documents using Langchain and PDF libraries.\n- [EasyTune Walkthrough - YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=yDYgbKYNk2I) - A walkthrough of fine-tuning LLM with QLoRA on a single GPU using Falcon-7b.\n- [Unlocking the Potential of ChatGPT Lessons in Training and Fine ...](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=Szi0NgmVg7Y) - THE STUDENT presents the instruction fine-tuning and in-context learning of LLMs with symbols.\n- [AI News: Creating LLMs without code! - YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=MRXj1pAB6cI) - Maya Akim discusses the top 5 LLM fine-tuning use cases you need to know.\n- [Top 5 LLM Fine-Tuning Use Cases You Need to Know - YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=v1R8uPqjNAU) - An in-depth video highlighting the top 5 LLM fine-tuning use cases with additional links for further exploration.\n- [clip2 llm emory - YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=UXBA7FKIUso) - Learn how to fine-tune Llama 2 on your own dataset and train an LLM for your specific use case.\n- [The EASIEST way to finetune LLAMA-v2 on a local machine! - YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=3fsn19OI_C8) - A step-by-step video guide demonstrating the easiest, simplest, and fastest way to fine-tune LLAMA-v2 on your local machine for a custom dataset.\n- [Training & Fine-Tuning LLMs: Introduction - YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=2QRlvKSzyVw) - An introduction to training and fine-tuning LLMs, including important concepts and the NeurIPS LLM Efficiency Challenge.\n- [Fine-tuning LLMs with PEFT and LoRA - YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=Us5ZFp16PaU) - A comprehensive video exploring how to use PEFT to fine-tune any decoder-style GPT model, including the basics of LoRA fine-tuning and uploading.\n- [Building and Curating Datasets for RLHF and LLM Fine-tuning ...](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=Ezz_5csCJqI) - Learn about building and curating datasets for RLHF (Reinforcement Learning from Human Feedback) and LLM (Large Language Model) fine-tuning, with sponsorship by Argilla.\n- [Fine Tuning LLM (OpenAI GPT) with Custom Data in Python - YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=6aOzoJKNLKQ) - Explore how to extend LLM (OpenAI GPT) by fine-tuning it with a custom dataset to provide Q&A, summary, and other ChatGPT-like functions.\n\n## Tools & Software\n\n- [LLaMA Efficient Tuning](https:\u002F\u002Fsourceforge.net\u002Fprojects\u002Fllama-efficient-tuning.mirror\u002F) 🛠️: Easy-to-use LLM fine-tuning framework (LLaMA-2, BLOOM, Falcon).\n- [H2O LLM Studio](https:\u002F\u002Fsourceforge.net\u002Fprojects\u002Fh2o-llm-studio.mirror\u002F) 🛠️: Framework and no-code GUI for fine-tuning LLMs.\n- [PEFT](https:\u002F\u002Fsourceforge.net\u002Fprojects\u002Fpeft.mirror\u002F) 🛠️: Parameter-Efficient Fine-Tuning (PEFT) methods for efficient adaptation of pre-trained language models to downstream applications.\n- [ChatGPT-like model](https:\u002F\u002Fsourceforge.net\u002Fdirectory\u002Flarge-language-models-llm\u002Fc\u002F) 🛠️: Run a fast ChatGPT-like model locally on your device.\n- [Petals](https:\u002F\u002Fsourceforge.net\u002Fprojects\u002Fpetals.mirror\u002F): Run large language models like BLOOM-176B collaboratively, allowing you to load a small part of the model and team up with others for inference or fine-tuning. 🌸\n- [NVIDIA NeMo](https:\u002F\u002Fsourceforge.net\u002Fdirectory\u002Flarge-language-models-llm\u002Flinux\u002F): A toolkit for building state-of-the-art conversational AI models and specifically designed for Linux. 🚀\n- [H2O LLM Studio](https:\u002F\u002Fsourceforge.net\u002Fdirectory\u002Flarge-language-models-llm\u002Fwindows\u002F): A framework and no-code GUI tool for fine-tuning large language models on Windows. 🎛️\n- [Ludwig AI](https:\u002F\u002Fsourceforge.net\u002Fprojects\u002Fludwig-ai.mirror\u002F): A low-code framework for building custom LLMs and other deep neural networks. Easily train state-of-the-art LLMs with a declarative YAML configuration file. 🤖\n- [bert4torch](https:\u002F\u002Fsourceforge.net\u002Fprojects\u002Fbert4torch.mirror\u002F): An elegant PyTorch implementation of transformers. Load various open-source large model weights for reasoning and fine-tuning. 🔥\n- [Alpaca.cpp](https:\u002F\u002Fsourceforge.net\u002Fprojects\u002Falpaca-cpp.mirror\u002F): Run a fast ChatGPT-like model locally on your device. A combination of the LLaMA foundation model and an open reproduction of Stanford Alpaca for instruction-tuned fine-tuning. 🦙\n- [promptfoo](https:\u002F\u002Fsourceforge.net\u002Fprojects\u002Fpromptfoo.mirror\u002F): Evaluate and compare LLM outputs, catch regressions, and improve prompts using automatic evaluations and representative user inputs. 📊\n\n## Conferences & Events\n\n- [ML\u002FAI Conversation: Neuro-Symbolic AI - an Alternative to LLM](https:\u002F\u002Fwww.meetup.com\u002Fnew-york-ai-ml-conversations\u002Fevents\u002F296127633\u002F) - This meetup will discuss the experience with fine-tuning LLMs and explore neuro-symbolic AI as an alternative.\n- [AI Dev Day - Seattle, Mon, Oct 30, 2023, 5:00 PM](https:\u002F\u002Fwww.meetup.com\u002Fgdgcloudseattle\u002Fevents\u002F296959536\u002F) - A tech talk on effective LLM observability and fine-tuning opportunities using vector similarity search.\n- [DeepLearning.AI Events](https:\u002F\u002Fwww.eventbrite.com\u002Fo\u002Fdeeplearningai-19822694300) - A series of events including mitigating LLM hallucinations, fine-tuning LLMs with PyTorch 2.0 and ChatGPT, and AI education programs.\n- [AI Dev Day - New York, Thu, Oct 26, 2023, 5:30 PM](https:\u002F\u002Fwww.meetup.com\u002Fbig-data\u002Fevents\u002F296665127\u002F) - Tech talks on best practices in GenAI applications and using LLMs for real-time, personalized notifications.\n- [Chat LLMs & AI Agents - Use Gen AI to Build AI Systems and Agents](https:\u002F\u002Fwww.meetup.com\u002Ftel-aviv-ai-tech-talks\u002Fevents\u002F296739549\u002F) - An event focusing on LLMs, AI agents, and chain data, with opportunities for interaction through event chat.\n- [NYC AI\u002FLLM\u002FChatGPT Developers Group](https:\u002F\u002Fwww.meetup.com\u002Fnyc-llm-talks\u002F) - Regular tech talks\u002Fworkshops for developers interested in AI, LLMs, ChatGPT, NLP, ML, Data, etc.\n- [Leveraging LLMs for Enterprise Data, Tue, Nov 14, 2023, 2:00 PM](https:\u002F\u002Fwww.meetup.com\u002Fdata-science-dojo-new-york\u002Fevents\u002F296708155\u002F) - Dive into essential LLM strategies tailored for non-public data applications, including prompt engineering and retrieval.\n- [Bellevue Applied Machine Learning Meetup](https:\u002F\u002Fwww.meetup.com\u002Fbellevue-applied-machine-learning-meetup\u002F) - A meetup focusing on applied machine learning techniques and improving the skills of data scientists and ML practitioners.\n- [AI & Prompt Engineering Meetup Munich, Do., 5. Okt. 2023, 18:15](https:\u002F\u002Fwww.meetup.com\u002Fde-DE\u002Fai-prompt-engineering-munich\u002Fevents\u002F295437909\u002F) - Introduce H2O LLM Studio for fine-tuning LLMs and bring together AI enthusiasts from various backgrounds.\n- [Seattle AI\u002FML\u002FData Developers Group](https:\u002F\u002Fwww.meetup.com\u002Faittg-seattle\u002F) - Tech talks on evaluating LLM agents and learning AI\u002FML\u002FData through practice.\n- [Data Science Dojo - DC | Meetup](https:\u002F\u002Fwww.meetup.com\u002Fdata-science-dojo-washington-dc\u002F): This is a DC-based meetup group for business professionals interested in teaching, learning, and sharing knowledge and understanding of data science.\n- [Find Data Science Events & Groups in Dubai, AE](https:\u002F\u002Fwww.meetup.com\u002Ffind\u002Fae--dubai\u002Fdata-science\u002F): Discover data science events and groups in Dubai, AE, to connect with people who share your interests.\n- [AI Meetup (in-person): Generative AI and LLMs - Halloween Edition](https:\u002F\u002Fwww.meetup.com\u002Fdc-ai-llms\u002Fevents\u002F296543682\u002F): Join this AI meetup for a tech talk about generative AI and Large Language Models (LLMs), including open-source tools and best practices.\n- [ChatGPT Unleashed: Live Demo and Best Practices for NLP](https:\u002F\u002Fwww.meetup.com\u002Fdata-science-dojo-karachi\u002Fevents\u002F296977810\u002F): This online event explores fine-tuning hacks for Large Language Models and showcases the practical applications of ChatGPT and LLMs.\n- [Find Data Science Events & Groups in Pune, IN](https:\u002F\u002Fwww.meetup.com\u002Ffind\u002Fin--pune\u002Fdata-science\u002F): Explore online or in-person events and groups related to data science in Pune, IN.\n- [DC AI\u002FML\u002FData Developers Group | Meetup](https:\u002F\u002Fwww.meetup.com\u002Faidev-dc\u002F): This group aims to bring together AI enthusiasts in the D.C. area to learn and practice AI technologies, including AI, machine learning, deep learning, and data science.\n- [Boston AI\u002FLLMs\u002FChatGPT Developers Group | Meetup](https:\u002F\u002Fwww.meetup.com\u002Fbostondeeplearningai\u002F): Join this group in Boston to learn and practice AI technologies like LLMs, ChatGPT, machine learning, deep learning, and data science.\n- [Paris NLP | Meetup](https:\u002F\u002Fwww.meetup.com\u002Fparis-nlp\u002F): This meetup focuses on applications of natural language processing (NLP) in various fields, discussing techniques, research, and applications of both traditional and modern NLP approaches.\n- [SF AI\u002FLLMs\u002FChatGPT Developers Group | Meetup](https:\u002F\u002Fwww.meetup.com\u002Fsan-francisco-ai-llms\u002F): Connect with AI enthusiasts in the San Francisco\u002FBay area to learn and practice AI tech, including LLMs, ChatGPT, NLP, machine learning, deep learning, and data science.\n- [AI meetup (In-person): GenAI and LLMs for Health](https:\u002F\u002Fwww.meetup.com\u002Faittg-boston\u002Fevents\u002F296567040\u002F): Attend this tech talk about the application of LLMs in healthcare and learn about quick wins in using LLMs for health-related tasks.\n\n## Slides & Presentations\n\n- [Fine tuning large LMs](https:\u002F\u002Fwww.slideshare.net\u002FSylvainGugger\u002Ffine-tuning-large-lms-243430468): Presentation discussing the process of fine-tuning large language models like GPT, BERT, and RoBERTa.\n- [LLaMa 2.pptx](https:\u002F\u002Fwww.slideshare.net\u002FRkRahul16\u002Fllama-2pptx): Slides introducing LLaMa 2, a powerful large language model successor developed by Meta AI.\n- [LLM.pdf](https:\u002F\u002Fwww.slideshare.net\u002FMedBelatrach\u002Fllmpdf-261239806): Presentation exploring the role of Transformers in NLP, from BERT to GPT-3.\n- [Large Language Models Bootcamp](https:\u002F\u002Fwww.slideshare.net\u002FDataScienceDojo\u002Flarge-language-models-bootcamp): Bootcamp slides covering various aspects of large language models, including training from scratch and fine-tuning.\n- [The LHC Explained by CNN](https:\u002F\u002Fwww.slideshare.net\u002Fhijiki_s\u002Fthe-lhc-explained-by-cnn): Slides explaining the LHC (Large Hadron Collider) using CNN and fine-tuning image models.\n- [Using Large Language Models in 10 Lines of Code](https:\u002F\u002Fwww.slideshare.net\u002FGautierMarti\u002Fusing-large-language-models-in-10-lines-of-code): Presentation demonstrating how to use large language models in just 10 lines of code.\n- [LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention.pdf](https:\u002F\u002Fwww.slideshare.net\u002FjacksonChen22\u002Fllamaadapter-efficient-finetuning-of-language-models-with-zeroinit-attentionpdf): Slides discussing LLaMA-Adapter, an efficient technique for fine-tuning language models with zero-init attention.\n- [Intro to LLMs](https:\u002F\u002Fwww.slideshare.net\u002FLoicMerckel\u002Fintro-to-llms): Presentation providing an introduction to large language models, including base models and fine-tuning with prompt-completion pairs.\n- [LLM Fine-Tuning (東大松尾研LLM講座 Day5資料) - Speaker Deck](https:\u002F\u002Fspeakerdeck.com\u002Fschulta\u002Fllm-fine-tuning-dong-da-song-wei-yan-llmjiang-zuo-day5zi-liao): Slides used for a lecture on fine-tuning large language models, specifically for the 東大松尾研サマースクール2023.\n- [Automate your Job and Business with ChatGPT #3](https:\u002F\u002Fpt.slideshare.net\u002FAnantCorp\u002Fautomate-your-job-and-business-with-chatgpt-3-fundamentals-of-llmgpt): Presentation discussing the fundamentals of ChatGPT and its applications for job automation and business tasks.\n- [Unlocking the Power of Generative AI An Executive's Guide.pdf](https:\u002F\u002Fwww.slideshare.net\u002FPremNaraindas1\u002Funlocking-the-power-of-generative-ai-an-executives-guidepdf) - A guide that explains the process of fine-tuning Large Language Models (LLMs) to tailor them to an organization's needs.\n- [Fine tune and deploy Hugging Face NLP models | PPT](https:\u002F\u002Fwww.slideshare.net\u002Fovhcom\u002Fpres-hugging-facefinetuning) - A presentation that provides insights on how to build and deploy LLM models using Hugging Face NLP.\n- [大規模言語モデル時代のHuman-in-the-Loop機械学習 - Speaker Deck](https:\u002F\u002Fspeakerdeck.com\u002Fyukinobaba\u002Fhuman-in-the-loop-ml-llm) - A slide deck discussing the process of fine-tuning Language Models to find agreement among humans with diverse preferences.\n- [AI and ML Series - Introduction to Generative AI and LLMs | PPT](https:\u002F\u002Fwww.slideshare.net\u002FDianaGray10\u002Fai-and-ml-series-introduction-to-generative-ai-and-llms-session-1) - A presentation introducing Generative AI and LLMs, including their usage in specific applications.\n- [Retrieval Augmented Generation in Practice: Scalable GenAI ...](https:\u002F\u002Fwww.slideshare.net\u002Fcmihai\u002Fretrieval-augmented-generation-in-practice-scalable-genai-platforms-with-k8s-langchain-huggingface-and-vector) - A presentation discussing use cases for Generative AI, limitations of Large Language Models, and the use of Retrieval Augmented Generation (RAG) and fine-tuning techniques.\n- [LLM presentation final | PPT](https:\u002F\u002Fwww.slideshare.net\u002FRuthGriffin3\u002Fllm-presentation-final) - A presentation covering the Child & Family Agency Act 2013 and the Best Interest Principle in the context of LLMs.\n- [LLM Paradigm Adaptations in Recommender Systems.pdf](https:\u002F\u002Fwww.slideshare.net\u002FNagaBathula1\u002Fllm-paradigm-adaptations-in-recommender-systemspdf) - A PDF explaining the fine-tuning process and objective adaptations in LLM-based recommender systems.\n- [Conversational AI with Transformer Models | PPT](https:\u002F\u002Fwww.slideshare.net\u002Fdatabricks\u002Fconversational-ai-with-transformer-models) - A presentation highlighting the use of Transformer Models in Conversational AI applications.\n- [Llama-index | PPT](https:\u002F\u002Fpt.slideshare.net\u002FDenis973830\u002Fllamaindex) - A presentation on the rise of LLMs and building LLM-powered applications.\n- [LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention.pdf](https:\u002F\u002Fwww.slideshare.net\u002FjacksonChen22\u002Fllamaadapter-efficient-finetuning-of-language-models-with-zeroinit-attentionpdf) - A PDF discussing the efficient fine-tuning of Language Models with zero-init attention using LLaMA.\n\n## Podcasts\n\n- [Practical AI: Machine Learning, Data Science](https:\u002F\u002Fopen.spotify.com\u002Fshow\u002F1LaCr5TFAgYPK5qHjP3XDp) 🎧 - Making artificial intelligence practical, productive, and accessible to everyone. Engage in lively discussions about AI, machine learning, deep learning, neural networks, and more. Accessible insights and real-world scenarios for both beginners and seasoned practitioners.\n- [Gradient Dissent: Exploring Machine Learning, AI, Deep Learning](https:\u002F\u002Fpodcasts.apple.com\u002Fus\u002Fpodcast\u002Fgradient-dissent-exploring-machine-learning-ai-deep\u002Fid1504567418) 🎧 - Go behind the scenes to learn from industry leaders about how they are implementing deep learning in real-world scenarios. Gain insights into the machine learning industry and stay updated with the latest trends.\n- [Weaviate Podcast](https:\u002F\u002Fopen.spotify.com\u002Fshow\u002F4TlG6dnrWYdgN2YHpoSnM7) 🎧 - Join Connor Shorten for the Weaviate Podcast series, featuring interviews with experts and discussions on AI-related topics.\n- [Latent Space: The AI Engineer Podcast — CodeGen, Agents, Computer Vision, Data Science, AI UX and all things Software 3.0](https:\u002F\u002Fpodcasts.apple.com\u002Fin\u002Fpodcast\u002Flatent-space-the-ai-engineer-podcast-codegen-agents\u002Fid1674008350) 🎧 - Dive into the world of AI engineering, covering topics like code generation, computer vision, data science, and the latest advancements in AI UX.\n- [Unsupervised Learning](https:\u002F\u002Fpodcasts.apple.com\u002Fil\u002Fpodcast\u002Funsupervised-learning\u002Fid1672188924) 🎧 - Gain insights into the rapidly developing AI landscape and its impact on businesses and the world. Explore discussions on LLM applications, trends, and disrupting technologies.\n- [The TWIML AI Podcast (formerly This Week in Machine Learning)](https:\u002F\u002Fpodcasts.apple.com\u002Fno\u002Fpodcast\u002Fthe-twiml-ai-podcast-formerly-this-week-in-machine\u002Fid1116303051) 🎧 - Dive deep into fine-tuning approaches used in AI, LLM capabilities and limitations, and learn from experts in the field.\n- [AI and the Future of Work on Apple Podcasts](https:\u002F\u002Fpodcasts.apple.com\u002Fus\u002Fpodcast\u002Fai-and-the-future-of-work\u002Fid1476885647): A podcast hosted by SC Moatti discussing the impact of AI on the future of work.\n- [Practical AI: Machine Learning, Data Science: Fine-tuning vs RAG](https:\u002F\u002Fpodcasts.apple.com\u002Fus\u002Fpodcast\u002Ffine-tuning-vs-rag\u002Fid1406537385?i=1000626951912): This episode explores the comparison between fine-tuning and retrieval augmented generation in machine learning and data science.\n- [Unsupervised Learning on Apple Podcasts](https:\u002F\u002Fpodcasts.apple.com\u002Ffi\u002Fpodcast\u002Funsupervised-learning\u002Fid1672188924): Episode 20 features an interview with Anthropic CEO Dario Amodei on the future of AGI and AI.\n- [Papers Read on AI | Podcast on Spotify](https:\u002F\u002Fopen.spotify.com\u002Fshow\u002F2w8DRieJhMGFSTUhnsTVrw): This podcast keeps you updated with the latest trends and best performing architectures in the field of computer science.\n- [This Day in AI Podcast on Apple Podcasts](https:\u002F\u002Fpodcasts.apple.com\u002Fin\u002Fpodcast\u002Fthis-day-in-ai-podcast\u002Fid1671087656): Covering various AI-related topics, this podcast offers exciting insights into the world of AI.\n- [All About Evaluating LLM Applications \u002F\u002F Shahul Es \u002F\u002F #179 MLOps](https:\u002F\u002Fplayer.fm\u002Fseries\u002Fmlopscommunity\u002Fall-about-evaluating-llm-applications-shahul-es-mlops-podcast-179): In this episode, Shahul Es shares his expertise on evaluation in open source models, including insights on debugging, troubleshooting, and benchmarks.\n- [AI Daily on Apple Podcasts](https:\u002F\u002Fpodcasts.apple.com\u002Fus\u002Fpodcast\u002Fai-daily\u002Fid1686002118): Hosted by Conner, Ethan, and Farb, this podcast explores fascinating AI-related stories.\n- [Yannic Kilcher Videos (Audio Only) | Podcast on Spotify](https:\u002F\u002Fopen.spotify.com\u002Fshow\u002F6cHS7bXU2JPLTgjA0z0xNz): Yannic Kilcher discusses machine learning research papers, programming, and the broader impact of AI in society.\n- [LessWrong Curated Podcast | Podcast on Spotify](https:\u002F\u002Fopen.spotify.com\u002Fshow\u002F7vqBzO0ejqiLiXyTECEeBY): Audio version of the posts shared in the LessWrong Curated newsletter.\n- [SAI: The Security and AI Podcast on Apple Podcasts](https:\u002F\u002Fpodcasts.apple.com\u002Fil\u002Fpodcast\u002Fsai-the-security-and-ai-podcast\u002Fid1690378369): An episode focused on OpenAI's cybersecurity grant program.\n\n---\n\nThis initial version of the Awesome List was generated with the help of the [Awesome List Generator](https:\u002F\u002Fgithub.com\u002Falialsaeedi19\u002FGPT-Awesome-List-Maker). It's an open-source Python package that uses the power of GPT models to automatically curate and generate starting points for resource lists related to a specific topic. \n","# 令人惊叹的大型语言模型微调\n\n欢迎来到专为微调 GPT、BERT、RoBERTa 及其众多变体等大型语言模型（LLMs）而精心整理的资源合集！在当今的人工智能时代，将预训练模型适配到特定任务和领域的能力，已成为研究人员、数据科学家和机器学习从业者不可或缺的一项技能。\n\n大型语言模型基于海量数据进行训练，能够捕捉广泛的知识和语言细微差别。然而，要充分发挥它们在特定应用中的潜力，关键在于使用目标数据集对其进行微调。这一过程不仅能提升模型性能，还能确保其与当前任务的具体上下文、术语和需求相契合。\n\n在这份精彩的列表中，我们精心汇集了教程、论文、工具、框架以及最佳实践等多种资源，旨在帮助您顺利完成微调之旅。无论您是希望拓展专业知识的资深从业者，还是渴望踏入 LLM 领域的初学者，本资源库都将为您提供宝贵的见解和指导，以简化您的工作流程。\n\n## 目录\n\n- [GitHub 项目](#github-projects)\n- [文章与博客](#articles--blogs)\n- [在线课程](#online-courses)\n- [书籍](#books)\n- [研究论文](#research-papers)\n- [视频](#videos)\n- [工具与软件](#tools-&-software)\n- [会议与活动](#conferences-&-events)\n- [幻灯片与演示文稿](#slides-&-presentations)\n- [播客](#podcasts)\n\n## GitHub 项目\n- [AutoTrain](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fautotrain-advanced) - 无需代码即可对 LLM 及其他机器学习任务进行微调。\n- [LlamaIndex](https:\u002F\u002Fgithub.com\u002Frun-llama\u002Fllama_index) 🦙：用于 LLM 应用的数据框架。（23010 颗星）\n- [Petals](https:\u002F\u002Fgithub.com\u002Fbigscience-workshop\u002Fpetals) 🌸：以 BitTorrent 方式在家运行 LLM。微调和推理速度比卸载方式快高达 10 倍。（7768 颗星）\n- [LLaMA-Factory](https:\u002F\u002Fgithub.com\u002Fhiyouga\u002FLLaMA-Factory)：一个易于使用的 LLM 微调框架（支持 LLaMA-2、BLOOM、Falcon、Baichuan、Qwen、ChatGLM3 等）。（5532 颗星）\n- [lit-gpt](https:\u002F\u002Fgithub.com\u002FLightning-AI\u002Flit-gpt)：基于 nanoGPT 的可 hack 实现的最先进开源 LLM。支持 flash attention、4 位和 8 位量化、LoRA 和 LLaMA-Adapter 微调以及预训练。采用 Apache 2.0 许可证。（3469 颗星）\n- [H2O LLM Studio](https:\u002F\u002Fgithub.com\u002Fh2oai\u002Fh2o-llmstudio)：一个用于微调 LLM 的框架及无代码 GUI。文档：[https:\u002F\u002Fh2oai.github.io\u002Fh2o-llmstudio\u002F](https:\u002F\u002Fh2oai.github.io\u002Fh2o-llmstudio\u002F)（2880 颗星）\n- [Phoenix](https:\u002F\u002Fgithub.com\u002FArize-ai\u002Fphoenix)：AI 可观测性与评估——在笔记本中评估、排查故障并微调您的 LLM、CV 和 NLP 模型。（1596 颗星）\n- [LLM-Adapters](https:\u002F\u002Fgithub.com\u002FAGI-Edgerunners\u002FLLM-Adapters)：EMNLP 2023 论文《LLM-Adapters：一种用于高效参数微调大型语言模型的适配器家族》的相关代码。（769 颗星）\n- [Platypus](https:\u002F\u002Fgithub.com\u002Farielnlee\u002FPlatypus)：使用 LoRA 对 Platypus 系列 LLM 进行微调的代码。（589 颗星）\n- [xtuner](https:\u002F\u002Fgithub.com\u002FInternLM\u002Fxtuner)：一个高效微调 LLM（InternLM、Llama、Baichuan、QWen、ChatGLM2 等）的工具包。（540 颗星）\n- [DB-GPT-Hub](https:\u002F\u002Fgithub.com\u002Feosphoros-ai\u002FDB-GPT-Hub)：包含 DB-GPT 的模型、数据集和微调技术的仓库，旨在提升模型性能，尤其是在 Text-to-SQL 方面；使用该项目的 13B LLM 在 spider 评测中取得了高于 GPT-4 的执行准确率。（422 颗星）\n- [LLM-Finetuning-Hub](https:\u002F\u002Fgithub.com\u002Fgeorgian-io\u002FLLM-Finetuning-Hub)：包含 LLM 微调与部署脚本以及我们的研究成果的仓库。：star: 416\n- [Finetune_LLMs](https:\u002F\u002Fgithub.com\u002Fmallorbc\u002FFinetune_LLMs)：用于微调 Casual LLMs 的仓库。：star: 391\n- [MFTCoder](https:\u002F\u002Fgithub.com\u002Fcodefuse-ai\u002FMFTCoder)：高精度、高效率的多任务微调框架，适用于代码 LLM；业内首个支持多任务、多模型、多训练算法的大模型代码能力微调框架。：star: 337\n- [llmware](https:\u002F\u002Fgithub.com\u002Fllmware-ai\u002Fllmware)：提供企业级 LLM 基础开发框架、工具和微调后的模型。：star: 289\n- [LLM-Kit](https:\u002F\u002Fgithub.com\u002Fwpydcr\u002FLLM-Kit)：🚀集成 WebUI 的最新 LLM 综合平台 | 各大语言模型的全流程工具 WebUI 整合包。支持主流大模型 API 接口和开源模型。支持知识库、数据库、角色扮演、mj 文生图、LoRA 和全参数微调、数据集制作、live2d 等全流程应用工具。：star: 232\n- [h2o-wizardlm](https:\u002F\u002Fgithub.com\u002Fh2oai\u002Fh2o-wizardlm)：WizardLM 的开源实现，可将文档转换为 Q&A 对，用于 LLM 微调。：star: 228\n- [hcgf](https:\u002F\u002Fgithub.com\u002Fhscspring\u002Fhcgf)：人类可理解的聊天生成模型微调 | LLM微调。：star: 196\n- [llm_qlora](https:\u002F\u002Fgithub.com\u002Fgeorgesung\u002Fllm_qlora)：使用 QLoRA 对 LLM 进行微调。：star: 136\n- [awesome-llm-human-preference-datasets](https:\u002F\u002Fgithub.com\u002Fglgh\u002Fawesome-llm-human-preference-datasets)：一份精选的用于 LLM 微调、RLHF 和评估的人类偏好数据集列表。：star: 124\n- [llm_finetuning](https:\u002F\u002Fgithub.com\u002Ftaprosoft\u002Fllm_finetuning)：一款便捷的封装工具，用于对大型语言模型（LLMs）进行微调和推理，并支持多种量化技术（GTPQ、bitsandbytes）。：star: 114\n\n## 文章与博客\n- [2024 年使用 Hugging Face 微调 LLM：TRL 与 Flash Attention](https:\u002F\u002Fwww.philschmid.de\u002Ffine-tune-llms-in-2024-with-trl) 🤗：这篇博客文章提供了详尽的指南，介绍如何在消费级显卡（24GB）上，利用 Hugging Face 的 TRL 和 Flash Attention 对 LLM（例如 Llama 2）进行微调。\n- [面向初学者的 LLM 微调完全指南](https:\u002F\u002Fmedium.com\u002F@mayaakim\u002Fcomplete-guide-to-llm-fine-tuning-for-beginners-c2c38a3252be) 📚：这是一份全面的指南，解释了将预训练模型微调至新任务的过程，涵盖了关键概念并提供了具体示例。\n- [大型语言模型（LLM）的微调](https:\u002F\u002Ftowardsdatascience.com\u002Ffine-tuning-large-language-models-llms-23473d763b91) 📖：这篇博客文章概述了对预训练 LLM 进行微调的方法，讨论了重要概念，并提供了包含 Python 代码的实用示例。\n- [打造领域专家级 LLM：微调指南](https:\u002F\u002Fhackernoon.com\u002Fcreating-a-domain-expert-llm-a-guide-to-fine-tuning) 📝：这篇文章深入探讨了使用 OpenAI API 进行微调的概念，并展示了一个为理解亨德尔歌剧剧情而微调大型语言模型的实例。\n- [LLM 微调入门指南](https:\u002F\u002Ftowardsdatascience.com\u002Fa-beginners-guide-to-llm-fine-tuning-4bae7d4da672) 🌱：该指南涵盖了 LLM 微调的流程，包括使用 QLoRA 等工具来配置和微调模型。\n- [知识图谱与 LLM：微调 vs 检索增强生成](https:\u002F\u002Fmedium.com\u002Fneo4j\u002Fknowledge-graphs-llms-fine-tuning-vs-retrieval-augmented-generation-30e875d63a35) 📖：这篇博客文章探讨了 LLM 的局限性，并深入分析了结合知识图谱对其进行微调的方法。\n- [基于个人数据微调 LLM：创建《指环王》故事讲述者](https:\u002F\u002Fmedium.com\u002F@jeremyarancio\u002Ffine-tune-an-llm-on-your-personal-data-create-a-the-lord-of-the-rings-storyteller-6826dd614fa9) ✏️：这篇文章演示了如何利用个人数据训练自己的 LLM，从而在不依赖 OpenAI GPT-4 的情况下实现对个人信息的自主控制。\n- [使用 H2O LLM Studio 微调 LLM 模型以生成 Cypher 语句](https:\u002F\u002Ftowardsdatascience.com\u002Ffine-tuning-an-llm-model-with-h2o-llm-studio-to-generate-cypher-statements-3f34822ad5) 🧪：这篇博客文章提供了一个使用 H2O LLM Studio 微调 LLM 模型以生成 Cypher 语句的示例，从而支持基于知识图谱的聊天机器人应用。\n- [在 Colab 笔记本中微调属于你自己的 Llama 2 模型](https:\u002F\u002Ftowardsdatascience.com\u002Ffine-tune-your-own-llama-2-model-in-a-colab-notebook-df9823a04a32) 📝：这是一篇关于 LLM 微调的实用入门文章，展示了如何在 Google Colab 笔记本中实现这一过程，以创建属于自己的 Llama 2 模型。\n- [考虑微调 LLM 吗？开始前需注意的三点](https:\u002F\u002Ftowardsdatascience.com\u002Fthinking-about-fine-tuning-an-llm-heres-3-considerations-before-you-get-started-c1f483f293) 💡：本文讨论了微调 LLM 时需要考虑的三个要点，包括超越 PEFT 和 LoRA 改进 GPT 的方法，以及合理分配资源的重要性。\n- [LLM 与生成式 AI 入门：第 3 部分——基于指令的 LLM 微调](https:\u002F\u002Fmedium.com\u002F@yash9439\u002Fintroduction-to-llms-and-the-generative-ai-part-3-fine-tuning-llm-with-instruction-and-326bc95e07ae) 📚：这篇文章探讨了 LLM 在人工智能应用中的作用，并对其微调进行了概述。\n- [RAG 与微调——哪种工具最适合提升你的 LLM 应用？](https:\u002F\u002Ftowardsdatascience.com\u002Frag-vs-finetuning-which-is-the-best-tool-to-boost-your-llm-application-94654b1eaba7) - 一篇探讨构建 LLM 应用时需考虑的因素，以及如何根据具体场景选择合适方法的博客。 👨‍💻\n- [LLM 微调：RLHF 及其替代方案（第一部分）](https:\u002F\u002Fmedium.com\u002Fmantisnlp\u002Ffinetuning-an-llm-rlhf-and-alternatives-part-i-2106b95c8087) - 一篇文章展示了 RLHF 的替代方法，特别是直接偏好优化（DPO）。 🔄\n- [何时应微调 LLM？](https:\u002F\u002Ftowardsdatascience.com\u002Fwhen-should-you-fine-tune-llms-2dddc09a404a) - 探讨开源 LLM 的微调与大规模使用封闭式 API 进行 LLM 查询之间的比较。 🤔\n- [大型语言模型的微调](https:\u002F\u002Fcobusgreyling.medium.com\u002Ffine-tuning-large-language-models-f937869cef17) - 考虑大型语言模型的微调，并将其与零样本和少样本方法进行对比。 🎯\n- [Private GPT：在企业数据上微调 LLM](https:\u002F\u002Ftowardsdatascience.com\u002Fprivate-gpt-fine-tune-llm-on-enterprise-data-7e663d808e6a) - 探讨允许在较小显卡上微调 LLM 的训练技术。 🖥️\n- [使用 Scikit-LLM 微调 Google PaLM 2](https:\u002F\u002Fmedium.com\u002F@iryna230520\u002Ffine-tune-google-palm-2-with-scikit-llm-d41b0aa673a5) - 展示如何使用 Scikit-LLM 微调 Google 最先进的 LLM——PaLM 2。 📈\n- [深入解析大型语言模型的微调](https:\u002F\u002Frpradeepmenon.medium.com\u002Fa-deep-dive-into-fine-tuning-of-large-language-models-96f7029ac0e1) - 一篇关于微调 GPT-4、BERT 等 LLM 的综合性博客，提供了深入见解、发展趋势及优势。 🚀\n- [大型语言模型中的预训练、微调与上下文学习](https:\u002F\u002Fmedium.com\u002F@atmabodha\u002Fpre-training-fine-tuning-and-in-context-learning-in-large-language-models-llms-dd483707b122) - 讨论了 LLM 中的预训练、微调和上下文学习等概念。 📚\n- [开源微调大型语言模型列表](https:\u002F\u002Fsungkim11.medium.com\u002Flist-of-open-sourced-fine-tuned-large-language-models-llm-8d95a2e0dc76) - 一份精选的开源微调 LLM 列表，可在本地计算机上运行。 📋\n- [针对特定领域用例的 LLM 微调实践指南](https:\u002F\u002Fcismography.medium.com\u002Fpractitioners-guide-to-fine-tune-llms-for-domain-specific-use-case-part-1-4561714d874f) - 该指南总结了针对特定领域用例微调 LLM 的关键经验与结论。 📝\n- [使用生产级堆栈在 AWS、GCP 或 Azure 上微调 Llama 3.1](https:\u002F\u002Fwww.zenml.io\u002Fblog\u002Fhow-to-finetune-llama-3-1-with-zenml) - 一份关于在专为 MLOps 最佳实践设计的生产环境中微调 Llama 3.1（[或 Phi 3.5](https:\u002F\u002Fwww.zenml.io\u002Fblog\u002Fhow-to-finetune-phi-3-5-with-zenml)）的指南与教程。 📓\n\n## 在线课程\n\n- [微调基础：释放大语言模型的潜力 | Udemy](https:\u002F\u002Fwww.udemy.com\u002Fcourse\u002Fbuilding-chatgpt-style-agents\u002F)：面向初学者的实用课程，教授如何构建类ChatGPT的模型，并将其适配到特定应用场景。\n- [基于大型语言模型的生成式AI | Coursera](https:\u002F\u002Fwww.coursera.org\u002Flearn\u002Fgenerative-ai-with-llms)：学习生成式AI与LLM的基础知识，以及如何在实际应用中部署它们。可免费报名。\n- [大型语言模型：从应用到生产 | edX](https:\u002F\u002Fwww.edx.org\u002Flearn\u002Fcomputer-science\u002Fdatabricks-large-language-models-application-through-production)：面向开发者、数据科学家和工程师的进阶课程，教授如何使用流行框架构建以LLM为核心的应用，并实现端到端的生产就绪。\n- [大型语言模型的微调 | Coursera 指导项目](https:\u002F\u002Fwww.coursera.org\u002Fprojects\u002Ffinetuning-large-language-models-project)：一个简短的指导项目，涵盖大型语言模型微调的核心概念及训练流程。\n- [OpenAI与ChatGPT API：开发者的专家级微调 | Udemy](https:\u002F\u002Fwww.udemy.com\u002Fcourse\u002Fmastering-chatgpt-models-from-fine-tuning-to-deployment-openai\u002F)：探索GPT-3在创建对话式AI解决方案中的强大能力，内容包括提示工程、微调、集成以及ChatGPT模型的部署等。\n- [大型语言模型专业证书 | edX](https:\u002F\u002Fwww.edx.org\u002Fcertificates\u002Fprofessional-certificate\u002Fdatabricks-large-language-models)：学习如何利用最新的框架、技术和基础模型背后的理论，构建并投产基于大型语言模型的应用程序。\n- [超越微调：提升你的LLM性能 | Udemy](https:\u002F\u002Fwww.udemy.com\u002Fcourse\u002Fimproving-the-performance-of-your-llm-beyond-fine-tuning\u002F)：本课程专为对LLM微调感兴趣的企业领导者和开发者设计，旨在探索提升LLM性能的技术方法。\n- [大型语言模型导论 | Coursera](https:\u002F\u002Fwww.coursera.org\u002Flearn\u002Fintroduction-to-large-language-models)：由Google Cloud提供的入门级微学习课程，讲解大型语言模型（LLMs）的基本概念及其应用场景。可免费报名。\n- [课程大纲 | LLM101x | edX](https:\u002F\u002Fcourses.edx.org\u002Fcourses\u002Fcourse-v1:Databricks+LLM101x+2T2023\u002Fc861b0726ce24e099ad80111145f4217\u002F)：学习如何使用数据嵌入、向量数据库，并结合领域特定数据对LLM进行微调，从而增强LLM工作流。\n- [深度学习模型性能调优大师班 | Udemy](https:\u002F\u002Fwww.udemy.com\u002Fcourse\u002Fperformance-tuning-deep-learning-models-master-class\u002F)：关于深度学习模型调优的大师级课程，涵盖加速训练和优化性能的各种技术。\n- [最佳大型语言模型（LLMs）课程与认证 | Coursera](https:\u002F\u002Fwww.coursera.org\u002Fcourses?query=large%20language%20models)：精选自顶尖教育机构和行业领军企业的LLMs课程，旨在为个人和企业团队提供高质量的培训，帮助他们学习或提升LLM微调技能。\n- [精通语言模型：释放LLMs的强大威力 | Udemy](https:\u002F\u002Fwww.udemy.com\u002Fcourse\u002Fmastering-language-models-unleashing-the-power-of-llms\u002F)：在这门综合课程中，你将深入探讨自然语言处理的基本原理，并了解LLMs如何重塑AI应用的格局。这是一份关于高级NLP和LLMs的全面指南。\n- [LLMs精通：Transformer与生成式AI完全指南 | Udemy](https:\u002F\u002Fwww.udemy.com\u002Fcourse\u002Fllms-mastery-complete-guide-to-transformers-generative-ai\u002F)：本课程提供了对AI发展史的精彩概述，并覆盖BERT、GPT和T5三大主流LLM的微调技术。适合对生成式AI、LLMs及生产级应用感兴趣的学员。\n- [探索ChatGPT、GPT-4及LLMs背后的技术 | Udemy](https:\u002F\u002Fwww.udemy.com\u002Fcourse\u002Fexploring-the-technologies-behind-chatgpt-openai\u002F)：这是你了解ChatGPT、GPT-4、BERT等大型语言模型所需的唯一课程。深入了解这些LLMs背后的技术。\n- [非技术人员的大型语言模型入门 | Udemy](https:\u002F\u002Fwww.udemy.com\u002Fcourse\u002Fnon-technical-introduction-to-large-language-models\u002F)：为非技术人员提供的大型语言模型概览，解释当前面临的挑战，并以通俗易懂的方式进行说明，避免复杂的专业术语。\n- [大型语言模型：从头开始理解基础模型 | edX](https:\u002F\u002Fwww.edx.org\u002Flearn\u002Fcomputer-science\u002Fdatabricks-large-language-models-foundation-models-from-the-ground-up)：深入探讨LLMs中的基础模型，如BERT、GPT和T5，了解最新进展如何提升LLMs的功能。\n\n## 书籍\n\n- [生成式AI与大型语言模型——由DeepLearning.AI和AWS联合推出的全新实践课程](https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Faws\u002Fgenerative-ai-with-large-language-models-new-hands-on-course-by-deeplearning-ai-and-aws\u002F)\n- 一门实践课程，教授如何使用奖励模型和强化学习对大型语言模型（LLM）进行微调，重点聚焦于生成式AI。\n- [从数据选择到微调：构建LLM模型的技术指南](https:\u002F\u002Fwww.amazon.com\u002FData-Selection-Fine-Tuning-Constructing\u002Fdp\u002FB0CH2FLV2Q)\n- 一本技术指南，涵盖了从数据选择到微调的LLM模型构建全流程。\n- [LLM知识烹饪书：从RAG、QLoRA到微调，以及其间的所有技巧！](https:\u002F\u002Fwww.amazon.com\u002FLLM-Knowledge-Cookbook-Recipes-Between-ebook\u002Fdp\u002FB0CKH9B58N)\n- 一本全面的烹饪书，深入探讨了多种LLM模型技术，包括检索与生成（RAG）、查询语言表示（QLoRA）等，以及微调流程。\n- [LLM微调原则](https:\u002F\u002Fwww.packtpub.com\u002Farticle-hub\u002Fprinciples-for-fine-tuning-llms)\n- 一篇文章，揭秘LLM微调的过程，并探讨多种技术，如上下文学习、经典微调方法、参数高效微调以及人类反馈强化学习（RLHF）。\n- [从数据选择到微调：构建LLM模型的技术指南](https:\u002F\u002Fwww.goodreads.com\u002Fbook\u002Fshow\u002F198555505-from-data-selection-to-fine-tuning)\n- 一本技术指南，深入解析大型语言模型（LLM）的构建与训练过程。\n- [动手实践大型语言模型](https:\u002F\u002Fwww.oreilly.com\u002Flibrary\u002Fview\u002Fhands-on-large-language\u002F9781098150952\u002F)\n- 一本书，介绍了由深度学习驱动的语言AI系统的最新进展，重点关注大型语言模型。\n- [在Amazon SageMaker JumpStart上为文本生成微调Llama 2模型](https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fmachine-learning\u002Ffine-tune-llama-2-for-text-generation-on-amazon-sagemaker-jumpstart\u002F)\n- 学习如何使用Amazon SageMaker JumpStart对Llama 2模型进行微调，以优化对话生成效果。\n- [利用AWS Trainium实现快速且经济高效的LLaMA 2微调](https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fmachine-learning\u002Ffast-and-cost-effective-llama-2-fine-tuning-with-aws-trainium\u002F)\n- 一篇博客文章，详细说明如何借助AWS Trainium实现LLaMA 2模型的快速且经济高效的微调。\n- [微调——高级深度学习与Python[书籍]](https:\u002F\u002Fwww.oreilly.com\u002Flibrary\u002Fview\u002Fadvanced-deep-learning\u002F9781789956177\u002Fbd91eb1d-87cf-463d-bbd3-4177a42f3da7.xhtml) 💡：本书探讨了在高级深度学习中，继预训练之后的微调任务。\n- [LLM知识烹饪书：从RAG、QLoRA到微调……](https:\u002F\u002Fwww.barnesandnoble.com\u002Fw\u002Fthe-llm-knowledge-cookbook-richard-anthony-aragon\u002F1144180729) 💡：一本全面指南，介绍如何将大型语言模型（LLM）应用于各种任务，内容涵盖基础知识及高级微调技术。\n- [大型语言模型快速入门指南：策略与最佳实践……](https:\u002F\u002Fwww.oreilly.com\u002Flibrary\u002Fview\u002Fquick-start-guide\u002F9780138199425\u002F) 💡：一本专注于大型语言模型（LLMs），如BERT、T5和ChatGPT的策略与最佳实践的指南，展示了它们在各类自然语言处理任务中的卓越表现。\n- [4. GPT-4与ChatGPT高级技巧——开发应用……](https:\u002F\u002Fwww.oreilly.com\u002Flibrary\u002Fview\u002Fdeveloping-apps-with\u002F9781098152475\u002Fch04.html) 💡：本章深入探讨了GPT-4和ChatGPT的高级技巧，包括提示工程、零样本学习、少样本学习以及针对特定任务的微调。\n- [什么是大型语言模型？——LLM AI详解——AWS](https:\u002F\u002Faws.amazon.com\u002Fwhat-is\u002Flarge-language-model\u002F) 💡：对大型语言模型（LLMs）的解释，讨论了少样本学习和微调的概念，以提升模型性能。\n\n## 研究论文\n\n- [LLM-Adapters：用于参数高效微调的适配器家族](https:\u002F\u002Farxiv.org\u002Fabs\u002F2304.01933) 📄：本文提出 LLM-Adapters，一个易于使用的框架，可将多种适配器集成到大语言模型中，以实现针对不同任务的参数高效微调（PEFT）。\n- [减少专化的两阶段大语言模型微调](https:\u002F\u002Farxiv.org\u002Fabs\u002F2211.00635) 📄：ProMoT 是一种两阶段微调框架，通过“提示微调结合模型微调”的方式，解决大语言模型在格式上的专化问题，从而提升其上下文学习的通用性能。\n- [基于本体推理的大企业语言模型微调](https:\u002F\u002Farxiv.org\u002Fabs\u002F2306.10723) 📄：本文提出一种神经符号架构，将大语言模型（LLMs）与企业知识图谱（EKGs）相结合，以实现 LLM 的领域特定微调。\n- [QLoRA：量化大语言模型的高效微调](https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.14314) 📄：QLoRA 是一种高效的微调方法，可在保持任务性能的同时降低内存占用，并为量化预训练语言模型提供了新的见解。\n- [资源有限条件下大语言模型的全参数微调](https:\u002F\u002Farxiv.org\u002Fabs\u002F2306.09782) 📄：该工作提出了 LOMO，一种低内存优化技术，能够在 GPU 资源有限的情况下实现大语言模型的全参数微调。\n- [LoRA：大语言模型的低秩适应](https:\u002F\u002Farxiv.org\u002Fabs\u002F2106.09685) 📄：LoRA 提出了一种方法，通过在每一层注入可训练的低秩分解矩阵，将大型预训练模型适配到特定任务，从而在保持模型质量的同时减少可训练参数的数量。\n- [利用进化式微调增强大语言模型进行新闻摘要生成](https:\u002F\u002Farxiv.org\u002Fabs\u002F2307.02839) 📄：本文提出了一种基于大语言模型的新闻摘要生成新范式，结合了遗传算法和强大的自然语言理解能力。\n- [语言之间如何相互影响？研究大语言模型微调过程中的跨语言数据共享](https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.13286) 📄：本研究探讨了多语言大语言模型（MLLMs）微调过程中跨语言数据共享的现象，并分析了不同语言对模型性能的影响。\n- [仅需前向传播即可微调语言模型](https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.17333) 📄：MeZO 是一种内存高效的零阶优化器，能够在显著降低内存需求的同时实现大语言模型的微调。\n- [学习对场景图进行推理：以大语言模型微调为例](https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.07716) 📄：该工作探索了 GPT-2 大语言模型在机器人任务规划中的应用，展示了 LLM 在长时程任务规划场景中的潜力。\n- [使用差分隐私对大语言模型进行私密微调](https:\u002F\u002Farxiv.org\u002Fabs\u002F2210.15042)：本文探讨了将差分隐私应用于大语言模型（LLMs）微调，为其添加隐私保障。\n- [DISC-LawLLM：面向智能法律系统的大型语言模型微调](http:\u002F\u002Farxiv.org\u002Fabs\u002F2309.11325)：本文介绍了 DISC-LawLLM，这是一种利用经过微调、具备法律推理能力的 LLM 提供广泛法律服务的智能法律系统。\n- [针对特定场景的 LLaMa 多任务指令微调：A](https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.13225)：该论文研究了对基础性 LLM——LLaMa——在特定写作任务上的微调效果，结果表明其写作能力得到了显著提升。\n- [通过人类反馈训练语言模型遵循指令](https:\u002F\u002Farxiv.org\u002Fabs\u002F2203.02155)：本文提出了一种通过人类反馈对语言模型进行微调的方法，使其更好地符合用户意图，最终在人工评估中表现优于更大规模的模型。\n- [大型语言模型可以自我改进](https:\u002F\u002Farxiv.org\u002Fabs\u002F2210.11610)：该论文证明，LLM 可以通过使用自动生成的解决方案进行微调来提升自身的推理能力，在无需真实标签的情况下达到最先进的性能。\n- [拥抱大型语言模型在医疗领域的应用](https:\u002F\u002Fwww.ncbi.nlm.nih.gov\u002Fpmc\u002Farticles\u002FPMC10292051\u002F)：本文强调了经微调的 LLM 在医疗应用中的潜力，能够提高诊断准确率并支持临床决策。\n- [指令微调语言模型的扩展](https:\u002F\u002Farxiv.org\u002Fabs\u002F2210.11416)：该论文探讨了在 LLM 上进行指令微调的效果，结果显示其性能显著提升，并且具有良好的未见任务泛化能力。\n- [跨设备联邦微调数十亿参数级语言模型](https:\u002F\u002Farxiv.org\u002Fabs\u002F2308.13894)：该工作提出了 FwdLLM，一种旨在提升移动设备上大型 LLM 微调效率的联邦学习协议，可同时提高内存和时间效率。\n- [大型语言模型的全面概述](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2307.06435)：本文综述了大型语言模型的发展历程及其应用，并探讨了其迁移学习能力。\n- [微调语言模型以在人群中达成共识](https:\u002F\u002Farxiv.org\u002Fabs\u002F2211.15006)：该论文探讨了如何微调大型 LLM，使其生成能够最大化不同意见群体认可度的一致性声明。\n\n## 视频\n- [安德烈·卡帕西的大语言模型入门](https:\u002F\u002Fyoutu.be\u002FzjkBMFhNj_g?si=5S9tI-G2AD7xUuhf)：这是一段长达1小时的大语言模型入门介绍。内容包括大语言模型是什么、其发展方向、与当今操作系统之间的对比和类比，以及这一新型计算范式面临的一些安全相关挑战。\n- [在您自己的数据集上微调 Llama 2 | 训练适合您的...的 LLM](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=MDA3LUKNl1E)：学习如何在自定义数据集上微调 Llama 2 模型。\n- [使用 QLoRA 在单个 GPU 上微调 LLM：在...上训练 Falcon-7b](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=DcBC4yGHV4Q)：本视频演示了使用 QLoRA 微调 Falcon 7b LLM 的过程。\n- [使用 PEFT 微调 LLM | 大语言模型入门](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=6SpZkwWuldU)：了解如何使用资源需求更少的 PEFT 技术来微调 LLM。\n- [LLAMA-2 开源 LLM：在...上轻松进行自定义微调](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=8cc4bJtycOA)：逐步指导您如何在自定义数据集上微调 LLama 2 LLM 模型。\n- [新课程：微调大型语言模型 - YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=9PxhCekQYNI)：本视频介绍了一门关于微调 LLM 的课程，涵盖模型选择、数据准备、训练和评估等内容。\n- [问：如何为我的 LLM 微调创建指令数据集...](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=BJQrQT2Xfyo)：本教程面向初学者，讲解了 LLM 微调的相关知识，包括何时、如何以及为何进行微调。\n- [LLM 第 4 模块：微调和评估 LLM | 4.13.1 笔记本...](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=2vEpMb4ofVU)：一个关于微调和评估 LLM 的笔记本演示。\n- [Google LLM 微调\u002F适配\u002F定制 - 入门...](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=cJ96rqW8L84)：通过分步指南，开始对 Google 的 PaLM 2 大语言模型进行微调。\n- [LLM 的预训练、微调与上下文学习（GPT-x ...](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=_FYwnO_g-4E)：一份终极指南，解释了 GPT-x 等 LLM 的预训练、微调和上下文学习。\n- [如何用 PDF 微调 LLM - Langchain 教程 - YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=bOS929yCkGE)：学习如何使用 Langchain 和 PDF 库，将 OpenAI 的 GPT LLM 微调为能够处理 PDF 文档的功能。\n- [EasyTune 演示 - YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=yDYgbKYNk2I)：使用 Falcon-7b 和 QLoRA 在单个 GPU 上微调 LLM 的演示。\n- [解锁 ChatGPT 的潜力：训练与微调中的经验教训](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=Szi0NgmVg7Y)：THE STUDENT 展示了使用符号进行 LLM 的指令微调和上下文学习。\n- [AI 新闻：无需代码即可创建 LLM！ - YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=MRXj1pAB6cI)：玛雅·阿金讨论了您需要了解的前 5 种 LLM 微调应用场景。\n- [您需要了解的前 5 种 LLM 微调应用场景 - YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=v1R8uPqjNAU)：一段深入的视频，重点介绍了 LLM 微调的前 5 种应用场景，并附有更多探索链接。\n- [clip2 llm emory - YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=UXBA7FKIUso)：学习如何在您自己的数据集上微调 Llama 2，并针对特定用途训练 LLM。\n- [在本地机器上微调 LLAMA-v2 的最简单方法！ - YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=3fsn19OI_C8)：一段分步视频指南，展示了在本地机器上为自定义数据集微调 LLAMA-v2 的最简单、最快捷的方法。\n- [训练与微调 LLM：入门 - YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=2QRlvKSzyVw)：介绍训练和微调 LLM 的相关内容，包括重要概念以及 NeurIPS LLM 效率挑战赛。\n- [使用 PEFT 和 LoRA 微调 LLM - YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=Us5ZFp16PaU)：一段全面的视频，探讨如何利用 PEFT 微调任何解码器风格的 GPT 模型，包括 LoRA 微调的基础知识和上传操作。\n- [为 RLHF 和 LLM 微调构建与整理数据集...](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=Ezz_5csCJqI)：在 Argilla 的赞助下，学习如何为 RLHF（基于人类反馈的强化学习）和 LLM（大型语言模型）微调构建及整理数据集。\n- [使用 Python 中的自定义数据微调 LLM（OpenAI GPT）- YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=6aOzoJKNLKQ)：探索如何通过使用自定义数据集对 LLM（OpenAI GPT）进行微调，以实现问答、摘要等类似 ChatGPT 的功能。\n\n## 工具与软件\n\n- [LLaMA 高效微调](https:\u002F\u002Fsourceforge.net\u002Fprojects\u002Fllama-efficient-tuning.mirror\u002F) 🛠️：易于使用的 LLM 微调框架（LLaMA-2、BLOOM、Falcon）。\n- [H2O LLM Studio](https:\u002F\u002Fsourceforge.net\u002Fprojects\u002Fh2o-llm-studio.mirror\u002F) 🛠️：用于微调 LLM 的框架和无代码 GUI。\n- [PEFT](https:\u002F\u002Fsourceforge.net\u002Fprojects\u002Fpeft.mirror\u002F) 🛠️：参数高效微调（PEFT）方法，用于将预训练的语言模型高效地适应下游应用。\n- [类似 ChatGPT 的模型](https:\u002F\u002Fsourceforge.net\u002Fdirectory\u002Flarge-language-models-llm\u002Fc\u002F) 🛠️：在您的设备上本地运行快速的类似 ChatGPT 的模型。\n- [Petals](https:\u002F\u002Fsourceforge.net\u002Fprojects\u002Fpetals.mirror\u002F)：协作运行像 BLOOM-176B 这样的大型语言模型，允许您加载模型的一小部分，并与其他用户合作进行推理或微调。🌸\n- [NVIDIA NeMo](https:\u002F\u002Fsourceforge.net\u002Fdirectory\u002Flarge-language-models-llm\u002Flinux\u002F)：用于构建最先进的对话式 AI 模型的工具包，专为 Linux 设计。🚀\n- [H2O LLM Studio](https:\u002F\u002Fsourceforge.net\u002Fdirectory\u002Flarge-language-models-llm\u002Fwindows\u002F)：用于在 Windows 上微调大型语言模型的框架和无代码 GUI 工具。🎛️\n- [Ludwig AI](https:\u002F\u002Fsourceforge.net\u002Fprojects\u002Fludwig-ai.mirror\u002F)：用于构建自定义 LLM 和其他深度神经网络的低代码框架。只需使用声明式的 YAML 配置文件，即可轻松训练最先进的 LLM。🤖\n- [bert4torch](https:\u002F\u002Fsourceforge.net\u002Fprojects\u002Fbert4torch.mirror\u002F)：优雅的 PyTorch 变压器实现。加载各种开源大型模型权重，用于推理和微调。🔥\n- [Alpaca.cpp](https:\u002F\u002Fsourceforge.net\u002Fprojects\u002Falpaca-cpp.mirror\u002F)：在您的设备上本地运行快速的类似 ChatGPT 的模型。结合了 LLaMA 基础模型和斯坦福 Alpaca 的开源复现版本，用于指令微调。🦙\n- [promptfoo](https:\u002F\u002Fsourceforge.net\u002Fprojects\u002Fpromptfoo.mirror\u002F)：通过自动评估和具有代表性的用户输入，评估和比较 LLM 输出、检测回归并改进提示语。📊\n\n## 会议与活动\n\n- [ML\u002FAI对话：神经符号AI——LLM的替代方案](https:\u002F\u002Fwww.meetup.com\u002Fnew-york-ai-ml-conversations\u002Fevents\u002F296127633\u002F) - 本次聚会将讨论微调LLM的经验，并探讨神经符号AI作为替代方案。\n- [AI开发者日——西雅图，2023年10月30日（周一）下午5:00](https:\u002F\u002Fwww.meetup.com\u002Fgdgcloudseattle\u002Fevents\u002F296959536\u002F) - 关于高效LLM可观测性以及利用向量相似度搜索进行微调机会的技术分享。\n- [DeepLearning.AI活动](https:\u002F\u002Fwww.eventbrite.com\u002Fo\u002Fdeeplearningai-19822694300) - 系列活动包括缓解LLM幻觉、使用PyTorch 2.0和ChatGPT微调LLM，以及AI教育项目。\n- [AI开发者日——纽约，2023年10月26日（周四）下午5:30](https:\u002F\u002Fwww.meetup.com\u002Fbig-data\u002Fevents\u002F296665127\u002F) - 技术分享主题为GenAI应用的最佳实践，以及如何利用LLM实现实时个性化通知。\n- [聊天LLM与AI智能体——利用生成式AI构建AI系统与智能体](https:\u002F\u002Fwww.meetup.com\u002Ftel-aviv-ai-tech-talks\u002Fevents\u002F296739549\u002F) - 活动聚焦LLM、AI智能体和链式数据，并通过活动聊天提供互动机会。\n- [纽约AI\u002FLLM\u002FChatGPT开发者小组](https:\u002F\u002Fwww.meetup.com\u002Fnyc-llm-talks\u002F) - 定期为对AI、LLM、ChatGPT、NLP、ML、数据等领域感兴趣的开发者举办技术分享会或工作坊。\n- [周二，2023年11月14日下午2:00：利用LLM处理企业数据](https:\u002F\u002Fwww.meetup.com\u002Fdata-science-dojo-new-york\u002Fevents\u002F296708155\u002F) - 深入探讨专为非公开数据应用设计的LLM关键策略，包括提示工程和检索技术。\n- [贝尔维尤应用机器学习聚会](https:\u002F\u002Fwww.meetup.com\u002Fbellevue-applied-machine-learning-meetup\u002F) - 聚会专注于应用机器学习技术，旨在提升数据科学家和ML从业者的能力。\n- [慕尼黑AI与提示工程聚会，2023年10月5日（周四）18:15](https:\u002F\u002Fwww.meetup.com\u002Fde-DE\u002Fai-prompt-engineering-munich\u002Fevents\u002F295437909\u002F) - 介绍H2O LLM Studio用于微调LLM，并汇聚来自不同背景的AI爱好者。\n- [西雅图AI\u002FML\u002F数据开发者小组](https:\u002F\u002Fwww.meetup.com\u002Faittg-seattle\u002F) - 技术分享内容涵盖评估LLM代理，以及通过实践学习AI\u002FML\u002F数据相关知识。\n- [数据科学道场——华盛顿特区 | 聚会](https:\u002F\u002Fwww.meetup.com\u002Fdata-science-dojo-washington-dc\u002F)：这是一个位于华盛顿特区的聚会组织，面向对数据科学的教学、学习及知识共享感兴趣的企业专业人士。\n- [在阿联酋迪拜查找数据科学活动与小组](https:\u002F\u002Fwww.meetup.com\u002Ffind\u002Fae--dubai\u002Fdata-science\u002F)：探索迪拜的数据科学活动和小组，结识志同道合的朋友。\n- [AI线下聚会：生成式AI与LLM——万圣节特别版](https:\u002F\u002Fwww.meetup.com\u002Fdc-ai-llms\u002Fevents\u002F296543682\u002F)：加入此次AI聚会，聆听关于生成式AI和大型语言模型（LLM）的技术分享，内容包括开源工具及最佳实践。\n- [ChatGPT释放潜能：NLP实战演示与最佳实践](https:\u002F\u002Fwww.meetup.com\u002Fdata-science-dojo-karachi\u002Fevents\u002F296977810\u002F)：本次线上活动将探讨大型语言模型的微调技巧，并展示ChatGPT及LLM的实际应用场景。\n- [在印度浦那查找数据科学活动与小组](https:\u002F\u002Fwww.meetup.com\u002Ffind\u002Fin--pune\u002Fdata-science\u002F)：探索浦那地区线上线下相关的数据科学活动和小组。\n- [华盛顿特区AI\u002FML\u002F数据开发者小组 | 聚会](https:\u002F\u002Fwww.meetup.com\u002Faidev-dc\u002F)：该小组旨在汇集华盛顿特区地区的AI爱好者，共同学习和实践AI技术，包括AI、机器学习、深度学习和数据科学。\n- [波士顿AI\u002FLLMs\u002FChatGPT开发者小组 | 聚会](https:\u002F\u002Fwww.meetup.com\u002Fbostondeeplearningai\u002F)：加入波士顿的这个小组，学习并实践LLM、ChatGPT、机器学习、深度学习和数据科学等AI技术。\n- [巴黎NLP | 聚会](https:\u002F\u002Fwww.meetup.com\u002Fparis-nlp\u002F)：该聚会专注于自然语言处理（NLP）在各领域的应用，讨论传统与现代NLP方法的技术、研究及实际应用。\n- [旧金山AI\u002FLLMs\u002FChatGPT开发者小组 | 聚会](https:\u002F\u002Fwww.meetup.com\u002Fsan-francisco-ai-llms\u002F)：与旧金山湾区的AI爱好者交流，学习并实践包括LLM、ChatGPT、NLP、机器学习、深度学习和数据科学在内的AI技术。\n- [AI线下聚会：GenAI与LLM在医疗健康领域的应用](https:\u002F\u002Fwww.meetup.com\u002Faittg-boston\u002Fevents\u002F296567040\u002F)：参加此次技术分享，了解LLM在医疗健康领域的应用，并学习如何快速利用LLM完成健康相关任务。\n\n## 幻灯片与演示文稿\n\n- [大型语言模型的微调](https:\u002F\u002Fwww.slideshare.net\u002FSylvainGugger\u002Ffine-tuning-large-lms-243430468)：探讨如何对GPT、BERT和RoBERTa等大型语言模型进行微调的演示文稿。\n- [LLaMa 2.pptx](https:\u002F\u002Fwww.slideshare.net\u002FRkRahul16\u002Fllama-2pptx)：介绍由Meta AI开发的强大大型语言模型LLaMa 2的幻灯片。\n- [LLM.pdf](https:\u002F\u002Fwww.slideshare.net\u002FMedBelatrach\u002Fllmpdf-261239806)：探索Transformer在自然语言处理中的作用，从BERT到GPT-3的演示文稿。\n- [大型语言模型训练营](https:\u002F\u002Fwww.slideshare.net\u002FDataScienceDojo\u002Flarge-language-models-bootcamp)：涵盖大型语言模型各个方面（包括从头开始训练和微调）的训练营幻灯片。\n- [CNN解释的LHC](https:\u002F\u002Fwww.slideshare.net\u002Fhijiki_s\u002Fthe-lhc-explained-by-cnn)：利用CNN和图像模型微调技术讲解大型强子对撞机（LHC）的幻灯片。\n- [用10行代码使用大型语言模型](https:\u002F\u002Fwww.slideshare.net\u002FGautierMarti\u002Fusing-large-language-models-in-10-lines-of-code)：演示仅用10行代码即可使用大型语言模型的方法。\n- [LLaMA-Adapter：零初始化注意力机制下的高效语言模型微调.pdf](https:\u002F\u002Fwww.slideshare.net\u002FjacksonChen22\u002Fllamaadapter-efficient-finetuning-of-language-models-with-zeroinit-attentionpdf)：讨论LLaMA-Adapter这一采用零初始化注意力机制高效微调语言模型技术的幻灯片。\n- [LLM入门](https:\u002F\u002Fwww.slideshare.net\u002FLoicMerckel\u002Fintro-to-llms)：介绍大型语言模型的演示文稿，内容包括基础模型以及基于提示-完成对的微调方法。\n- [LLM微调（东大松尾研LLM讲座第5天资料） - Speaker Deck](https:\u002F\u002Fspeakerdeck.com\u002Fschulta\u002Fllm-fine-tuning-dong-da-song-wei-yan-llmjiang-zuo-day5zi-liao)：用于大型语言模型微调讲座的幻灯片，特别针对东大松尾研2023年暑期学校。\n- [用ChatGPT自动化你的工作与业务 #3](https:\u002F\u002Fpt.slideshare.net\u002FAnantCorp\u002Fautomate-your-job-and-business-with-chatgpt-3-fundamentals-of-llmgpt)：讨论ChatGPT基础知识及其在工作自动化和业务任务中应用的演示文稿。\n- [释放生成式AI的力量——高管指南.pdf](https:\u002F\u002Fwww.slideshare.net\u002FPremNaraindas1\u002Funlocking-the-power-of-generative-ai-an-executives-guidepdf)：一本指南，详细说明如何通过微调大型语言模型（LLM），使其更好地满足组织需求。\n- [微调并部署Hugging Face NLP模型 | PPT](https:\u002F\u002Fwww.slideshare.net\u002Fovhcom\u002Fpres-hugging-facefinetuning)：一份提供关于如何使用Hugging Face NLP构建和部署LLM模型见解的演示文稿。\n- [大规模语言模型时代的人工参与式机器学习 - Speaker Deck](https:\u002F\u002Fspeakerdeck.com\u002Fyukinobaba\u002Fhuman-in-the-loop-ml-llm)：一张幻灯片，讨论如何微调语言模型，以在具有不同偏好的人类之间达成共识。\n- [AI与ML系列——生成式AI与LLM简介 | PPT](https:\u002F\u002Fwww.slideshare.net\u002FDianaGray10\u002Fai-and-ml-series-introduction-to-generative-ai-and-llms-session-1)：介绍生成式AI和LLM的演示文稿，包括它们在特定应用中的使用。\n- [实践中的检索增强生成：可扩展的GenAI…](https:\u002F\u002Fwww.slideshare.net\u002Fcmihai\u002Fretrieval-augmented-generation-in-practice-scalable-genai-platforms-with-k8s-langchain-huggingface-and-vector)：讨论生成式AI的应用场景、大型语言模型的局限性，以及检索增强生成（RAG）和微调技术的使用。\n- [LLM演示最终版 | PPT](https:\u002F\u002Fwww.slideshare.net\u002FRuthGriffin3\u002Fllm-presentation-final)：一份涵盖2013年儿童与家庭机构法案及LLM背景下“最佳利益原则”的演示文稿。\n- [推荐系统中的LLM范式适应.pdf](https:\u002F\u002Fwww.slideshare.net\u002FNagaBathula1\u002Fllm-paradigm-adaptations-in-recommender-systemspdf)：一份PDF文件，解释基于LLM的推荐系统中的微调过程及目标调整。\n- [使用Transformer模型的对话式AI | PPT](https:\u002F\u002Fwww.slideshare.net\u002Fdatabricks\u002Fconversational-ai-with-transformer-models)：突出Transformer模型在对话式AI应用中使用的演示文稿。\n- [Llama-index | PPT](https:\u002F\u002Fpt.slideshare.net\u002FDenis973830\u002Fllamaindex)：关于LLM兴起及构建LLM驱动应用的演示文稿。\n- [LLaMA-Adapter：零初始化注意力机制下的高效语言模型微调.pdf](https:\u002F\u002Fwww.slideshare.net\u002FjacksonChen22\u002Fllamaadapter-efficient-finetuning-of-language-models-with-zeroinit-attentionpdf)：一份PDF文件，讨论使用LLaMA实现零初始化注意力机制下语言模型的高效微调。\n\n## 播客\n\n- [实用AI：机器学习、数据科学](https:\u002F\u002Fopen.spotify.com\u002Fshow\u002F1LaCr5TFAgYPK5qHjP3XDp) 🎧 - 让人工智能变得实用、高效，并为所有人所用。参与关于AI、机器学习、深度学习、神经网络等话题的精彩讨论。无论你是初学者还是资深从业者，都能在这里获得易于理解的见解和真实场景案例。\n- [梯度异议：探索机器学习、AI与深度学习](https:\u002F\u002Fpodcasts.apple.com\u002Fus\u002Fpodcast\u002Fgradient-dissent-exploring-machine-learning-ai-deep\u002Fid1504567418) 🎧 - 走进幕后，聆听行业领袖分享他们在实际场景中如何应用深度学习的经验。深入了解机器学习行业，并掌握最新趋势。\n- [Weaviate播客](https:\u002F\u002Fopen.spotify.com\u002Fshow\u002F4TlG6dnrWYdgN2YHpoSnM7) 🎧 - 与Connor Shorten一起收听Weaviate播客系列，节目邀请各领域专家进行访谈，探讨与AI相关的话题。\n- [潜在空间：AI工程师播客——代码生成、智能体、计算机视觉、数据科学、AI用户体验及软件3.0的一切](https:\u002F\u002Fpodcasts.apple.com\u002Fin\u002Fpodcast\u002Flatent-space-the-ai-engineer-podcast-codegen-agents\u002Fid1674008350) 🎧 - 深入AI工程领域，涵盖代码生成、计算机视觉、数据科学以及AI用户体验方面的最新进展。\n- [无监督学习](https:\u002F\u002Fpodcasts.apple.com\u002Fil\u002Fpodcast\u002Funsupervised-learning\u002Fid1672188924) 🎧 - 洞察快速发展的AI格局及其对企业和全球的影响。探讨大语言模型的应用、行业趋势及颠覆性技术。\n- [TWIML AI播客（原“本周机器学习”）](https:\u002F\u002Fpodcasts.apple.com\u002Fno\u002Fpodcast\u002Fthe-twiml-ai-podcast-formerly-this-week-in-machine\u002Fid1116303051) 🎧 - 深入探讨AI领域的微调方法、大语言模型的能力与局限，并向该领域的专家学习。\n- [苹果播客上的《AI与未来工作》](https:\u002F\u002Fpodcasts.apple.com\u002Fus\u002Fpodcast\u002Fai-and-the-future-of-work\u002Fid1476885647)：由SC Moatti主持的播客，讨论AI对未来工作的影响。\n- [实用AI：机器学习、数据科学——微调 vs RAG](https:\u002F\u002Fpodcasts.apple.com\u002Fus\u002Fpodcast\u002Ffine-tuning-vs-rag\u002Fid1406537385?i=1000626951912)：本期节目探讨了机器学习和数据科学中微调与检索增强生成之间的对比。\n- [苹果播客上的《无监督学习》](https:\u002F\u002Fpodcasts.apple.com\u002Ffi\u002Fpodcast\u002Funsupervised-learning\u002Fid1672188924)：第20集邀请Anthropic首席执行官Dario Amodei，探讨AGI与AI的未来。\n- [Spotify上的《AI论文解读》播客](https:\u002F\u002Fopen.spotify.com\u002Fshow\u002F2w8DRieJhMGFSTUhnsTVrw)：该播客为你带来计算机科学领域的最新趋势和表现最佳的架构信息。\n- [苹果播客上的《AI今日》](https:\u002F\u002Fpodcasts.apple.com\u002Fin\u002Fpodcast\u002Fthis-day-in-ai-podcast\u002Fid1671087656)：涵盖各类AI相关话题，提供引人入胜的AI世界洞察。\n- [关于评估LLM应用的一切 \u002F\u002F Shahul Es \u002F\u002F #179 MLOps](https:\u002F\u002Fplayer.fm\u002Fseries\u002Fmlopscommunity\u002Fall-about-evaluating-llm-applications-shahul-es-mlops-podcast-179)：在本集中，Shahul Es分享了他在开源模型评估方面的专业知识，包括调试、故障排除和基准测试等方面的见解。\n- [苹果播客上的《AI每日》](https:\u002F\u002Fpodcasts.apple.com\u002Fus\u002Fpodcast\u002Fai-daily\u002Fid1686002118)：由Conner、Ethan和Farb主持，该播客探索各种有趣的AI相关故事。\n- [Yannic Kilcher视频（仅音频）| Spotify播客](https:\u002F\u002Fopen.spotify.com\u002Fshow\u002F6cHS7bXU2JPLTgjA0z0xNz)：Yannic Kilcher讨论机器学习研究论文、编程以及AI对社会的更广泛影响。\n- [LessWrong精选播客 | Spotify播客](https:\u002F\u002Fopen.spotify.com\u002Fshow\u002F7vqBzO0ejqiLiXyTECEeBY)：这是LessWrong精选通讯中文章的音频版本。\n- [苹果播客上的《SAI：安全与AI播客》](https:\u002F\u002Fpodcasts.apple.com\u002Fil\u002Fpodcast\u002Fsai-the-security-and-ai-podcast\u002Fid1690378369)：一集聚焦于OpenAI的网络安全资助计划。\n\n---\n\n这份Awesome List的初始版本是在[Awesome List生成器](https:\u002F\u002Fgithub.com\u002Falialsaeedi19\u002FGPT-Awesome-List-Maker)的帮助下生成的。它是一个开源的Python工具包，利用GPT模型的强大能力，自动整理并生成与特定主题相关的资源列表起点。","# awesome-llms-fine-tuning 快速上手指南\n\n`awesome-llms-fine-tuning` 并非一个单一的可安装软件包，而是一个精选的大语言模型（LLM）微调资源合集。为了让您快速开始微调实践，本指南将基于该列表中推荐的最流行、易上手的开源框架 **LLaMA-Factory** 和 **H2O LLM Studio** 提供操作指引。这两个工具涵盖了从命令行到图形界面的主流微调需求。\n\n## 环境准备\n\n在开始之前，请确保您的开发环境满足以下要求：\n\n*   **操作系统**: Linux (推荐 Ubuntu 20.04+) 或 macOS。Windows 用户建议使用 WSL2 或 Docker。\n*   **硬件要求**:\n    *   **GPU**: 建议 NVIDIA GPU，显存至少 16GB（全量微调需更高，LoRA\u002FQLoRA 可低至 8GB-12GB）。\n    *   **内存**: 系统 RAM 建议 32GB 以上。\n*   **前置依赖**:\n    *   Python 3.8 - 3.10\n    *   CUDA Toolkit (版本需与 PyTorch 匹配，通常建议 11.8 或 12.1)\n    *   Git\n    *   (可选) Conda 或 Mamba 用于管理虚拟环境\n\n## 安装步骤\n\n以下提供两种主流方案的安装命令。国内开发者推荐使用国内镜像源加速下载。\n\n### 方案一：LLaMA-Factory (命令行\u002F代码友好，支持多种模型)\n\n1.  **克隆项目**:\n    ```bash\n    git clone https:\u002F\u002Fgithub.com\u002Fhiyouga\u002FLLaMA-Factory.git\n    cd LLaMA-Factory\n    ```\n\n2.  **创建虚拟环境并安装依赖**:\n    ```bash\n    conda create -n llama-factory python=3.10\n    conda activate llama-factory\n    ```\n\n3.  **安装核心库 (使用清华\u002F阿里镜像加速)**:\n    ```bash\n    pip install -e \".[torch,metrics]\" -i https:\u002F\u002Fpypi.tuna.tsinghua.edu.cn\u002Fsimple\n    ```\n    *注：若需 Flash Attention 加速，请确保硬件支持并安装 `flash-attn`。*\n\n### 方案二：H2O LLM Studio (无代码图形界面友好)\n\n1.  **克隆项目**:\n    ```bash\n    git clone https:\u002F\u002Fgithub.com\u002Fh2oai\u002Fh2o-llmstudio.git\n    cd h2o-llmstudio\n    ```\n\n2.  **安装依赖**:\n    ```bash\n    pip install -r requirements.txt -i https:\u002F\u002Fpypi.tuna.tsinghua.edu.cn\u002Fsimple\n    ```\n    *或者使用官方提供的 Docker 方式（推荐）*:\n    ```bash\n    docker run --gpus all --shm-size 512m -p 10101:10101 -v $(pwd)\u002Fdata:\u002Fdata ghcr.io\u002Fh2oai\u002Fh2o-llmstudio:latest\n    ```\n\n## 基本使用\n\n### 场景 A：使用 LLaMA-Factory 进行 LoRA 微调\n\n这是目前最流行的参数高效微调方法。以下示例展示如何使用命令行对 `Qwen-7B` 进行指令微调。\n\n1.  **准备数据**:\n    确保你有一个 `alpaca_zh.json` 格式的数据集（该框架内置了示例数据）。\n\n2.  **执行微调命令**:\n    在项目根目录下运行以下命令（以单卡为例）：\n\n    ```bash\n    llamafactory-cli train \\\n        --stage sft \\\n        --do_train \\\n        --model_name_or_path Qwen\u002FQwen-7B-Chat \\\n        --dataset alpaca_zh \\\n        --template qwen \\\n        --finetuning_type lora \\\n        --lora_target q_proj,v_proj \\\n        --output_dir .\u002Fsaves\u002Fqwen-7b\u002Flora \\\n        --overwrite_cache \\\n        --per_device_train_batch_size 4 \\\n        --gradient_accumulation_steps 4 \\\n        --lr_scheduler_type cosine \\\n        --logging_steps 10 \\\n        --save_steps 1000 \\\n        --learning_rate 5e-5 \\\n        --num_train_epochs 3.0 \\\n        --plot_loss \\\n        --fp16\n    ```\n\n3.  **启动推理测试**:\n    微调完成后，可直接加载适配器进行对话测试：\n    ```bash\n    llamafactory-cli chat \\\n        --model_name_or_path Qwen\u002FQwen-7B-Chat \\\n        --adapter_name_or_path .\u002Fsaves\u002Fqwen-7b\u002Flora \\\n        --template qwen \\\n        --finetuning_type lora\n    ```\n\n### 场景 B：使用 H2O LLM Studio 进行可视化微调\n\n适合不希望编写代码的用户。\n\n1.  **启动服务**:\n    如果在本地安装（非 Docker），运行：\n    ```bash\n    python train.py\n    ```\n    终端将显示访问地址，通常为 `http:\u002F\u002Flocalhost:10101`。\n\n2.  **操作流程**:\n    *   **登录**: 浏览器打开上述地址，设置管理员账号。\n    *   **导入数据**: 点击 \"Import Dataset\"，上传 CSV 或 Parquet 格式的指令数据集（需包含 `prompt` 和 `answer` 列）。\n    *   **创建实验**: 点击 \"Create Experiment\"，选择基础模型（如 Llama-2, Falcon 等），配置超参数（Learning Rate, Batch Size, LoRA rank 等）。\n    *   **开始训练**: 点击 \"Run\"，界面将实时展示 Loss 曲线和显存占用。\n    *   **导出模型**: 训练结束后，可在 \"Artifacts\" 中下载微调后的权重文件。\n\n---\n*提示：更多高级用法、特定模型支持列表及最新论文解读，请参阅 `awesome-llms-fine-tuning` 原始仓库中的 Articles & Blogs 和 GitHub projects 章节。*","某金融科技公司数据团队急需将通用大模型改造为精通内部合规术语与业务流程的智能客服助手。\n\n### 没有 awesome-llms-fine-tuning 时\n- **资源分散难筛选**：团队成员需在海量论文、博客和 GitHub 仓库中盲目搜索，耗费数周才找到零散的微调教程，且难以辨别技术方案的时效性。\n- **工具选型试错成本高**：面对 AutoTrain、LLaMA-Factory 等众多框架，缺乏横向对比指南，团队误选了不支持量化训练的旧工具，导致显存溢出且推理速度缓慢。\n- **最佳实践缺失**：由于不了解 LoRA、QLoRA 等参数高效微调的最新策略，初期尝试全量微调，不仅训练周期长达数天，还因过拟合导致模型泛化能力极差。\n- **调试无据可依**：遇到收敛困难或幻觉问题时，找不到针对性的排查案例与评估工具（如 Phoenix），只能凭经验盲目调整超参数。\n\n### 使用 awesome-llms-fine-tuning 后\n- **一站式资源导航**：直接利用整理好的分类列表，快速定位到针对金融场景的 SOTA 论文与高星项目，将技术调研时间从数周压缩至两天。\n- **精准匹配高效框架**：参考列表中关于工具特性的详细描述，迅速选定支持 4-bit 量化与 Flash Attention 的 LLaMA-Factory，在单卡环境下即可启动训练。\n- **复用成熟微调策略**：直接采纳列表中推荐的适配器（Adapter）微调最佳实践，显著降低显存占用，将训练效率提升 10 倍并有效抑制过拟合。\n- **系统化评估调优**：借助推荐的评估工具链与故障排查指南，快速定位数据噪声问题，使模型在内部合规测试集上的准确率大幅提升。\n\nawesome-llms-fine-tuning 通过聚合全球顶尖的微调资源与实战方案，帮助团队避开了重复造轮子的陷阱，实现了从“盲目摸索”到“精准落地”的高效转型。","https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FCurated-Awesome-Lists_awesome-llms-fine-tuning_80f5c772.png","Curated-Awesome-Lists","Curated Awesome Lists","https:\u002F\u002Foss.gittoolsai.com\u002Favatars\u002FCurated-Awesome-Lists_f0912828.png","",null,"https:\u002F\u002Fgithub.com\u002FCurated-Awesome-Lists",508,73,"2026-03-30T19:21:48","未说明","部分项目（如 AutoTrain, lit-gpt）提及支持消费级 GPU（如 24GB 显存），支持 4-bit\u002F8-bit 量化以降低显存需求；具体取决于所选子项目",{"notes":88,"python":85,"dependencies":89},"该仓库是一个资源合集列表，而非单一软件工具。它列出了多个独立的开源项目（如 LLaMA-Factory, AutoTrain, H2O LLM Studio 等），每个项目都有各自独立的运行环境要求。例如，有文章提到可在 24GB 显存的消费级显卡上使用 TRL 和 Flash Attention 进行微调。用户需根据列表中具体选择的项目查阅其对应的文档以获取准确的依赖和配置信息。",[85],[15,14,26,13],[92,93,94,95,96,97,98,99,100,101],"ai","awesome-list","deep-learning","fine-tuning","gpt","large-language-models","llms","machine-learning","nlp","transformers","2026-03-27T02:49:30.150509","2026-04-06T08:48:08.700870",[],[]]