[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"similar-jacobhilton--deep_learning_curriculum":3,"tool-jacobhilton--deep_learning_curriculum":65},[4,17,27,35,48,57],{"id":5,"name":6,"github_repo":7,"description_zh":8,"stars":9,"difficulty_score":10,"last_commit_at":11,"category_tags":12,"status":16},1381,"everything-claude-code","affaan-m\u002Feverything-claude-code","everything-claude-code 是一套专为 AI 编程助手（如 Claude Code、Codex、Cursor 等）打造的高性能优化系统。它不仅仅是一组配置文件，而是一个经过长期实战打磨的完整框架，旨在解决 AI 代理在实际开发中面临的效率低下、记忆丢失、安全隐患及缺乏持续学习能力等核心痛点。\n\n通过引入技能模块化、直觉增强、记忆持久化机制以及内置的安全扫描功能，everything-claude-code 能显著提升 AI 在复杂任务中的表现，帮助开发者构建更稳定、更智能的生产级 AI 代理。其独特的“研究优先”开发理念和针对 Token 消耗的优化策略，使得模型响应更快、成本更低，同时有效防御潜在的攻击向量。\n\n这套工具特别适合软件开发者、AI 研究人员以及希望深度定制 AI 工作流的技术团队使用。无论您是在构建大型代码库，还是需要 AI 协助进行安全审计与自动化测试，everything-claude-code 都能提供强大的底层支持。作为一个曾荣获 Anthropic 黑客大奖的开源项目，它融合了多语言支持与丰富的实战钩子（hooks），让 AI 真正成长为懂上",153609,2,"2026-04-13T11:34:59",[13,14,15],"开发框架","Agent","语言模型","ready",{"id":18,"name":19,"github_repo":20,"description_zh":21,"stars":22,"difficulty_score":23,"last_commit_at":24,"category_tags":25,"status":16},4487,"LLMs-from-scratch","rasbt\u002FLLMs-from-scratch","LLMs-from-scratch 是一个基于 PyTorch 的开源教育项目，旨在引导用户从零开始一步步构建一个类似 ChatGPT 的大型语言模型（LLM）。它不仅是同名技术著作的官方代码库，更提供了一套完整的实践方案，涵盖模型开发、预训练及微调的全过程。\n\n该项目主要解决了大模型领域“黑盒化”的学习痛点。许多开发者虽能调用现成模型，却难以深入理解其内部架构与训练机制。通过亲手编写每一行核心代码，用户能够透彻掌握 Transformer 架构、注意力机制等关键原理，从而真正理解大模型是如何“思考”的。此外，项目还包含了加载大型预训练权重进行微调的代码，帮助用户将理论知识延伸至实际应用。\n\nLLMs-from-scratch 特别适合希望深入底层原理的 AI 开发者、研究人员以及计算机专业的学生。对于不满足于仅使用 API，而是渴望探究模型构建细节的技术人员而言，这是极佳的学习资源。其独特的技术亮点在于“循序渐进”的教学设计：将复杂的系统工程拆解为清晰的步骤，配合详细的图表与示例，让构建一个虽小但功能完备的大模型变得触手可及。无论你是想夯实理论基础，还是为未来研发更大规模的模型做准备",90106,3,"2026-04-06T11:19:32",[15,26,14,13],"图像",{"id":28,"name":29,"github_repo":30,"description_zh":31,"stars":32,"difficulty_score":10,"last_commit_at":33,"category_tags":34,"status":16},3704,"NextChat","ChatGPTNextWeb\u002FNextChat","NextChat 是一款轻量且极速的 AI 助手，旨在为用户提供流畅、跨平台的大模型交互体验。它完美解决了用户在多设备间切换时难以保持对话连续性，以及面对众多 AI 模型不知如何统一管理的痛点。无论是日常办公、学习辅助还是创意激发，NextChat 都能让用户随时随地通过网页、iOS、Android、Windows、MacOS 或 Linux 端无缝接入智能服务。\n\n这款工具非常适合普通用户、学生、职场人士以及需要私有化部署的企业团队使用。对于开发者而言，它也提供了便捷的自托管方案，支持一键部署到 Vercel 或 Zeabur 等平台。\n\nNextChat 的核心亮点在于其广泛的模型兼容性，原生支持 Claude、DeepSeek、GPT-4 及 Gemini Pro 等主流大模型，让用户在一个界面即可自由切换不同 AI 能力。此外，它还率先支持 MCP（Model Context Protocol）协议，增强了上下文处理能力。针对企业用户，NextChat 提供专业版解决方案，具备品牌定制、细粒度权限控制、内部知识库整合及安全审计等功能，满足公司对数据隐私和个性化管理的高标准要求。",87618,"2026-04-05T07:20:52",[13,15],{"id":36,"name":37,"github_repo":38,"description_zh":39,"stars":40,"difficulty_score":10,"last_commit_at":41,"category_tags":42,"status":16},2268,"ML-For-Beginners","microsoft\u002FML-For-Beginners","ML-For-Beginners 是由微软推出的一套系统化机器学习入门课程，旨在帮助零基础用户轻松掌握经典机器学习知识。这套课程将学习路径规划为 12 周，包含 26 节精炼课程和 52 道配套测验，内容涵盖从基础概念到实际应用的完整流程，有效解决了初学者面对庞大知识体系时无从下手、缺乏结构化指导的痛点。\n\n无论是希望转型的开发者、需要补充算法背景的研究人员，还是对人工智能充满好奇的普通爱好者，都能从中受益。课程不仅提供了清晰的理论讲解，还强调动手实践，让用户在循序渐进中建立扎实的技能基础。其独特的亮点在于强大的多语言支持，通过自动化机制提供了包括简体中文在内的 50 多种语言版本，极大地降低了全球不同背景用户的学习门槛。此外，项目采用开源协作模式，社区活跃且内容持续更新，确保学习者能获取前沿且准确的技术资讯。如果你正寻找一条清晰、友好且专业的机器学习入门之路，ML-For-Beginners 将是理想的起点。",85092,"2026-04-10T11:13:16",[26,43,44,45,14,46,15,13,47],"数据工具","视频","插件","其他","音频",{"id":49,"name":50,"github_repo":51,"description_zh":52,"stars":53,"difficulty_score":54,"last_commit_at":55,"category_tags":56,"status":16},5784,"funNLP","fighting41love\u002FfunNLP","funNLP 是一个专为中文自然语言处理（NLP）打造的超级资源库，被誉为\"NLP 民工的乐园”。它并非单一的软件工具，而是一个汇集了海量开源项目、数据集、预训练模型和实用代码的综合性平台。\n\n面对中文 NLP 领域资源分散、入门门槛高以及特定场景数据匮乏的痛点，funNLP 提供了“一站式”解决方案。这里不仅涵盖了分词、命名实体识别、情感分析、文本摘要等基础任务的标准工具，还独特地收录了丰富的垂直领域资源，如法律、医疗、金融行业的专用词库与数据集，甚至包含古诗词生成、歌词创作等趣味应用。其核心亮点在于极高的全面性与实用性，从基础的字典词典到前沿的 BERT、GPT-2 模型代码，再到高质量的标注数据和竞赛方案，应有尽有。\n\n无论是刚刚踏入 NLP 领域的学生、需要快速验证想法的算法工程师，还是从事人工智能研究的学者，都能在这里找到急需的“武器弹药”。对于开发者而言，它能大幅减少寻找数据和复现模型的时间；对于研究者，它提供了丰富的基准测试资源和前沿技术参考。funNLP 以开放共享的精神，极大地降低了中文自然语言处理的开发与研究成本，是中文 AI 社区不可或缺的宝藏仓库。",79857,1,"2026-04-08T20:11:31",[15,43,46],{"id":58,"name":59,"github_repo":60,"description_zh":61,"stars":62,"difficulty_score":54,"last_commit_at":63,"category_tags":64,"status":16},5773,"cs-video-courses","Developer-Y\u002Fcs-video-courses","cs-video-courses 是一个精心整理的计算机科学视频课程清单，旨在为自学者提供系统化的学习路径。它汇集了全球知名高校（如加州大学伯克利分校、新南威尔士大学等）的完整课程录像，涵盖从编程基础、数据结构与算法，到操作系统、分布式系统、数据库等核心领域，并深入延伸至人工智能、机器学习、量子计算及区块链等前沿方向。\n\n面对网络上零散且质量参差不齐的教学资源，cs-video-courses 解决了学习者难以找到成体系、高难度大学级别课程的痛点。该项目严格筛选内容，仅收录真正的大学层级课程，排除了碎片化的简短教程或商业广告，确保用户能接触到严谨的学术内容。\n\n这份清单特别适合希望夯实计算机基础的开发者、需要补充特定领域知识的研究人员，以及渴望像在校生一样系统学习计算机科学的自学者。其独特的技术亮点在于分类极其详尽，不仅包含传统的软件工程与网络安全，还细分了生成式 AI、大语言模型、计算生物学等新兴学科，并直接链接至官方视频播放列表，让用户能一站式获取高质量的教育资源，免费享受世界顶尖大学的课堂体验。",79792,"2026-04-08T22:03:59",[46,26,43,13],{"id":66,"github_repo":67,"name":68,"description_en":69,"description_zh":70,"ai_summary_zh":71,"readme_en":72,"readme_zh":73,"quickstart_zh":74,"use_case_zh":75,"hero_image_url":76,"owner_login":77,"owner_name":78,"owner_avatar_url":79,"owner_bio":80,"owner_company":80,"owner_location":80,"owner_email":80,"owner_twitter":80,"owner_website":81,"owner_url":82,"languages":80,"stars":83,"forks":84,"last_commit_at":85,"license":80,"difficulty_score":54,"env_os":86,"env_gpu":87,"env_ram":87,"env_deps":88,"category_tags":94,"github_topics":80,"view_count":10,"oss_zip_url":80,"oss_zip_packed_at":80,"status":16,"created_at":95,"updated_at":96,"faqs":97,"releases":98},7169,"jacobhilton\u002Fdeep_learning_curriculum","deep_learning_curriculum","Language model alignment-focused deep learning curriculum","deep_learning_curriculum 是一份专注于大语言模型对齐（Alignment）的高级深度学习课程大纲，旨在帮助学习者快速掌握截至 2022 年的前沿技术动态。它主要解决了进阶学习者在从基础理论迈向尖端研究时面临的资源分散、路径模糊以及缺乏系统性指导的痛点，特别是针对大模型价值观对齐这一高难度领域提供了清晰的学习路线。\n\n该资源非常适合具备扎实量化背景（如熟练掌握线性代数、概率论、微积分及 Python 编程）的研究人员、工程师或高阶开发者使用。如果你已了解深度学习基础，希望深入探索大模型内部机制与安全对齐技术，这将是一份极佳的指南。需要注意的是，课程内容极具挑战性，官方强烈建议在导师指导或组建学习小组的情况下进行，不建议独自盲目攻坚。\n\n其独特亮点在于高度聚焦于作者的研究专长——大语言模型对齐，填补了通用深度学习教程与特定前沿研究方向之间的空白。虽然课程本身不追求实时更新至最新日期，但其构建的核心知识框架严谨且深刻，并开放社区贡献以保持活力。对于渴望在 AI 安全与对齐领域深耕的专业人士而言，deep_learning_curriculum 是一座连接理论基础与顶尖","deep_learning_curriculum 是一份专注于大语言模型对齐（Alignment）的高级深度学习课程大纲，旨在帮助学习者快速掌握截至 2022 年的前沿技术动态。它主要解决了进阶学习者在从基础理论迈向尖端研究时面临的资源分散、路径模糊以及缺乏系统性指导的痛点，特别是针对大模型价值观对齐这一高难度领域提供了清晰的学习路线。\n\n该资源非常适合具备扎实量化背景（如熟练掌握线性代数、概率论、微积分及 Python 编程）的研究人员、工程师或高阶开发者使用。如果你已了解深度学习基础，希望深入探索大模型内部机制与安全对齐技术，这将是一份极佳的指南。需要注意的是，课程内容极具挑战性，官方强烈建议在导师指导或组建学习小组的情况下进行，不建议独自盲目攻坚。\n\n其独特亮点在于高度聚焦于作者的研究专长——大语言模型对齐，填补了通用深度学习教程与特定前沿研究方向之间的空白。虽然课程本身不追求实时更新至最新日期，但其构建的核心知识框架严谨且深刻，并开放社区贡献以保持活力。对于渴望在 AI 安全与对齐领域深耕的专业人士而言，deep_learning_curriculum 是一座连接理论基础与顶尖研究的坚实桥梁。","# Deep Learning Curriculum\n\nThis is an advanced curriculum for getting up to speed with some of the latest developments in deep learning, as of July 2022. It is heavily biased towards my own research interests, especially large language model alignment, but it should be of general interest. It is targeted at people with a strong quantitative background who are familiar with the basics of deep learning, but may otherwise be new to the field.\n\n**WARNING**: this curriculum may be extremely challenging to take on alone. It is highly recommended to find a more experienced mentor, or at the very least a study partner. There are suggestions for more accessible alternatives, which have the same prerequisites, below.\n\nI do not intend to try to keep this curriculum up-to-date, but PRs are welcome, although I reserve the right to be picky about what gets included to avoid it becoming too bloated.\n\nCredit to John Schulman for an older curriculum which inspired this and from which I cribbed bits and pieces.\n\n## Pre-prerequisites\n\nBefore studying deep learning, I recommend being comfortable with the very basics of the following topics:\n\n- **Linear algebra**: It's essential to understand how vectors and matrices work, and helpful to understand eigenvalues and eigenvectors.\n- **Probability**: It's essential to understand the rules of probability, expected value and standard deviation, and helpful to understand independence and the normal distribution.\n- **Calculus**: It's essential to understand differentiation and partial differentiation, and helpful to understand the basics of vector calculus including the chain rule and Taylor series.\n- **Programming**: I recommend getting to know Python and numpy.\n- Optional extras:\n    - **Statistics**: It's helpful to understand estimators and standard errors.\n    - **Information theory**: It's helpful to understand information, entropy and KL divergence.\n\nYou do not by any means need to understand the whole of these subjects in great depth in order to approach deep learning, but being very familiar with the basic ideas will make your life easier. The benefit of studying these subjects in depth is that you will gain this familiarity by thinking a lot about the relevant ideas.\n\nThere are many resources for studying these topics. Here are some suggestions:\n\n- 3Blue1Brown's video series on [linear algebra](https:\u002F\u002Fwww.3blue1brown.com\u002Ftopics\u002Flinear-algebra) and [calculus](https:\u002F\u002Fwww.3blue1brown.com\u002Ftopics\u002Fcalculus) - Accessible and focused on the essentials.\n- [Mathematics for Machine Learning Coursera course](https:\u002F\u002Fwww.coursera.org\u002Fspecializations\u002Fmathematics-machine-learning) - A math course with a machine learning focus.\n- [Python for Data Science and Machine Learning Bootcamp](https:\u002F\u002Fwww.udemy.com\u002Fcourse\u002Fpython-for-data-science-and-machine-learning-bootcamp\u002F) - A Python course with a machine learning focus.\n- [Linear Algebra Done Right](https:\u002F\u002Flink.springer.com\u002Fbook\u002F10.1007\u002F978-3-319-11080-6) - A first-principles approach to linear algebra that covers much more than you need to know.\n- [Cover and Thomas](http:\u002F\u002Fstaff.ustc.edu.cn\u002F~cgong821\u002FWiley.Interscience.Elements.of.Information.Theory.Jul.2006.eBook-DDU.pdf) - A classic textbook on information theory, also covering much more than you need to know.\n- [Cracking the Coding interview](https:\u002F\u002Fgithub.com\u002FAvinash987\u002FCoding\u002Fblob\u002Fmaster\u002FCracking-the-Coding-Interview-6th-Edition-189-Programming-Questions-and-Solutions.pdf) - Covers the computer science essentials that you might actually use when programming in the real world, and has a lot of exercises. Also useful for interview preparation.\n- [LeetCode](https:\u002F\u002Fleetcode.com\u002F) - Programming challenges designed for interview preparation. A good way to get general programming practice.\n- [Project Euler](https:\u002F\u002Fprojecteuler.net\u002F) - A series of increasingly challenging mathematical programming puzzles. A fun way to get programming practice for the mathematically inclined.\n- [Learning the Shell](https:\u002F\u002Flinuxcommand.org\u002Flc3_learning_the_shell.php) - A tutorial for the Unix command line.\n\nI recommend putting a significant fraction of your study time into exercises – problems for math, and implementation for programming – since it forces you to think through the ideas for yourself.\n\n## Prerequisites\n\nBefore embarking on this curriculum (or one of the alternatives suggested below), it is necessary to understand the basics of deep learning, including basic machine learning terminology, what neural networks are, and how to train them. Here are some suggested resources for this:\n\n- [Neural Networks and Deep Learning](http:\u002F\u002Fneuralnetworksanddeeplearning.com\u002F) - An online textbook introducing deep learning, with a theoretical focus. Has some nice theoretical exercises.\n- [3Blue1Brown on Neural Networks](https:\u002F\u002Fwww.3blue1brown.com\u002Ftopics\u002Fneural-networks) - A video series explaining neural networks in an intuitive way.\n- [Machine Learning Coursera course](https:\u002F\u002Fwww.coursera.org\u002Flearn\u002Fmachine-learning) - An online lecture series covering both theoretical and practical topics. Has some practical exercises, but they may be a bit too \"fill in the blanks\".\n- [Practical Deep Learning for Coders from fast.ai](https:\u002F\u002Fcourse.fast.ai\u002F) - An online textbook with a practical focus. Has practical exercises.\n- [Full Stack Deep Learning](https:\u002F\u002Ffullstackdeeplearning.com\u002F) - An online deep learning course with lectures and labs.\n\nIn addition to this, I recommend learning [PyTorch](https:\u002F\u002Fpytorch.org\u002Ftutorials\u002Fbeginner\u002Fdeep_learning_60min_blitz.html), and, as an exercise, using it to train a small neural network to do [MNIST](http:\u002F\u002Fyann.lecun.com\u002Fexdb\u002Fmnist\u002F) classification. You should be able to achieve around 99% test accuracy using a 3-layer [convolutional neural network](https:\u002F\u002Fcs231n.github.io\u002Fconvolutional-networks\u002F). You can also use your setup to run a few experiments on some simple methods of [regularization](https:\u002F\u002Fcs231n.github.io\u002Fneural-networks-2\u002F).\n\n## How to use this curriculum\n\nThe curriculum is divided into 9 chapters:\n\n- [Chapter 1: Transformers](1-Transformers.md)\n- [Chapter 2: Scaling Laws](2-Scaling-Laws.md)\n- [Chapter 3: Training at Scale](3-Training-at-Scale.md)\n- [Chapter 4: Optimization](4-Optimization.md)\n- [Chapter 5: Modeling Objectives](5-Modeling-Objectives.md)\n- [Chapter 6: Reinforcement Learning](6-Reinforcement-Learning.md)\n- [Chapter 7: Alignment](7-Alignment.md)\n- [Chapter 8: Interpretability](8-Interpretability.md)\n- [Chapter 9: Adversarial Training](9-Adversarial-Training.md)\n\nThe chapters are **not** necessarily of equal importance. For a typical person, the order of importance is something like: 1, 6, 2, 7, 4, 8, 9, 3, 5.\n\nChapter 1 is helpful for understanding chapters 2, 3, 8 and 9, and chapter 6 may also be somewhat helpful for understanding chapter 7, but otherwise the chapters can be completed in any order.\n\nFor a longer version of Chapter 7, replace it by the [AGI Safety Fundamentals](https:\u002F\u002Fwww.agisafetyfundamentals.com\u002F) technical AI alignment track. For a longer version of Chapter 8, replace it by [Concrete Steps to Get Started in Transformer Mechanistic Interpretability](https:\u002F\u002Fwww.neelnanda.io\u002Fmechanistic-interpretability\u002Fgetting-started).\n\nEach chapter has three sections:\n\n- **Recommended reading**: A small amount of material covering the most basic or important ideas of the chapter. Often a particular section of a paper will be singled out, and it's not necessary to read the rest of the paper.\n- **Optional reading**: Related material that's still very relevant to the chapter, but can be either skimmed or skipped entirely and used as a reference later on.\n- **Suggested exercise**: An idea for an exercise, generally implementation-focused, to help drive some of the ideas of the chapter home. It is generally more important to do some sort of exercise than to follow the exact exercise suggested.\n\nEach chapter should take between half a week and 2 weeks of full-time study, depending on the chapter and how much depth you go into, but don't be discouraged if it takes longer than this.\n\nOnce you have completed an exercise, you can take a look at other people's solutions [here](Solutions.md). You are welcome to add a link to your own solutions to that page by submitting a PR.\n\nThe main other useful skill for working in this area is software engineering, especially working with distributed systems and large, shared codebases. I think the most common way to improve at this is through professional experience, but it can also be done by contributing to open source projects. In order to pick up best practices, you generally want to be reading other people's code and having others review your code.\n\n## Alternatives to this curriculum\n\nThis curriculum may be very challenging, especially without mentorship. A more realistic alternative for most people is to work on larger programming projects involving deep learning, and\u002For to work through more advanced textbooks or online courses. These can be engaging, provide structure, and last a long time. The downside is that they may be less focused on the most relevant material. But it's always possible to return to this curriculum at a later date.\n\nAn example of a larger programming project I worked on was training a neural network to play [backgammon](https:\u002F\u002Fgithub.com\u002Fjacobhilton\u002Fbackgammon) using TD-learning, but you should choose something you're motivated by. It's also a good idea to create a short write-up of the project and any experiments involved.\n\nSome textbook suggestions:\n\n- [Goodfellow et al](https:\u002F\u002Fwww.deeplearningbook.org\u002F) - Probably still the best deep learning-specific textbook, although it can be unclear in places. Has no exercises.\n- [Jared Kaplan's notes for physicists](https:\u002F\u002Fsites.krieger.jhu.edu\u002Fjared-kaplan\u002Ffiles\u002F2019\u002F04\u002FContemporaryMLforPhysicists.pdf) - An extended introduction to deep learning from a fairly theoretical perspective. Also has no exercises.\n- [Sutton & Barto](http:\u002F\u002Fwww.incompleteideas.net\u002Fbook\u002Fthe-book.html) - A good introduction to reinforcement learning from first principles. Has exercises.\n- [Murphy](https:\u002F\u002Fwww.google.com\u002Fsearch?q=machine+learning+a+probabilistic+perspective), [Bishop](https:\u002F\u002Fwww.google.com\u002Fsearch?q=pattern+recognition+and+machine+learning) and [ESL](https:\u002F\u002Fwww.google.com\u002Fsearch?q=elements+of+statistical+learning) - Classic machine learning textbooks. These cover a lot of material that isn't that relevant to deep learning, but it can be nice to have a broader perspective, and they have plenty of exercises. Overall they wouldn't be my first choice but they can be useful.\n\nSome suggestions for more advanced online courses:\n\n- [Deep Learning Specialization from deeplearning.ai](https:\u002F\u002Fwww.deeplearning.ai\u002Fprogram\u002Fdeep-learning-specialization\u002F) - A much longer extension of the machine learning Coursera course linked above.\n- [Deep Learning Udacity course](https:\u002F\u002Fwww.udacity.com\u002Fcourse\u002Fdeep-learning-nanodegree--nd101) - Another longer deep learning course.\n- [Reinforcement Learning Coursera course](https:\u002F\u002Fwww.ualberta.ca\u002Fadmissions-programs\u002Fonline-courses\u002Freinforcement-learning\u002Findex.html) - A reinforcement learning course based on Sutton & Barto.\n\n## Additional advice\n\nThese are miscellaneous opinions based on personal experience, so take them with a pinch of salt. [An Opinionated Guide to ML Research](http:\u002F\u002Fjoschu.net\u002Fblog\u002Fopinionated-guide-ml-research.html) is also worth a read. For advice on pursuing a career in technical AI alignment, I recommend [this guide](https:\u002F\u002Fforum.effectivealtruism.org\u002Fposts\u002F7WXPkpqKGKewAymJf\u002Fhow-to-pursue-a-career-in-technical-ai-alignment).\n\nI've emphasized exercises throughout this page, because they force you to think ideas through in a way that passive learning doesn't. In my experience, this is especially important when learning about unfamiliar topics. Once you have built up experience in an area, it's much easier to remember something in that area that you've only seen or heard once, because of how it fits into your existing web of knowledge.\n\nTo be productive, physical and mental health are paramount. Beyond that, there's no good substitute for intrinsic motivation. Productivity tricks can be useful for getting laborious work done when necessary, but can only do so much. Intrinsic motivation can come both from higher-level excitement about a project, as well as from lower-level flow states (which can be common with programming). When it doesn't matter much what you're working on because you're mostly learning, choose things you'll be intrinsically motivated by, as that's the easiest way to excel. But exciting areas aren't always important, and so in the long run, you'll need to cultivate excitement about important areas by learning more about them.\n\nVertical integration can be powerful, because abstractions are generally leaky. The most effective researchers have an understanding of their entire research stack, from the ins and outs of different benchmarks through to the details of GPU caches. Especially early on in your career, learning as much as you can about everything that's relevant is probably a better strategy than focusing only on what you need to get by.\n\nAs an important special case of the above principle, if you want to have a positive impact on the world with your research, I highly recommend focusing on how best to achieve this as part of your studies. Choosing the right problems to work on can be just as important as the quality of your execution, even if you end up spending most of your time on the latter. Moreover, it's not always enough to defer to others on such questions, since your higher-level motivations will influence your lower-level decisions, not to mention the fact that other people can be wrong or hard to understand.\n\nI personally expect AI to have a far greater impact on the world in the coming decades than it is having now, and that this is enough to outweigh the urgency and tractability of present-day problems (though I also see a lot in common between both sets of problems). For more discussion of this perspective, I recommend [Cold Takes's \"most important century\" series](https:\u002F\u002Fwww.cold-takes.com\u002Fmost-important-century\u002F). There also used to be the [Alignment Newsletter](https:\u002F\u002Frohinshah.com\u002Falignment-newsletter\u002F), which was good for staying up-to-date with related research, but it hasn't been active in a while.\n","# 深度学习课程大纲\n\n这是截至2022年7月的高级课程大纲，旨在帮助你快速掌握深度学习领域的最新进展。本课程大纲 heavily 倾向于我个人的研究兴趣，尤其是大型语言模型对齐方向，但也具有普遍参考价值。它面向具备扎实定量背景、熟悉深度学习基础知识，但可能刚接触该领域的人群。\n\n**警告**：单独完成本课程可能会极具挑战性。强烈建议寻找一位经验丰富的导师，或至少一位学习伙伴。下方也提供了一些门槛较低的替代方案，它们同样需要满足相同的前置条件。\n\n我无意持续更新本课程大纲，不过欢迎提交 PR，但我保留对内容筛选的权利，以避免其变得过于臃肿。\n\n感谢 John Schulman 提供的一份较早的课程大纲，它启发了本课程，并且我在其中借鉴了不少内容。\n\n## 前置预备知识\n\n在开始学习深度学习之前，建议你对以下主题的最基础内容有较为熟练的掌握：\n\n- **线性代数**：理解向量和矩阵的基本运算至关重要，了解特征值和特征向量则会更有帮助。\n- **概率论**：掌握概率的基本规则、期望值和标准差是必需的；理解独立性以及正态分布也有助益。\n- **微积分**：理解导数和偏导数是必要的；掌握向量微积分的基础知识，包括链式法则和泰勒级数，将大有裨益。\n- **编程**：推荐熟悉 Python 和 NumPy。\n- 可选补充：\n  - **统计学**：理解估计量和标准误差会很有帮助。\n  - **信息论**：了解信息量、熵和 KL 散度的概念也会有所帮助。\n\n你并不需要对这些学科进行全面而深入的学习才能进入深度学习领域，但对基本概念的熟悉无疑会使你的学习过程更加顺畅。深入学习这些学科的好处在于，你会通过反复思考相关概念而自然地建立起这种熟悉感。\n\n针对这些主题，有许多优质的学习资源可供选择。以下是一些建议：\n\n- 3Blue1Brown 的 [线性代数](https:\u002F\u002Fwww.3blue1brown.com\u002Ftopics\u002Flinear-algebra) 和 [微积分](https:\u002F\u002Fwww.3blue1brown.com\u002Ftopics\u002Fcalculus) 视频系列——通俗易懂，聚焦核心要点。\n- [Coursera 机器学习数学课程](https:\u002F\u002Fwww.coursera.org\u002Fspecializations\u002Fmathematics-machine-learning)——一门以机器学习为导向的数学课程。\n- [Python 数据科学与机器学习训练营](https:\u002F\u002Fwww.udemy.com\u002Fcourse\u002Fpython-for-data-science-and-machine-learning-bootcamp\u002F)——一门以机器学习为重点的 Python 课程。\n- 《线性代数应该这样学》（Linear Algebra Done Right）——一本从第一性原理出发的线性代数教材，内容远超入门所需。\n- Cover 和 Thomas 的《信息论基础》——一本经典的信息论教材，涵盖的内容同样超出入门需求。\n- 《破解编码面试》——涵盖了实际编程中可能用到的计算机科学基础知识，并配有大量习题，同时对面试准备也很有帮助。\n- LeetCode——专为面试准备设计的编程挑战平台，是提升通用编程能力的好方式。\n- Project Euler——一系列难度递增的数学编程谜题，适合对数学感兴趣的学习者，以有趣的方式锻炼编程技能。\n- 《学习 Shell》——Unix 命令行操作教程。\n\n建议将大部分学习时间投入到练习中——数学方面做题，编程方面动手实现——因为这能迫使你真正去思考并内化所学内容。\n\n## 前置条件\n\n在开始本课程（或下方建议的替代方案）之前，你需要掌握深度学习的基础知识，包括机器学习的基本术语、什么是神经网络以及如何训练它们。以下是一些推荐的学习资源：\n\n- [神经网络与深度学习](http:\u002F\u002Fneuralnetworksanddeeplearning.com\u002F)——一本在线深度学习入门教材，侧重理论讲解，并配有不错的理论习题。\n- 3Blue1Brown 的 [神经网络专题](https:\u002F\u002Fwww.3blue1brown.com\u002Ftopics\u002Fneural-networks)——一系列以直观方式解释神经网络的视频。\n- [Coursera 机器学习课程](https:\u002F\u002Fwww.coursera.org\u002Flearn\u002Fmachine-learning)——一门涵盖理论与实践的在线讲座系列，包含一些实践练习，但可能略显“填空式”。\n- fast.ai 的 [面向程序员的实用深度学习](https:\u002F\u002Fcourse.fast.ai\u002F)——一本注重实践的在线教材，配有大量实操练习。\n- Full Stack Deep Learning——一门包含讲座和实验环节的在线深度学习课程。\n\n此外，建议学习 PyTorch，并将其作为练习，用它来训练一个小型神经网络完成 MNIST 手写数字分类任务。使用三层卷积神经网络，你应该能够达到约 99% 的测试准确率。你还可以利用现有环境，尝试几种简单的正则化方法进行实验。\n\n## 如何使用本课程\n\n本课程共分为9个章节：\n\n- [第1章：Transformer](1-Transformers.md)\n- [第2章：规模法则](2-Scaling-Laws.md)\n- [第3章：大规模训练](3-Training-at-Scale.md)\n- [第4章：优化](4-Optimization.md)\n- [第5章：建模目标](5-Modeling-Objectives.md)\n- [第6章：强化学习](6-Reinforcement-Learning.md)\n- [第7章：对齐](7-Alignment.md)\n- [第8章：可解释性](8-Interpretability.md)\n- [第9章：对抗训练](9-Adversarial-Training.md)\n\n这些章节的**重要性并不一定相等**。对于一般学习者而言，其重要性排序大致为：1、6、2、7、4、8、9、3、5。\n\n第1章有助于理解第2、3、8和9章的内容，而第6章也可能在一定程度上帮助理解第7章；除此之外，各章节可以按任意顺序学习。\n\n若希望深入学习第7章，可用[AGI安全基础](https:\u002F\u002Fwww.agisafetyfundamentals.com\u002F)的技术型AI对齐路径替代；若希望深入学习第8章，则可用[开启Transformer机制可解释性的具体步骤](https:\u002F\u002Fwww.neelnanda.io\u002Fmechanistic-interpretability\u002Fgetting-started)替代。\n\n每个章节包含三个部分：\n\n- **推荐阅读**：少量材料，涵盖该章节中最基础或最重要的内容。通常会单独选取某篇论文中的特定部分，其余内容无需通读。\n- **可选阅读**：与本章相关但并非核心的内容，可略读或直接跳过，日后作为参考。\n- **建议练习**：一个以实践为主的练习思路，旨在加深对该章节核心概念的理解。完成某种形式的练习往往比严格按照建议的练习来得更重要。\n\n每个章节的学习时间约为半周至两周，具体时长取决于章节内容及深入程度，但即便耗时更久也不必气馁。\n\n完成练习后，可在此处查看他人的解答：[Solutions.md]。欢迎通过提交PR的方式，在该页面添加您自己的解答链接。\n\n在这一领域工作所需的另一项重要技能是软件工程，尤其是分布式系统和大型共享代码库的开发经验。提升这方面能力最常见的方式是积累专业经验，也可以通过参与开源项目来实现。为了掌握最佳实践，通常需要阅读他人的代码，并请他人审查自己的代码。\n\n## 本课程的替代方案\n\n本课程可能颇具挑战性，尤其是在缺乏导师指导的情况下。对大多数人而言，更为现实的选择是参与涉及深度学习的大型编程项目，或者研读更高级的教材或在线课程。这些方式更具吸引力、结构化且持续时间较长。缺点在于，它们可能不如本课程那样聚焦于最相关的主题。不过，您仍可在日后随时回到本课程继续学习。\n\n我曾参与的一个大型编程项目是利用TD学习训练神经网络来玩【双陆棋】（[GitHub链接](https:\u002F\u002Fgithub.com\u002Fjacobhilton\u002Fbackgammon)），但您应选择自己真正感兴趣的课题。同时，为该项目及其相关实验撰写一份简短的总结报告也是不错的选择。\n\n以下是一些教材推荐：\n\n- [Goodfellow等著《深度学习》](https:\u002F\u002Fwww.deeplearningbook.org\u002F)——目前仍是最好的深度学习专业教材，尽管部分内容较为晦涩。该书未提供习题。\n- [Jared Kaplan为物理学家撰写的笔记](https:\u002F\u002Fsites.krieger.jhu.edu\u002Fjared-kaplan\u002Ffiles\u002F2019\u002F04\u002FContemporaryMLforPhysicists.pdf)——从较为理论化的角度出发，对深度学习进行了深入介绍。同样没有习题。\n- [Sutton & Barto《强化学习导论》](http:\u002F\u002Fwww.incompleteideas.net\u002Fbook\u002Fthe-book.html)——一本基于基本原理的优秀强化学习入门书籍，附有习题。\n- [Murphy《机器学习：概率视角》](https:\u002F\u002Fwww.google.com\u002Fsearch?q=machine+learning+a+probabilistic+perspective)、[Bishop《模式识别与机器学习》](https:\u002F\u002Fwww.google.com\u002Fsearch?q=pattern+recognition+and+machine+learning)以及[ESL《统计学习要素》](https:\u002F\u002Fwww.google.com\u002Fsearch?q=elements+of+statistical+learning)——经典的机器学习教材。这些书籍涵盖了大量与深度学习关联不大的内容，但有助于拓宽视野，且习题丰富。总体而言，它们并非我的首选，但在某些情况下仍具参考价值。\n\n此外，还有一些更高级的在线课程可供参考：\n\n- [deeplearning.ai推出的深度学习专项课程](https:\u002F\u002Fwww.deeplearning.ai\u002Fprogram\u002Fdeep-learning-specialization\u002F)——这是上述Coursera机器学习课程的大幅扩展版。\n- [Udacity深度学习纳米学位课程](https:\u002F\u002Fwww.udacity.com\u002Fcourse\u002Fdeep-learning-nanodegree--nd101)——另一门较长时间的深度学习课程。\n- [阿尔伯塔大学Coursera强化学习课程](https:\u002F\u002Fwww.ualberta.ca\u002Fadmissions-programs\u002Fonline-courses\u002Freinforcement-learning\u002Findex.html)——基于Sutton & Barto教材的强化学习课程。\n\n## 补充建议\n\n以下是一些基于个人经验的零散看法，仅供参考。[一份带有主观色彩的机器学习研究指南](http:\u002F\u002Fjoschu.net\u002Fblog\u002Fopinionated-guide-ml-research.html) 也值得一读。若想了解如何从事技术性的人工智能对齐相关职业，我推荐 [这份指南](https:\u002F\u002Fforum.effectivealtruism.org\u002Fposts\u002F7WXPkpqKGKewAymJf\u002Fhow-to-pursue-a-career-in-technical-ai-alignment)。\n\n我在整页内容中都强调了练习的重要性，因为与被动式学习不同，练习能够迫使你深入思考并理清思路。根据我的经验，这一点在接触陌生领域时尤为重要。一旦你在某个领域积累了足够的经验，即便只见过或听过一次的内容，也会更容易被记住——这是因为它已经融入了你现有的知识网络之中。\n\n要保持高效，身心健康至关重要。除此之外，内在动机是无可替代的。一些提高效率的小技巧在不得不完成繁重任务时或许有用，但其作用终究有限。内在动机既可以来自对项目的高层次热情，也可以源于低层次的“心流”状态（编程时尤其常见）。当你的主要目标只是学习新东西、具体做什么并不太重要时，不妨选择那些能激发你内在动力的事物，因为这是取得卓越成果最简单的方式。然而，令人兴奋的领域并不总是最重要的；从长远来看，你仍需通过深入了解重要议题来培养对它们的兴趣与热情。\n\n纵向整合往往非常有效，因为抽象层通常存在泄漏效应。最顶尖的研究者往往对其整个研究链条都有深刻理解，从各类基准测试的细节到 GPU 缓存的具体机制，无所不包。尤其是在职业生涯早期，尽可能多地学习所有相关知识，往往比仅仅专注于眼前所需更为明智。\n\n作为上述原则的一个重要特例：如果你想通过自己的研究对世界产生积极影响，我强烈建议你在学习过程中就思考如何以最佳方式实现这一目标。选择合适的课题，其重要性甚至可能不亚于执行的质量——即便你最终将大部分时间花在后者上。此外，在这类问题上一味依赖他人并不足够，因为你的高层次动机会影响低层次的决策；更何况，他人的判断也可能出错，或者难以理解。\n\n我个人认为，在接下来的几十年里，人工智能对世界的影响将远远超过当前水平，而这种潜在的巨大影响足以超越当下问题的紧迫性和可解决性（尽管我也看到这两类问题之间存在许多共通之处）。如需进一步探讨这一观点，我推荐 Cold Takes 的“最重要的一百年”系列文章：[冷思考：“最重要的一百年”系列](https:\u002F\u002Fwww.cold-takes.com\u002Fmost-important-century\u002F)。过去还有一份名为《对齐简报》的通讯（[Alignment Newsletter](https:\u002F\u002Frohinshah.com\u002Falignment-newsletter\u002F)），对于及时了解相关研究动态很有帮助，不过目前已暂停更新。","# Deep Learning Curriculum 快速上手指南\n\n`deep_learning_curriculum` 并非一个可安装的软件包或框架，而是一份**高级深度学习课程大纲**（截至 2022 年 7 月）。它旨在引导具备定量背景的开发者深入理解大语言模型对齐、Transformer 架构等前沿技术。本指南将帮助你配置学习环境并开始按照大纲进行学习。\n\n## 环境准备\n\n在开始本课程之前，你需要具备扎实的数学基础（线性代数、概率论、微积分）并熟悉 Python 编程。以下是运行课程中建议练习所需的系统环境和依赖。\n\n### 系统要求\n- **操作系统**: Linux, macOS 或 Windows (推荐 WSL2)\n- **Python 版本**: 3.8 或更高\n- **硬件**: 建议使用支持 CUDA 的 NVIDIA GPU 以加速模型训练实验（如 MNIST 分类、Transformer 复现）\n\n### 前置依赖\n核心依赖包括 `PyTorch`（课程推荐框架）、`NumPy` 以及用于阅读和运行代码示例的通用工具。\n\n```bash\n# 检查 Python 版本\npython --version\n\n# 检查 pip 版本\npip --version\n```\n\n## 安装步骤\n\n由于这是一个学习大纲而非单一工具，\"安装\"过程实际上是搭建一个适合进行深度学习实验的开发环境。推荐使用国内镜像源加速安装。\n\n### 1. 创建虚拟环境\n建议使用 `conda` 或 `venv` 隔离环境。\n\n```bash\n# 使用 conda 创建环境 (推荐)\nconda create -n dl_curriculum python=3.9\nconda activate dl_curriculum\n\n# 或者使用 venv\npython -m venv dl_curriculum\nsource dl_curriculum\u002Fbin\u002Factivate  # Linux\u002FmacOS\n# dl_curriculum\\Scripts\\activate   # Windows\n```\n\n### 2. 安装深度学习框架\n课程强烈建议使用 **PyTorch**。以下命令使用清华大学开源软件镜像站加速安装。\n\n```bash\n# 安装 PyTorch (CPU 版本，适合入门)\npip install torch torchvision torchaudio -i https:\u002F\u002Fpypi.tuna.tsinghua.edu.cn\u002Fsimple\n\n# 安装 PyTorch (GPU 版本，需根据实际 CUDA 版本调整，以下为 CUDA 11.8 示例)\n# pip install torch torchvision torchaudio --index-url https:\u002F\u002Fdownload.pytorch.org\u002Fwhl\u002Fcu118 -i https:\u002F\u002Fpypi.tuna.tsinghua.edu.cn\u002Fsimple\n\n# 安装其他常用数据科学库\npip install numpy matplotlib jupyterlab tqdm -i https:\u002F\u002Fpypi.tuna.tsinghua.edu.cn\u002Fsimple\n```\n\n### 3. 获取课程资料\n克隆课程仓库以获取详细的章节阅读材料和练习建议。\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002Fjacobhilton\u002Fdeep_learning_curriculum.git\ncd deep_learning_curriculum\n```\n\n## 基本使用\n\n本课程的使用方式是**按章节阅读推荐论文并完成编程练习**。以下是基于课程 \"Prerequisites\" 部分的最简单入门示例：使用 PyTorch 训练一个神经网络完成 MNIST 手写数字分类。\n\n### 1. 启动开发环境\n进入课程目录并启动 Jupyter Lab 以便交互式学习。\n\n```bash\njupyter lab\n```\n\n### 2. 基础练习示例 (MNIST 分类)\n在 Jupyter Notebook 中新建文件，运行以下代码验证环境并完成任务目标（达到约 99% 的测试准确率）。\n\n```python\nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torchvision import datasets, transforms\nfrom torch.utils.data import DataLoader\n\n# 1. 数据预处理与加载\ntransform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))])\ntrain_dataset = datasets.MNIST(root='.\u002Fdata', train=True, download=True, transform=transform)\ntest_dataset = datasets.MNIST(root='.\u002Fdata', train=False, download=True, transform=transform)\n\ntrain_loader = DataLoader(train_dataset, batch_size=64, shuffle=True)\ntest_loader = DataLoader(test_dataset, batch_size=1000, shuffle=False)\n\n# 2. 定义简单的卷积神经网络 (CNN)\nclass SimpleCNN(nn.Module):\n    def __init__(self):\n        super(SimpleCNN, self).__init__()\n        self.conv1 = nn.Conv2d(1, 32, kernel_size=3, padding=1)\n        self.conv2 = nn.Conv2d(32, 64, kernel_size=3, padding=1)\n        self.fc1 = nn.Linear(64 * 7 * 7, 128)\n        self.fc2 = nn.Linear(128, 10)\n        self.relu = nn.ReLU()\n        self.pool = nn.MaxPool2d(2, 2)\n\n    def forward(self, x):\n        x = self.pool(self.relu(self.conv1(x)))\n        x = self.pool(self.relu(self.conv2(x)))\n        x = x.view(-1, 64 * 7 * 7)\n        x = self.relu(self.fc1(x))\n        return self.fc2(x)\n\nmodel = SimpleCNN()\ncriterion = nn.CrossEntropyLoss()\noptimizer = optim.Adam(model.parameters(), lr=0.001)\n\n# 3. 训练循环\ndef train(epoch):\n    model.train()\n    for batch_idx, (data, target) in enumerate(train_loader):\n        optimizer.zero_grad()\n        output = model(data)\n        loss = criterion(output, target)\n        loss.backward()\n        optimizer.step()\n\n# 4. 测试循环\ndef test():\n    model.eval()\n    correct = 0\n    with torch.no_grad():\n        for data, target in test_loader:\n            output = model(data)\n            pred = output.argmax(dim=1, keepdim=True)\n            correct += pred.eq(target.view_as(pred)).sum().item()\n    \n    accuracy = 100. * correct \u002F len(test_loader.dataset)\n    print(f'Test Accuracy: {correct}\u002F{len(test_loader.dataset)} ({accuracy:.2f}%)')\n\n# 执行训练与测试\nfor epoch in range(1, 4):  # 训练 3 个 epoch\n    train(epoch)\n    test()\n```\n\n### 3. 进阶学习路径\n完成上述基础验证后，请按照仓库中的 `README.md` 指引，依次深入学习以下核心章节：\n\n1.  **Transformers**: 理解现代大模型的基础架构。\n2.  **Scaling Laws**: 学习模型规模与性能的关系。\n3.  **Alignment**: 深入研究大语言模型的对齐技术（课程重点）。\n\n你可以随时参考仓库中的 `Solutions.md` 查看其他人完成的练习方案，并通过提交 PR 分享你自己的代码实现。","某拥有扎实数学基础的算法工程师试图从传统深度学习转型，专注于大语言模型（LLM）的对齐（Alignment）研究。\n\n### 没有 deep_learning_curriculum 时\n- **学习路径迷茫**：面对海量且碎片化的最新论文，难以分辨哪些是理解对齐技术所必需的核心内容，导致在无关细节上浪费大量时间。\n- **前置知识断层**：虽然熟悉基础深度学习，但对信息论中的 KL 散度或高阶向量微积分等对齐领域关键概念理解不深，阅读核心论文时频频卡壳。\n- **缺乏实战指引**：独自钻研高难度课题时没有明确的进阶阶梯，容易因挫败感而放弃，且找不到合适的替代性入门资源作为缓冲。\n- **知识体系陈旧**：依赖过时的教程，无法及时获取截至 2022 年 7 月后的最新技术动态，导致研究起点落后于社区前沿。\n\n### 使用 deep_learning_curriculum 后\n- **路径清晰高效**：直接遵循专为大模型对齐设计的进阶大纲，精准锁定关键文献，将原本数月的摸索期缩短为几周的系统化学习。\n- **基础针对性补强**：根据清单中明确列出的线性代数、概率论及信息论具体知识点（如熵与估计量），快速查漏补缺，扫清阅读障碍。\n- **难度分级合理**：利用课程中提供的“更易上手”的替代方案建议，在挑战高难内容前建立信心，有效降低了独自研究的门槛。\n- **紧跟前沿趋势**：直接切入基于最新研究成果构建的知识体系，确保掌握的技术栈与当前主流对齐研究方向高度一致。\n\ndeep_learning_curriculum 通过提供一条经过筛选、侧重对齐技术且兼顾数学深度的学习路径，帮助量化背景人才高效跨越从基础深度学习到前沿大模型研究的鸿沟。","https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fjacobhilton_deep_learning_curriculum_2f3d825a.png","jacobhilton","Jacob Hilton","https:\u002F\u002Foss.gittoolsai.com\u002Favatars\u002Fjacobhilton_b940a9a3.png",null,"https:\u002F\u002Fwww.jacobh.co.uk\u002F","https:\u002F\u002Fgithub.com\u002Fjacobhilton",1584,120,"2026-04-12T18:27:54","","未说明",{"notes":89,"python":90,"dependencies":91},"该工具是一个深度学习课程大纲（文档\u002F资源列表），而非可执行的软件程序，因此没有具体的操作系统、GPU、内存或依赖库版本要求。用户需自行搭建深度学习环境（推荐阅读中建议使用 PyTorch）来完成课程中的编程练习。前置知识要求包括线性代数、概率论、微积分和编程基础。","推荐熟悉 Python 和 numpy，具体版本未说明",[92,93],"numpy","PyTorch",[15,46],"2026-03-27T02:49:30.150509","2026-04-14T00:02:44.844017",[],[]]