我读过的论文

我正在尝试一项新计划——每周读一篇论文。这个仓库将存放所有这些论文以及相关的摘要和笔记。

论文列表

• Toolformer - Language Models Can Teach Themselves to Use Tools
• Hints for Computer System Design
• Synthesized Policies for Transfer and Adaptation across Tasks and Environments
• Deep Neural Networks for YouTube Recommendations
• The Tail at Scale
• Practical Lessons from Predicting Clicks on Ads at Facebook
• Ad Click Prediction - a View from the Trenches
• Anatomy of Catastrophic Forgetting - Hidden Representations and Task Semantics
• When Do Curricula Work?
• Continual learning with hypernetworks
• Zero-shot Learning by Generating Task-specific Adapters
• HyperNetworks
• Energy-based Models for Continual Learning
• GPipe - Easy Scaling with Micro-Batch Pipeline Parallelism
• Compositional Explanations of Neurons
• Design patterns for container-based distributed systems
• Cassandra - a decentralized structured storage system
• CAP twelve years later - How the rules have changed
• Consistency Tradeoffs in Modern Distributed Database System Design
• Exploring Simple Siamese Representation Learning
• Data Management for Internet-Scale Single-Sign-On
• Searching for Build Debt - Experiences Managing Technical Debt at Google
• One Solution is Not All You Need - Few-Shot Extrapolation via Structured MaxEnt RL
• Learning Explanations That Are Hard To Vary
• Remembering for the Right Reasons - Explanations Reduce Catastrophic Forgetting
• A Foliated View of Transfer Learning
• Harvest, Yield, and Scalable Tolerant Systems
• MONet - Unsupervised Scene Decomposition and Representation
• Revisiting Fundamentals of Experience Replay
• Deep Reinforcement Learning and the Deadly Triad
• Alpha Net: Adaptation with Composition in Classifier Space
• Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
• Gradient Surgery for Multi-Task Learning
• GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks
• TaskNorm: Rethinking Batch Normalization for Meta-Learning
• Averaging Weights leads to Wider Optima and Better Generalization
• Decentralized Reinforcement Learning: Global Decision-Making via Local Economic Transactions
• When to use parametric models in reinforcement learning?
• Network Randomization - A Simple Technique for Generalization in Deep Reinforcement Learning
• On the Difficulty of Warm-Starting Neural Network Training
• Supervised Contrastive Learning
• CURL - Contrastive Unsupervised Representations for Reinforcement Learning
• Competitive Training of Mixtures of Independent Deep Generative Models
• What Does Classifying More Than 10,000 Image Categories Tell Us?
• mixup - Beyond Empirical Risk Minimization
• ELECTRA - Pre-training Text Encoders as Discriminators Rather Than Generators
• Gradient based sample selection for online continual learning
• Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One
• Massively Multilingual Neural Machine Translation in the Wild - Findings and Challenges
• Observational Overfitting in Reinforcement Learning
• Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML
• Accurate, Large Minibatch SGD - Training ImageNet in 1 Hour
• Superposition of many models into one
• Towards a Unified Theory of State Abstraction for MDPs
• ALBERT - A Lite BERT for Self-supervised Learning of Language Representations
• Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model
• Contrastive Learning of Structured World Models
• Gossip based Actor-Learner Architectures for Deep RL
• How to train your MAML
• PHYRE - A New Benchmark for Physical Reasoning
• Large Memory Layers with Product Keys
• Abductive Commonsense Reasoning
• Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models
• Assessing Generalization in Deep Reinforcement Learning
• Quantifying Generalization in Reinforcement Learning
• Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks
• Measuring abstract reasoning in neural networks
• Hamiltonian Neural Networks
• Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations
• Meta-Reinforcement Learning of Structured Exploration Strategies
• Relational Reinforcement Learning
• Good-Enough Compositional Data Augmentation
• Multiple Model-Based Reinforcement Learning
• Towards a natural benchmark for continual learning
• Meta-Learning Update Rules for Unsupervised Representation Learning
• GNN Explainer - A Tool for Post-hoc Explanation of Graph Neural Networks
• To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks
• Model Primitive Hierarchical Lifelong Reinforcement Learning
• TuckER - Tensor Factorization for Knowledge Graph Completion
• Linguistic Knowledge as Memory for Recurrent Neural Networks
• Diversity is All You Need - Learning Skills without a Reward Function
• Modular meta-learning
• Hierarchical RL Using an Ensemble of Proprioceptive Periodic Policies
• Efficient Lifelong Learningi with A-GEM
• Pre-training Graph Neural Networks with Kernels
• Smooth Loss Functions for Deep Top-k Classification
• Hindsight Experience Replay
• Representation Tradeoffs for Hyperbolic Embeddings
• Learned Optimizers that Scale and Generalize
• One-shot Learning with Memory-Augmented Neural Networks
• BabyAI - First Steps Towards Grounded Language Learning With a Human In the Loop
• Poincaré Embeddings for Learning Hierarchical Representations
• When Recurrent Models Don’t Need To Be Recurrent
• HoME - a Household Multimodal Environment
• Emergence of Grounded Compositional Language in Multi-Agent Populations
• A Semantic Loss Function for Deep Learning with Symbolic Knowledge
• Hierarchical Graph Representation Learning with Differentiable Pooling
• Imagination-Augmented Agents for Deep Reinforcement Learning
• Kronecker Recurrent Units
• Learning Independent Causal Mechanisms
• Memory-based Parameter Adaptation
• Born Again Neural Networks
• Net2Net-Accelerating Learning via Knowledge Transfer
• Learning to Count Objects in Natural Images for Visual Question Answering
• Neural Message Passing for Quantum Chemistry
• Unsupervised Learning by Predicting Noise
• The Lottery Ticket Hypothesis - Training Pruned Neural Networks
• Cyclical Learning Rates for Training Neural Networks
• Improving Information Extraction by Acquiring External Evidence with Reinforcement Learning
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• Learning an SAT Solver from Single-Bit Supervision
• Neural Relational Inference for Interacting Systems
• Stylistic Transfer in Natural Language Generation Systems Using Recurrent Neural Networks
• Get To The Point: Summarization with Pointer-Generator Networks
• StarSpace - Embed All The Things!
• Emotional Chatting Machine - Emotional Conversation Generation with Internal and External Memory
• Exploring Models and Data for Image Question Answering
• How transferable are features in deep neural networks
• Distilling the Knowledge in a Neural Network
• Revisiting Semi-Supervised Learning with Graph Embeddings
• Two-Stage Synthesis Networks for Transfer Learning in Machine Comprehension
• Higher-order organization of complex networks
• Network Motifs - Simple Building Blocks of Complex Networks
• Word Representations via Gaussian Embedding
• HARP - Hierarchical Representation Learning for Networks
• Swish - a Self-Gated Activation Function
• Reading Wikipedia to Answer Open-Domain Questions
• Task-Oriented Query Reformulation with Reinforcement Learning
• Refining Source Representations with Relation Networks for Neural Machine Translation
• Pointer Networks
• Learning to Compute Word Embeddings On the Fly
• R-NET - Machine Reading Comprehension with Self-matching Networks
• ReasoNet - Learning to Stop Reading in Machine Comprehension
• Principled Detection of Out-of-Distribution Examples in Neural Networks
• Ask Me Anything: Dynamic Memory Networks for Natural Language Processing
• One Model To Learn Them All
• Two/Too Simple Adaptations of Word2Vec for Syntax Problems
• A Decomposable Attention Model for Natural Language Inference
• A Fast and Accurate Dependency Parser using Neural Networks
• Neural Module Networks
• Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering
• Conditional Similarity Networks
• Simple Baseline for Visual Question Answering
• VQA: Visual Question Answering
• Learning to Generate Reviews and Discovering Sentiment
• Seeing the Arrow of Time
• End-to-end optimization of goal-driven and visually grounded dialogue systems
• GuessWhat?! Visual object discovery through multi-modal dialogue
• Semantic Parsing via Paraphrasing
• Traversing Knowledge Graphs in Vector Space
• PPDB: The Paraphrase Database
• NewsQA: A Machine Comprehension Dataset
• A Persona-Based Neural Conversation Model
• “Why Should I Trust You?” Explaining the Predictions of Any Classifier
• Conditional Generative Adversarial Nets
• Addressing the Rare Word Problem in Neural Machine Translation
• Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Models
• Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
• Improving Word Representations via Global Context and Multiple Word Prototypes
• Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
• Skip-Thought Vectors
• Deep Convolutional Generative Adversarial Nets
• Generative Adversarial Nets
• A Roadmap towards Machine Intelligence
• Smart Reply: Automated Response Suggestion for Email
• Convolutional Neural Network For Sentence Classification
• Conditional Image Generation with PixelCNN Decoders
• Pixel Recurrent Neural Networks
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
• Bag of Tricks for Efficient Text Classification
• GloVe: Global Vectors for Word Representation
• SimRank: A Measure of Structural-Context Similarity
• How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation
• Neural Generation of Regular Expressions from Natural Language with Minimal Domain Knowledge
• WikiReading : A Novel Large-scale Language Understanding Task over Wikipedia
• WikiQA: A challenge dataset for open-domain question answering
• Teaching Machines to Read and Comprehend
• Evaluating Prerequisite Qualities for Learning End-to-end Dialog Systems
• Recurrent Neural Network Regularization
• Deep Math: Deep Sequence Models for Premise Selection
• A Neural Conversational Model
• Key-Value Memory Networks for Directly Reading Documents
• Advances In Optimizing Recurrent Networks
• Query Regression Networks for Machine Comprehension
• Sequence to Sequence Learning with Neural Networks
• The Difficulty of Training Deep Architectures and the Effect of Unsupervised Pre-Training
• Question Answering with Subgraph Embeddings
• Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
• Visualizing Large-scale and High-dimensional Data
• Visualizing Data using t-SNE
• Curriculum Learning
• End-To-End Memory Networks
• Memory Networks
• Learning To Execute
• Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud
• Large Scale Distributed Deep Networks
• Efficient Estimation of Word Representations in Vector Space
• Regularization and variable selection via the elastic net
• Fractional Max-Pooling
• TAO: Facebook’s Distributed Data Store for the Social Graph
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• The Unified Logging Infrastructure for Data Analytics at Twitter
• A Few Useful Things to Know about Machine Learning
• Hive – A Petabyte Scale Data Warehouse Using Hadoop
• Kafka: a Distributed Messaging System for Log Processing
• Power-law distributions in Empirical data
• Pregel: A System for Large-Scale Graph Processing
• GraphX: Unifying Data-Parallel and Graph-Parallel Analytics
• Pig Latin: A Not-So-Foreign Language for Data Processing
• Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing
• MapReduce: Simplified Data Processing on Large Clusters
• BigTable: A Distributed Storage System for Structured Data
• Spark SQL: Relational Data Processing in Spark
• Spark: Cluster Computing with Working Sets
• Fast Data in the Era of Big Data: Twitter’s Real-Time Related Query Suggestion Architecture
• Scaling Memcache at Facebook
• Dynamo: Amazon’s Highly Available Key-value Store
• f4 : Facebook's Warm BLOB Storage System
• A Theoretician’s Guide to the Experimental Analysis of Algorithms
• Cuckoo Hashing
• Never Ending Learning

papers-I-read 快速上手指南

papers-I-read 并非一个需要安装运行的软件工具或代码库，而是一个由 Shagun Sodhani 维护的学术论文阅读清单与笔记仓库。该项目记录了作者“每周一篇论文”计划中的研究成果、摘要及个人笔记，涵盖深度学习、强化学习、系统设计及持续学习等前沿领域。

因此，本指南将指导您如何直接访问、浏览及利用该资源进行学习，无需执行传统的安装步骤。

环境准备

由于本项目本质为静态文档集合，您仅需具备以下基础环境即可开始阅读：

• 操作系统：任意支持现代浏览器的系统（Windows, macOS, Linux）。
• 前置依赖：
  • 现代网页浏览器（推荐 Chrome, Edge, Firefox）。
  • （可选）Git：如果您希望克隆仓库到本地进行离线阅读或贡献。
• 网络环境：
  • 项目托管于个人域名 (shagunsodhani.com) 及 GitHub。
  • 国内访问建议：若直接访问论文链接速度较慢，建议通过 GitHub 镜像站（如 ghproxy.com）加速克隆仓库，或使用学术资源聚合平台搜索对应论文标题获取原文。

安装/获取步骤

您有两种方式获取这些内容：在线直接阅读或克隆到本地。

方式一：在线直接阅读（推荐）

直接访问项目维护的个人网站，这是体验最佳的方式，页面已针对阅读优化。

1. 打开浏览器。
2. 访问项目主页：https://shagunsodhani.com/papers-I-read/
3. 在列表中点击感兴趣的论文标题（例如 Toolformer 或 GPipe），即可跳转至对应的详细笔记页面。

方式二：本地克隆（适合离线查阅）

如果您希望将笔记保存到本地，可以使用 Git 克隆仓库。

# 使用标准 Git 命令克隆
git clone https://github.com/shagunsodhani/papers-I-read.git

# 【国内加速方案】如果直接克隆速度慢，可使用 ghproxy 加速
git clone https://ghproxy.com/https://github.com/shagunsodhani/papers-I-read.git

克隆完成后，进入目录查看文件结构（通常为 Markdown 文件或重定向配置）：

cd papers-I-read
ls

基本使用

本项目的核心用法是按需检索与学习。以下是具体的使用流程示例：

1. 浏览论文列表

在项目主页或本地 README.md 中，您会看到按主题分类的论文列表。例如，若您关注大模型工具使用，可寻找：

• Toolformer - Language Models Can Teach Themselves to Use Tools

若您关注分布式系统，可寻找：

• Cassandra - a decentralized structured storage system
• CAP twelve years later - How the rules have changed

2. 阅读深度笔记

点击链接进入具体页面后，您将获得以下内容：

• 论文原文链接：直接跳转至 arXiv 或会议官网。
• 核心摘要：作者用通俗语言总结的论文核心思想。
• 关键洞察：关于模型架构、训练技巧（如 GradNorm, Mixup）或数学推导的个人注解。
• 代码实现线索：部分笔记会提及相关的开源实现或关键算法逻辑。

3. 搜索特定技术点

如果您在本地克隆了仓库，可以使用命令行快速搜索特定技术关键词的笔记。例如，查找所有关于“持续学习 (Continual Learning)"的笔记：

# 在本地仓库中搜索包含 "Continual Learning" 的文件
grep -r "Continual Learning" .

或者搜索特定的算法名称，如 "MAML"：

grep -r "MAML" .

这将帮助您快速定位到 How to train your MAML 或 Rapid Learning or Feature Reuse? 等相关笔记进行深入研读。