[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"similar-guillaume-be--rust-bert":3,"tool-guillaume-be--rust-bert":61},[4,18,26,36,44,52],{"id":5,"name":6,"github_repo":7,"description_zh":8,"stars":9,"difficulty_score":10,"last_commit_at":11,"category_tags":12,"status":17},4358,"openclaw","openclaw\u002Fopenclaw","OpenClaw 是一款专为个人打造的本地化 AI 助手，旨在让你在自己的设备上拥有完全可控的智能伙伴。它打破了传统 AI 助手局限于特定网页或应用的束缚，能够直接接入你日常使用的各类通讯渠道，包括微信、WhatsApp、Telegram、Discord、iMessage 等数十种平台。无论你在哪个聊天软件中发送消息，OpenClaw 都能即时响应，甚至支持在 macOS、iOS 和 Android 设备上进行语音交互，并提供实时的画布渲染功能供你操控。\n\n这款工具主要解决了用户对数据隐私、响应速度以及“始终在线”体验的需求。通过将 AI 部署在本地，用户无需依赖云端服务即可享受快速、私密的智能辅助，真正实现了“你的数据，你做主”。其独特的技术亮点在于强大的网关架构，将控制平面与核心助手分离，确保跨平台通信的流畅性与扩展性。\n\nOpenClaw 非常适合希望构建个性化工作流的技术爱好者、开发者，以及注重隐私保护且不愿被单一生态绑定的普通用户。只要具备基础的终端操作能力（支持 macOS、Linux 及 Windows WSL2），即可通过简单的命令行引导完成部署。如果你渴望拥有一个懂你",349277,3,"2026-04-06T06:32:30",[13,14,15,16],"Agent","开发框架","图像","数据工具","ready",{"id":19,"name":20,"github_repo":21,"description_zh":22,"stars":23,"difficulty_score":10,"last_commit_at":24,"category_tags":25,"status":17},3808,"stable-diffusion-webui","AUTOMATIC1111\u002Fstable-diffusion-webui","stable-diffusion-webui 是一个基于 Gradio 构建的网页版操作界面，旨在让用户能够轻松地在本地运行和使用强大的 Stable Diffusion 图像生成模型。它解决了原始模型依赖命令行、操作门槛高且功能分散的痛点，将复杂的 AI 绘图流程整合进一个直观易用的图形化平台。\n\n无论是希望快速上手的普通创作者、需要精细控制画面细节的设计师，还是想要深入探索模型潜力的开发者与研究人员，都能从中获益。其核心亮点在于极高的功能丰富度：不仅支持文生图、图生图、局部重绘（Inpainting）和外绘（Outpainting）等基础模式，还独创了注意力机制调整、提示词矩阵、负向提示词以及“高清修复”等高级功能。此外，它内置了 GFPGAN 和 CodeFormer 等人脸修复工具，支持多种神经网络放大算法，并允许用户通过插件系统无限扩展能力。即使是显存有限的设备，stable-diffusion-webui 也提供了相应的优化选项，让高质量的 AI 艺术创作变得触手可及。",162132,"2026-04-05T11:01:52",[14,15,13],{"id":27,"name":28,"github_repo":29,"description_zh":30,"stars":31,"difficulty_score":32,"last_commit_at":33,"category_tags":34,"status":17},1381,"everything-claude-code","affaan-m\u002Feverything-claude-code","everything-claude-code 是一套专为 AI 编程助手（如 Claude Code、Codex、Cursor 等）打造的高性能优化系统。它不仅仅是一组配置文件，而是一个经过长期实战打磨的完整框架，旨在解决 AI 代理在实际开发中面临的效率低下、记忆丢失、安全隐患及缺乏持续学习能力等核心痛点。\n\n通过引入技能模块化、直觉增强、记忆持久化机制以及内置的安全扫描功能，everything-claude-code 能显著提升 AI 在复杂任务中的表现，帮助开发者构建更稳定、更智能的生产级 AI 代理。其独特的“研究优先”开发理念和针对 Token 消耗的优化策略，使得模型响应更快、成本更低，同时有效防御潜在的攻击向量。\n\n这套工具特别适合软件开发者、AI 研究人员以及希望深度定制 AI 工作流的技术团队使用。无论您是在构建大型代码库，还是需要 AI 协助进行安全审计与自动化测试，everything-claude-code 都能提供强大的底层支持。作为一个曾荣获 Anthropic 黑客大奖的开源项目，它融合了多语言支持与丰富的实战钩子（hooks），让 AI 真正成长为懂上",141543,2,"2026-04-06T11:32:54",[14,13,35],"语言模型",{"id":37,"name":38,"github_repo":39,"description_zh":40,"stars":41,"difficulty_score":32,"last_commit_at":42,"category_tags":43,"status":17},2271,"ComfyUI","Comfy-Org\u002FComfyUI","ComfyUI 是一款功能强大且高度模块化的视觉 AI 引擎，专为设计和执行复杂的 Stable Diffusion 图像生成流程而打造。它摒弃了传统的代码编写模式，采用直观的节点式流程图界面，让用户通过连接不同的功能模块即可构建个性化的生成管线。\n\n这一设计巧妙解决了高级 AI 绘图工作流配置复杂、灵活性不足的痛点。用户无需具备编程背景，也能自由组合模型、调整参数并实时预览效果，轻松实现从基础文生图到多步骤高清修复等各类复杂任务。ComfyUI 拥有极佳的兼容性，不仅支持 Windows、macOS 和 Linux 全平台，还广泛适配 NVIDIA、AMD、Intel 及苹果 Silicon 等多种硬件架构，并率先支持 SDXL、Flux、SD3 等前沿模型。\n\n无论是希望深入探索算法潜力的研究人员和开发者，还是追求极致创作自由度的设计师与资深 AI 绘画爱好者，ComfyUI 都能提供强大的支持。其独特的模块化架构允许社区不断扩展新功能，使其成为当前最灵活、生态最丰富的开源扩散模型工具之一，帮助用户将创意高效转化为现实。",107888,"2026-04-06T11:32:50",[14,15,13],{"id":45,"name":46,"github_repo":47,"description_zh":48,"stars":49,"difficulty_score":10,"last_commit_at":50,"category_tags":51,"status":17},4487,"LLMs-from-scratch","rasbt\u002FLLMs-from-scratch","LLMs-from-scratch 是一个基于 PyTorch 的开源教育项目，旨在引导用户从零开始一步步构建一个类似 ChatGPT 的大型语言模型（LLM）。它不仅是同名技术著作的官方代码库，更提供了一套完整的实践方案，涵盖模型开发、预训练及微调的全过程。\n\n该项目主要解决了大模型领域“黑盒化”的学习痛点。许多开发者虽能调用现成模型，却难以深入理解其内部架构与训练机制。通过亲手编写每一行核心代码，用户能够透彻掌握 Transformer 架构、注意力机制等关键原理，从而真正理解大模型是如何“思考”的。此外，项目还包含了加载大型预训练权重进行微调的代码，帮助用户将理论知识延伸至实际应用。\n\nLLMs-from-scratch 特别适合希望深入底层原理的 AI 开发者、研究人员以及计算机专业的学生。对于不满足于仅使用 API，而是渴望探究模型构建细节的技术人员而言，这是极佳的学习资源。其独特的技术亮点在于“循序渐进”的教学设计：将复杂的系统工程拆解为清晰的步骤，配合详细的图表与示例，让构建一个虽小但功能完备的大模型变得触手可及。无论你是想夯实理论基础，还是为未来研发更大规模的模型做准备",90106,"2026-04-06T11:19:32",[35,15,13,14],{"id":53,"name":54,"github_repo":55,"description_zh":56,"stars":57,"difficulty_score":10,"last_commit_at":58,"category_tags":59,"status":17},4292,"Deep-Live-Cam","hacksider\u002FDeep-Live-Cam","Deep-Live-Cam 是一款专注于实时换脸与视频生成的开源工具，用户仅需一张静态照片，即可通过“一键操作”实现摄像头画面的即时变脸或制作深度伪造视频。它有效解决了传统换脸技术流程繁琐、对硬件配置要求极高以及难以实时预览的痛点，让高质量的数字内容创作变得触手可及。\n\n这款工具不仅适合开发者和技术研究人员探索算法边界，更因其极简的操作逻辑（仅需三步：选脸、选摄像头、启动），广泛适用于普通用户、内容创作者、设计师及直播主播。无论是为了动画角色定制、服装展示模特替换，还是制作趣味短视频和直播互动，Deep-Live-Cam 都能提供流畅的支持。\n\n其核心技术亮点在于强大的实时处理能力，支持口型遮罩（Mouth Mask）以保留使用者原始的嘴部动作，确保表情自然精准；同时具备“人脸映射”功能，可同时对画面中的多个主体应用不同面孔。此外，项目内置了严格的内容安全过滤机制，自动拦截涉及裸露、暴力等不当素材，并倡导用户在获得授权及明确标注的前提下合规使用，体现了技术发展与伦理责任的平衡。",88924,"2026-04-06T03:28:53",[14,15,13,60],"视频",{"id":62,"github_repo":63,"name":64,"description_en":65,"description_zh":66,"ai_summary_zh":67,"readme_en":68,"readme_zh":69,"quickstart_zh":70,"use_case_zh":71,"hero_image_url":72,"owner_login":73,"owner_name":74,"owner_avatar_url":75,"owner_bio":74,"owner_company":74,"owner_location":74,"owner_email":74,"owner_twitter":74,"owner_website":76,"owner_url":77,"languages":78,"stars":87,"forks":88,"last_commit_at":89,"license":90,"difficulty_score":91,"env_os":92,"env_gpu":93,"env_ram":94,"env_deps":95,"category_tags":104,"github_topics":106,"view_count":32,"oss_zip_url":74,"oss_zip_packed_at":74,"status":17,"created_at":124,"updated_at":125,"faqs":126,"releases":155},4388,"guillaume-be\u002Frust-bert","rust-bert","Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)","rust-bert 是一个专为 Rust 语言打造的自然语言处理（NLP）工具库，让开发者能在 Rust 环境中直接调用业界领先的 Transformer 模型。它相当于 Hugging Face Transformers 库的 Rust 原生移植版，支持 BERT、DistilBERT、GPT2、RoBERTa 等多种主流架构。\n\n在开发中，跨语言调用 Python 模型往往面临环境配置复杂、部署困难及性能损耗等痛点。rust-bert 通过纯 Rust 实现，彻底解决了这些依赖问题，提供了从文本预处理到模型推理的完整流水线。用户只需几行代码，即可轻松实现问答系统、情感分析、机器翻译、文本摘要、命名实体识别及多轮对话等复杂任务。\n\n该工具特别适合追求高性能、低延迟及内存安全的后端工程师、系统架构师及 AI 研究人员。如果你希望在生产环境中构建稳健的 NLP 应用，而不想引入沉重的 Python 运行时，rust-bert 是理想选择。其核心技术亮点在于基于 tch-rs 或 ONNX Runtime 的底层绑定，不仅完美支持 GPU 加速推理，还实现了多线程分词处理，充分发挥了 Ru","rust-bert 是一个专为 Rust 语言打造的自然语言处理（NLP）工具库，让开发者能在 Rust 环境中直接调用业界领先的 Transformer 模型。它相当于 Hugging Face Transformers 库的 Rust 原生移植版，支持 BERT、DistilBERT、GPT2、RoBERTa 等多种主流架构。\n\n在开发中，跨语言调用 Python 模型往往面临环境配置复杂、部署困难及性能损耗等痛点。rust-bert 通过纯 Rust 实现，彻底解决了这些依赖问题，提供了从文本预处理到模型推理的完整流水线。用户只需几行代码，即可轻松实现问答系统、情感分析、机器翻译、文本摘要、命名实体识别及多轮对话等复杂任务。\n\n该工具特别适合追求高性能、低延迟及内存安全的后端工程师、系统架构师及 AI 研究人员。如果你希望在生产环境中构建稳健的 NLP 应用，而不想引入沉重的 Python 运行时，rust-bert 是理想选择。其核心技术亮点在于基于 tch-rs 或 ONNX Runtime 的底层绑定，不仅完美支持 GPU 加速推理，还实现了多线程分词处理，充分发挥了 Rust 在并发计算上的优势，确保在高负载场景下依然保持卓越的性能表现。","# rust-bert\n\n[![Build Status](https:\u002F\u002Fgithub.com\u002Fguillaume-be\u002Frust-bert\u002Fworkflows\u002FBuild\u002Fbadge.svg?event=push)](https:\u002F\u002Fgithub.com\u002Fguillaume-be\u002Frust-bert\u002Factions)\n[![Latest version](https:\u002F\u002Fimg.shields.io\u002Fcrates\u002Fv\u002Frust_bert.svg)](https:\u002F\u002Fcrates.io\u002Fcrates\u002Frust_bert)\n[![Documentation](https:\u002F\u002Fdocs.rs\u002Frust-bert\u002Fbadge.svg)](https:\u002F\u002Fdocs.rs\u002Frust-bert)\n![License](https:\u002F\u002Fimg.shields.io\u002Fcrates\u002Fl\u002Frust_bert.svg)\n\nRust-native state-of-the-art Natural Language Processing models and pipelines.\nPort of Hugging Face's\n[Transformers library](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Ftransformers), using\n[tch-rs](https:\u002F\u002Fgithub.com\u002FLaurentMazare\u002Ftch-rs) or\n[onnxruntime bindings](https:\u002F\u002Fgithub.com\u002Fpykeio\u002Fort) and pre-processing from\n[rust-tokenizers](https:\u002F\u002Fgithub.com\u002Fguillaume-be\u002Frust-tokenizers). Supports\nmulti-threaded tokenization and GPU inference. This repository exposes the model\nbase architecture, task-specific heads (see below) and\n[ready-to-use pipelines](#ready-to-use-pipelines). [Benchmarks](#benchmarks) are\navailable at the end of this document.\n\nGet started with tasks including question answering, named entity recognition,\ntranslation, summarization, text generation, conversational agents and more in\njust a few lines of code:\n\n```rust\n    let qa_model = QuestionAnsweringModel::new(Default::default ()) ?;\n\nlet question = String::from(\"Where does Amy live ?\");\nlet context = String::from(\"Amy lives in Amsterdam\");\n\nlet answers = qa_model.predict( & [QaInput { question, context }], 1, 32);\n```\n\nOutput:\n\n```\n[Answer { score: 0.9976, start: 13, end: 21, answer: \"Amsterdam\" }]\n```\n\nThe tasks currently supported include:\n\n- Translation\n- Summarization\n- Multi-turn dialogue\n- Zero-shot classification\n- Sentiment Analysis\n- Named Entity Recognition\n- Part of Speech tagging\n- Question-Answering\n- Language Generation\n- Masked Language Model\n- Sentence Embeddings\n- Keywords extraction\n\n\u003Cdetails>\n\u003Csummary> \u003Cb>Expand to display the supported models\u002Ftasks matrix \u003C\u002Fb> \u003C\u002Fsummary>\n\n|              | **Sequence classification** | **Token classification** | **Question answering** | **Text Generation** | **Summarization** | **Translation** | **Masked LM** | **Sentence Embeddings** |\n|:------------:|:---------------------------:|:------------------------:|:----------------------:|:-------------------:|:-----------------:|:---------------:|:-------------:|:-----------------------:|\n|  DistilBERT  |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |            ✅            |\n|  MobileBERT  |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |                         |\n|   DeBERTa    |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |                         |\n| DeBERTa (v2) |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |                         |\n|     FNet     |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |                         |\n|     BERT     |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |            ✅            |\n|   RoBERTa    |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |            ✅            |\n|     GPT      |                             |                          |                        |          ✅          |                   |                 |               |                         |\n|     GPT2     |                             |                          |                        |          ✅          |                   |                 |               |                         |\n|   GPT-Neo    |                             |                          |                        |          ✅          |                   |                 |               |                         |\n|    GPT-J     |                             |                          |                        |          ✅          |                   |                 |               |                         |\n|     BART     |              ✅              |                          |                        |          ✅          |         ✅         |                 |               |                         |\n|    Marian    |                             |                          |                        |                     |                   |        ✅        |               |                         |\n|    MBart     |              ✅              |                          |                        |          ✅          |                   |                 |               |                         |\n|    M2M100    |                             |                          |                        |          ✅          |                   |                 |               |                         |\n|     NLLB     |                             |                          |                        |          ✅          |                   |                 |               |                         |\n|   Electra    |                             |            ✅             |                        |                     |                   |                 |       ✅       |                         |\n|    ALBERT    |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |            ✅            |\n|      T5      |                             |                          |                        |          ✅          |         ✅         |        ✅        |               |            ✅            |\n|    LongT5    |                             |                          |                        |          ✅          |         ✅         |                 |               |                         |\n|    XLNet     |              ✅              |            ✅             |           ✅            |          ✅          |                   |                 |       ✅       |                         |\n|   Reformer   |              ✅              |                          |           ✅            |          ✅          |                   |                 |       ✅       |                         |\n|  ProphetNet  |                             |                          |                        |          ✅          |         ✅         |                 |               |                         |\n|  Longformer  |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |                         |\n|   Pegasus    |                             |                          |                        |                     |         ✅         |                 |               |                         |\n\n\u003C\u002Fdetails>\n\n## Getting started\n\nThis library relies on the [tch](https:\u002F\u002Fgithub.com\u002FLaurentMazare\u002Ftch-rs) crate\nfor bindings to the C++ Libtorch API. The libtorch library is required can be\ndownloaded either automatically or manually. The following provides a reference\non how to set-up your environment to use these bindings, please refer to the\n[tch](https:\u002F\u002Fgithub.com\u002FLaurentMazare\u002Ftch-rs) for detailed information or\nsupport.\n\nFurthermore, this library relies on a cache folder for downloading pre-trained\nmodels. This cache location defaults to `~\u002F.cache\u002F.rustbert`, but can be changed\nby setting the `RUSTBERT_CACHE` environment variable. Note that the language\nmodels used by this library are in the order of the 100s of MBs to GBs.\n\n### Manual installation (recommended)\n\n1. Download `libtorch` from https:\u002F\u002Fpytorch.org\u002Fget-started\u002Flocally\u002F. This\n   package requires `v2.4`: if this version is no longer available on the \"get\n   started\" page, the file should be accessible by modifying the target link,\n   for example\n   `https:\u002F\u002Fdownload.pytorch.org\u002Flibtorch\u002Fcu124\u002Flibtorch-cxx11-abi-shared-with-deps-2.4.0%2Bcu124.zip`\n   for a Linux version with CUDA12. **NOTE:** When using `rust-bert` as\n   dependency from [crates.io](https:\u002F\u002Fcrates.io), please check the required\n   `LIBTORCH` on the published package\n   [readme](https:\u002F\u002Fcrates.io\u002Fcrates\u002Frust-bert) as it may differ from the\n   version documented here (applying to the current repository version).\n2. Extract the library to a location of your choice\n3. Set the following environment variables\n\n##### Linux:\n\n```bash\nexport LIBTORCH=\u002Fpath\u002Fto\u002Flibtorch\nexport LD_LIBRARY_PATH=${LIBTORCH}\u002Flib:$LD_LIBRARY_PATH\n```\n\n##### Windows\n\n```powershell\n$Env:LIBTORCH = \"X:\\path\\to\\libtorch\"\n$Env:Path += \";X:\\path\\to\\libtorch\\lib\"\n```\n\n#### macOS + Homebrew\n\n```bash\nbrew install pytorch jq\nexport LIBTORCH=$(brew --cellar pytorch)\u002F$(brew info --json pytorch | jq -r '.[0].installed[0].version')\nexport LD_LIBRARY_PATH=${LIBTORCH}\u002Flib:$LD_LIBRARY_PATH\n```\n\n### Automatic installation\n\nAlternatively, you can let the `build` script automatically download the\n`libtorch` library for you. The `download-libtorch` feature flag needs to be\nenabled. The CPU version of libtorch will be downloaded by default. To download\na CUDA version, please set the environment variable `TORCH_CUDA_VERSION` to\n`cu124`. Note that the libtorch library is large (order of several GBs for the\nCUDA-enabled version) and the first build may therefore take several minutes to\ncomplete.\n\n### Verifying installation\n\nVerify your installation (and linking with libtorch) by adding the `rust-bert`\ndependency to your `Cargo.toml` or by cloning the rust-bert source and running\nan example:\n\n```bash\ngit clone git@github.com:guillaume-be\u002Frust-bert.git\ncd rust-bert\ncargo run --example sentence_embeddings\n```\n\n## ONNX Support (Optional)\n\nONNX support can be enabled via the optional `onnx` feature. This crate then\nleverages the [ort](https:\u002F\u002Fgithub.com\u002Fpykeio\u002Fort) crate with bindings to the\nonnxruntime C++ library. We refer the user to this page project for further\ninstallation instructions\u002Fsupport.\n\n1. Enable the optional `onnx` feature. The `rust-bert` crate does not include\n   any optional dependencies for `ort`, the end user should select the set of\n   features that would be adequate for pulling the required `onnxruntime` C++\n   library.\n2. The current recommended installation is to use dynamic linking by pointing to\n   an existing library location. Use the `load-dynamic` cargo feature for `ort`.\n3. set the `ORT_DYLIB_PATH` to point to the location of downloaded onnxruntime\n   library (`onnxruntime.dll`\u002F`libonnxruntime.so`\u002F`libonnxruntime.dylib`\n   depending on the operating system). These can be downloaded from the\n   [release page](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fonnxruntime\u002Freleases) of the\n   onnxruntime project\n\nMost architectures (including encoders, decoders and encoder-decoders) are\nsupported. the library aims at keeping compatibility with models exported using\nthe [Optimum](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Foptimum) library. A detailed guide\non how to export a Transformer model to ONNX using Optimum is available at\nhttps:\u002F\u002Fhuggingface.co\u002Fdocs\u002Foptimum\u002Fmain\u002Fen\u002Fexporters\u002Fonnx\u002Fusage_guides\u002Fexport_a_model\nThe resources used to create ONNX models are similar to those based on Pytorch,\nreplacing the pytorch by the ONNX model. Since ONNX models are less flexible\nthan their Pytorch counterparts in the handling of optional arguments, exporting\na decoder or encoder-decoder model to ONNX will usually result in multiple\nfiles. These files are expected (but not all are necessary) for use in this\nlibrary as per the table below:\n\n| Architecture                | Encoder file | Decoder without past file | Decoder with past file |\n|-----------------------------|--------------|---------------------------|------------------------|\n| Encoder (e.g. BERT)         | required     | not used                  | not used               |\n| Decoder (e.g. GPT2)         | not used     | required                  | optional               |\n| Encoder-decoder (e.g. BART) | required     | required                  | optional               |\n\nNote that the computational efficiency will drop when the `decoder with past`\nfile is optional but not provided since the model will not used cached past keys\nand values for the attention mechanism, leading to a high number of redundant\ncomputations. The Optimum library offers export options to ensure such a\n`decoder with past` model file is created. The base encoder and decoder model\narchitecture are available (and exposed for convenience) in the `encoder` and\n`decoder` modules, respectively.\n\nGeneration models (pure decoder or encoder\u002Fdecoder architectures) are available\nin the `models` module. ost pipelines are available for ONNX model checkpoints,\nincluding sequence classification, zero-shot classification, token\nclassification (including named entity recognition and part-of-speech tagging),\nquestion answering, text generation, summarization and translation. These models\nuse the same configuration and tokenizer files as their Pytorch counterparts\nwhen used in a pipeline. Examples leveraging ONNX models are given in the\n`.\u002Fexamples` directory\n\n## Ready-to-use pipelines\n\nBased on Hugging Face's pipelines, ready to use end-to-end NLP pipelines are\navailable as part of this crate. The following capabilities are currently\navailable:\n\n**Disclaimer** The contributors of this repository are not responsible for any\ngeneration from the 3rd party utilization of the pretrained systems proposed\nherein.\n\n\u003Cdetails>\n\u003Csummary> \u003Cb>1. Question Answering\u003C\u002Fb> \u003C\u002Fsummary>\n\nExtractive question answering from a given question and context. DistilBERT\nmodel fine-tuned on SQuAD (Stanford Question Answering Dataset)\n\n```rust\n    let qa_model = QuestionAnsweringModel::new(Default::default ()) ?;\n\nlet question = String::from(\"Where does Amy live ?\");\nlet context = String::from(\"Amy lives in Amsterdam\");\n\nlet answers = qa_model.predict( & [QaInput { question, context }], 1, 32);\n```\n\nOutput:\n\n```\n[Answer { score: 0.9976, start: 13, end: 21, answer: \"Amsterdam\" }]\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>2. Translation \u003C\u002Fb> \u003C\u002Fsummary>\n\nTranslation pipeline supporting a broad range of source and target languages.\nLeverages two main architectures for translation tasks:\n\n- Marian-based models, for specific source\u002Ftarget combinations\n- M2M100 models allowing for direct translation between 100 languages (at a\n  higher computational cost and lower performance for some selected languages)\n\nMarian-based pretrained models for the following language pairs are readily\navailable in the library - but the user can import any Pytorch-based model for\npredictions\n\n- English \u003C-> French\n- English \u003C-> Spanish\n- English \u003C-> Portuguese\n- English \u003C-> Italian\n- English \u003C-> Catalan\n- English \u003C-> German\n- English \u003C-> Russian\n- English \u003C-> Chinese\n- English \u003C-> Dutch\n- English \u003C-> Swedish\n- English \u003C-> Arabic\n- English \u003C-> Hebrew\n- English \u003C-> Hindi\n- French \u003C-> German\n\nFor languages not supported by the proposed pretrained Marian models, the user\ncan leverage a M2M100 model supporting direct translation between 100 languages\n(without intermediate English translation) The full list of supported languages\nis available in the\n[crate documentation](https:\u002F\u002Fdocs.rs\u002Frust-bert\u002Flatest\u002Frust_bert\u002Fpipelines\u002Ftranslation\u002Fenum.Language.html)\n\n```rust\nuse rust_bert::pipelines::translation::{Language, TranslationModelBuilder};\nfn main() -> anyhow::Result\u003C()> {\n    let model = TranslationModelBuilder::new()\n        .with_source_languages(vec![Language::English])\n        .with_target_languages(vec![Language::Spanish, Language::French, Language::Italian])\n        .create_model()?;\n    let input_text = \"This is a sentence to be translated\";\n    let output = model.translate(&[input_text], None, Language::French)?;\n    for sentence in output {\n        println!(\"{}\", sentence);\n    }\n    Ok(())\n}\n```\n\nOutput:\n\n```\nIl s'agit d'une phrase à traduire\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>3. Summarization \u003C\u002Fb> \u003C\u002Fsummary>\n\nAbstractive summarization using a pretrained BART model.\n\n```rust\n    let summarization_model = SummarizationModel::new(Default::default ()) ?;\n\nlet input = [\"In findings published Tuesday in Cornell University's arXiv by a team of scientists \\\nfrom the University of Montreal and a separate report published Wednesday in Nature Astronomy by a team \\\nfrom University College London (UCL), the presence of water vapour was confirmed in the atmosphere of K2-18b, \\\na planet circling a star in the constellation Leo. This is the first such discovery in a planet in its star's \\\nhabitable zone — not too hot and not too cold for liquid water to exist. The Montreal team, led by Björn Benneke, \\\nused data from the NASA's Hubble telescope to assess changes in the light coming from K2-18b's star as the planet \\\npassed between it and Earth. They found that certain wavelengths of light, which are usually absorbed by water, \\\nweakened when the planet was in the way, indicating not only does K2-18b have an atmosphere, but the atmosphere \\\ncontains water in vapour form. The team from UCL then analyzed the Montreal team's data using their own software \\\nand confirmed their conclusion. This was not the first time scientists have found signs of water on an exoplanet, \\\nbut previous discoveries were made on planets with high temperatures or other pronounced differences from Earth. \\\n\\\"This is the first potentially habitable planet where the temperature is right and where we now know there is water,\\\" \\\nsaid UCL astronomer Angelos Tsiaras. \\\"It's the best candidate for habitability right now.\\\" \\\"It's a good sign\\\", \\\nsaid Ryan Cloutier of the Harvard–Smithsonian Center for Astrophysics, who was not one of either study's authors. \\\n\\\"Overall,\\\" he continued, \\\"the presence of water in its atmosphere certainly improves the prospect of K2-18b being \\\na potentially habitable planet, but further observations will be required to say for sure. \\\"\nK2-18b was first identified in 2015 by the Kepler space telescope. It is about 110 light-years from Earth and larger \\\nbut less dense. Its star, a red dwarf, is cooler than the Sun, but the planet's orbit is much closer, such that a year \\\non K2-18b lasts 33 Earth days. According to The Guardian, astronomers were optimistic that NASA's James Webb space \\\ntelescope — scheduled for launch in 2021 — and the European Space Agency's 2028 ARIEL program, could reveal more \\\nabout exoplanets like K2-18b.\"];\n\nlet output = summarization_model.summarize( & input);\n```\n\n(example from:\n[WikiNews](https:\u002F\u002Fen.wikinews.org\u002Fwiki\u002FAstronomers_find_water_vapour_in_atmosphere_of_exoplanet_K2-18b))\n\nOutput:\n\n```\n\"Scientists have found water vapour on K2-18b, a planet 110 light-years from Earth. \nThis is the first such discovery in a planet in its star's habitable zone. \nThe planet is not too hot and not too cold for liquid water to exist.\"\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>4. Dialogue Model \u003C\u002Fb> \u003C\u002Fsummary>\n\nConversation model based on Microsoft's\n[DialoGPT](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002FDialoGPT). This pipeline allows the\ngeneration of single or multi-turn conversations between a human and a model.\nThe DialoGPT's page states that\n\n> The human evaluation results indicate that the response generated from\n> DialoGPT is comparable to human response quality under a single-turn\n> conversation Turing test.\n> ([DialoGPT repository](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002FDialoGPT))\n\nThe model uses a `ConversationManager` to keep track of active conversations and\ngenerate responses to them.\n\n```rust\nuse rust_bert::pipelines::conversation::{ConversationModel, ConversationManager};\n\nlet conversation_model = ConversationModel::new(Default::default ());\nlet mut conversation_manager = ConversationManager::new();\n\nlet conversation_id = conversation_manager.create(\"Going to the movies tonight - any suggestions?\");\nlet output = conversation_model.generate_responses( & mut conversation_manager);\n```\n\nExample output:\n\n```\n\"The Big Lebowski.\"\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>5. Natural Language Generation \u003C\u002Fb> \u003C\u002Fsummary>\n\nGenerate language based on a prompt. GPT2 and GPT available as base models.\nInclude techniques such as beam search, top-k and nucleus sampling, temperature\nsetting and repetition penalty. Supports batch generation of sentences from\nseveral prompts. Sequences will be left-padded with the model's padding token if\npresent, the unknown token otherwise. This may impact the results, it is\nrecommended to submit prompts of similar length for best results\n\n```rust\n    let model = GPT2Generator::new(Default::default ()) ?;\n\nlet input_context_1 = \"The dog\";\nlet input_context_2 = \"The cat was\";\n\nlet generate_options = GenerateOptions {\nmax_length: 30,\n..Default::default ()\n};\n\nlet output = model.generate(Some( & [input_context_1, input_context_2]), generate_options);\n```\n\nExample output:\n\n```\n[\n    \"The dog's owners, however, did not want to be named. According to the lawsuit, the animal's owner, a 29-year\"\n    \"The dog has always been part of the family. \\\"He was always going to be my dog and he was always looking out for me\"\n    \"The dog has been able to stay in the home for more than three months now. \\\"It's a very good dog. She's\"\n    \"The cat was discovered earlier this month in the home of a relative of the deceased. The cat\\'s owner, who wished to remain anonymous,\"\n    \"The cat was pulled from the street by two-year-old Jazmine.\\\"I didn't know what to do,\\\" she said\"\n    \"The cat was attacked by two stray dogs and was taken to a hospital. Two other cats were also injured in the attack and are being treated.\"\n]\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>6. Zero-shot classification \u003C\u002Fb> \u003C\u002Fsummary>\n\nPerforms zero-shot classification on input sentences with provided labels using\na model fine-tuned for Natural Language Inference.\n\n```rust\n    let sequence_classification_model = ZeroShotClassificationModel::new(Default::default ()) ?;\n\nlet input_sentence = \"Who are you voting for in 2020?\";\nlet input_sequence_2 = \"The prime minister has announced a stimulus package which was widely criticized by the opposition.\";\nlet candidate_labels = & [\"politics\", \"public health\", \"economics\", \"sports\"];\n\nlet output = sequence_classification_model.predict_multilabel(\n& [input_sentence, input_sequence_2],\ncandidate_labels,\nNone,\n128,\n);\n```\n\nOutput:\n\n```\n[\n  [ Label { \"politics\", score: 0.972 }, Label { \"public health\", score: 0.032 }, Label {\"economics\", score: 0.006 }, Label {\"sports\", score: 0.004 } ],\n  [ Label { \"politics\", score: 0.975 }, Label { \"public health\", score: 0.0818 }, Label {\"economics\", score: 0.852 }, Label {\"sports\", score: 0.001 } ],\n]\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>7. Sentiment analysis \u003C\u002Fb> \u003C\u002Fsummary>\n\nPredicts the binary sentiment for a sentence. DistilBERT model fine-tuned on\nSST-2.\n\n```rust\n    let sentiment_classifier = SentimentModel::new(Default::default ()) ?;\n\nlet input = [\n\"Probably my all-time favorite movie, a story of selflessness, sacrifice and dedication to a noble cause, but it's not preachy or boring.\",\n\"This film tried to be too many things all at once: stinging political satire, Hollywood blockbuster, sappy romantic comedy, family values promo...\",\n\"If you like original gut wrenching laughter you will like this movie. If you are young or old then you will love this movie, hell even my mom liked it.\",\n];\n\nlet output = sentiment_classifier.predict( & input);\n```\n\n(Example courtesy of [IMDb](http:\u002F\u002Fwww.imdb.com))\n\nOutput:\n\n```\n[\n    Sentiment { polarity: Positive, score: 0.9981985493795946 },\n    Sentiment { polarity: Negative, score: 0.9927982091903687 },\n    Sentiment { polarity: Positive, score: 0.9997248985164333 }\n]\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>8. Named Entity Recognition \u003C\u002Fb> \u003C\u002Fsummary>\n\nExtracts entities (Person, Location, Organization, Miscellaneous) from text.\nBERT cased large model fine-tuned on CoNNL03, contributed by the\n[MDZ Digital Library team at the Bavarian State Library](https:\u002F\u002Fgithub.com\u002Fdbmdz).\nModels are currently available for English, German, Spanish and Dutch.\n\n```rust\n    let ner_model = NERModel::new( default::default ()) ?;\n\nlet input = [\n\"My name is Amy. I live in Paris.\",\n\"Paris is a city in France.\"\n];\n\nlet output = ner_model.predict( & input);\n```\n\nOutput:\n\n```\n[\n  [\n    Entity { word: \"Amy\", score: 0.9986, label: \"I-PER\" }\n    Entity { word: \"Paris\", score: 0.9985, label: \"I-LOC\" }\n  ],\n  [\n    Entity { word: \"Paris\", score: 0.9988, label: \"I-LOC\" }\n    Entity { word: \"France\", score: 0.9993, label: \"I-LOC\" }\n  ]\n]\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>9. Keywords\u002Fkeyphrases extraction\u003C\u002Fb> \u003C\u002Fsummary>\n\nExtract keywords and keyphrases extractions from input documents\n\n```rust\nfn main() -> anyhow::Result\u003C()> {\n    let keyword_extraction_model = KeywordExtractionModel::new(Default::default())?;\n\n    let input = \"Rust is a multi-paradigm, general-purpose programming language. \\\n       Rust emphasizes performance, type safety, and concurrency. Rust enforces memory safety—that is, \\\n       that all references point to valid memory—without requiring the use of a garbage collector or \\\n       reference counting present in other memory-safe languages. To simultaneously enforce \\\n       memory safety and prevent concurrent data races, Rust's borrow checker tracks the object lifetime \\\n       and variable scope of all references in a program during compilation. Rust is popular for \\\n       systems programming but also offers high-level features including functional programming constructs.\";\n\n    let output = keyword_extraction_model.predict(&[input])?;\n}\n```\n\nOutput:\n\n```\n\"rust\" - 0.50910604\n\"programming\" - 0.35731024\n\"concurrency\" - 0.33825397\n\"concurrent\" - 0.31229728\n\"program\" - 0.29115444\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>10. Part of Speech tagging \u003C\u002Fb> \u003C\u002Fsummary>\n\nExtracts Part of Speech tags (Noun, Verb, Adjective...) from text.\n\n```rust\n    let pos_model = POSModel::new( default::default ()) ?;\n\nlet input = [\"My name is Bob\"];\n\nlet output = pos_model.predict( & input);\n```\n\nOutput:\n\n```\n[\n    Entity { word: \"My\", score: 0.1560, label: \"PRP\" }\n    Entity { word: \"name\", score: 0.6565, label: \"NN\" }\n    Entity { word: \"is\", score: 0.3697, label: \"VBZ\" }\n    Entity { word: \"Bob\", score: 0.7460, label: \"NNP\" }\n]\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>11. Sentence embeddings \u003C\u002Fb> \u003C\u002Fsummary>\n\nGenerate sentence embeddings (vector representation). These can be used for\napplications including dense information retrieval.\n\n```rust\n    let model = SentenceEmbeddingsBuilder::remote(\nSentenceEmbeddingsModelType::AllMiniLmL12V2\n).create_model() ?;\n\nlet sentences = [\n\"this is an example sentence\",\n\"each sentence is converted\"\n];\n\nlet output = model.encode( & sentences) ?;\n```\n\nOutput:\n\n```\n[\n    [-0.000202666, 0.08148022, 0.03136178, 0.002920636 ...],\n    [0.064757116, 0.048519745, -0.01786038, -0.0479775 ...]\n]\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>12. Masked Language Model \u003C\u002Fb> \u003C\u002Fsummary>\n\nPredict masked words in input sentences.\n\n```rust\n    let model = MaskedLanguageModel::new(Default::default ()) ?;\n\nlet sentences = [\n\"Hello I am a \u003Cmask> student\",\n\"Paris is the \u003Cmask> of France. It is \u003Cmask> in Europe.\",\n];\n\nlet output = model.predict( & sentences);\n```\n\nOutput:\n\n```\n[\n    [MaskedToken { text: \"college\", id: 2267, score: 8.091}],\n    [\n        MaskedToken { text: \"capital\", id: 3007, score: 16.7249}, \n        MaskedToken { text: \"located\", id: 2284, score: 9.0452}\n    ]\n]\n```\n\n\u003C\u002Fdetails>\n\n## Benchmarks\n\nFor simple pipelines (sequence classification, tokens classification, question\nanswering) the performance between Python and Rust is expected to be comparable.\nThis is because the most expensive part of these pipeline is the language model\nitself, sharing a common implementation in the Torch backend. The\n[End-to-end NLP Pipelines in Rust](https:\u002F\u002Fwww.aclweb.org\u002Fanthology\u002F2020.nlposs-1.4\u002F)\nprovides a benchmarks section covering all pipelines.\n\nFor text generation tasks (summarization, translation, conversation, free text\ngeneration), significant benefits can be expected (up to 2 to 4 times faster\nprocessing depending on the input and application). The article\n[Accelerating text generation with Rust](https:\u002F\u002Fguillaume-be.github.io\u002F2020-11-21\u002Fgeneration_benchmarks)\nfocuses on these text generation applications and provides more details on the\nperformance comparison to Python.\n\n## Loading pretrained and custom model weights\n\nThe base model and task-specific heads are also available for users looking to\nexpose their own transformer based models. Examples on how to prepare the date\nusing a native tokenizers Rust library are available in `.\u002Fexamples` for BERT,\nDistilBERT, RoBERTa, GPT, GPT2 and BART. Note that when importing models from\nPytorch, the convention for parameters naming needs to be aligned with the Rust\nschema. Loading of the pre-trained weights will fail if any of the model\nparameters weights cannot be found in the weight files. If this quality check is\nto be skipped, an alternative method `load_partial` can be invoked from the\nvariables store.\n\nPretrained models are available on Hugging face's\n[model hub](https:\u002F\u002Fhuggingface.co\u002Fmodels?filter=rust) and can be loaded using\n`RemoteResources` defined in this library.\n\nA conversion utility script is included in `.\u002Futils` to convert Pytorch weights\nto a set of weights compatible with this library. This script requires Python\nand `torch` to be set-up, and can be used as follows:\n`python .\u002Futils\u002Fconvert_model.py path\u002Fto\u002Fpytorch_model.bin` where\n`path\u002Fto\u002Fpytorch_model.bin` is the location of the original Pytorch weights.\n\n```bash\npython3 -m venv .venv\nsource .venv\u002Fbin\u002Factivate\n\npip install -r requirements.txt\n\npython utils\u002Fconvert_model.py path\u002Fto\u002Fpytorch_model.bin\n```\n\n## Citation\n\nIf you use `rust-bert` for your work, please cite\n[End-to-end NLP Pipelines in Rust](https:\u002F\u002Fwww.aclweb.org\u002Fanthology\u002F2020.nlposs-1.4\u002F):\n\n```bibtex\n@inproceedings{becquin-2020-end,\n    title = \"End-to-end {NLP} Pipelines in Rust\",\n    author = \"Becquin, Guillaume\",\n    booktitle = \"Proceedings of Second Workshop for NLP Open Source Software (NLP-OSS)\",\n    year = \"2020\",\n    publisher = \"Association for Computational Linguistics\",\n    url = \"https:\u002F\u002Fwww.aclweb.org\u002Fanthology\u002F2020.nlposs-1.4\",\n    pages = \"20--25\",\n}\n```\n\n## Acknowledgements\n\nThank you to [Hugging Face](https:\u002F\u002Fhuggingface.co) for hosting a set of weights\ncompatible with this Rust library. The list of ready-to-use pretrained models is\nlisted at\n[https:\u002F\u002Fhuggingface.co\u002Fmodels?filter=rust](https:\u002F\u002Fhuggingface.co\u002Fmodels?filter=rust).\n","# rust-bert\n\n[![构建状态](https:\u002F\u002Fgithub.com\u002Fguillaume-be\u002Frust-bert\u002Fworkflows\u002FBuild\u002Fbadge.svg?event=push)](https:\u002F\u002Fgithub.com\u002Fguillaume-be\u002Frust-bert\u002Factions)\n[![最新版本](https:\u002F\u002Fimg.shields.io\u002Fcrates\u002Fv\u002Frust_bert.svg)](https:\u002F\u002Fcrates.io\u002Fcrates\u002Frust_bert)\n[![文档](https:\u002F\u002Fdocs.rs\u002Frust-bert\u002Fbadge.svg)](https:\u002F\u002Fdocs.rs\u002Frust-bert)\n![许可证](https:\u002F\u002Fimg.shields.io\u002Fcrates\u002Fl\u002Frust_bert.svg)\n\nRust原生的最先进自然语言处理模型和流水线。移植自Hugging Face的\n[Transformers库](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Ftransformers)，使用\n[tch-rs](https:\u002F\u002Fgithub.com\u002FLaurentMazare\u002Ftch-rs)或\n[onnxruntime绑定](https:\u002F\u002Fgithub.com\u002Fpykeio\u002Fort)，并结合来自\n[rust-tokenizers](https:\u002F\u002Fgithub.com\u002Fguillaume-be\u002Frust-tokenizers)的预处理功能。支持多线程分词和GPU推理。本仓库公开了模型基础架构、特定任务的头部（见下文）以及\n[开箱即用的流水线](#ready-to-use-pipelines)。本文档末尾提供了\n[基准测试](#benchmarks)。\n\n只需几行代码，即可开始进行问答、命名实体识别、翻译、摘要生成、文本生成、对话代理等任务：\n\n```rust\n    let qa_model = QuestionAnsweringModel::new(Default::default ()) ?;\n\nlet question = String::from(\"Where does Amy live ?\");\nlet context = String::from(\"Amy lives in Amsterdam\");\n\nlet answers = qa_model.predict( & [QaInput { question, context }], 1, 32);\n```\n\n输出：\n\n```\n[Answer { score: 0.9976, start: 13, end: 21, answer: \"Amsterdam\" }]\n```\n\n目前支持的任务包括：\n\n- 翻译\n- 摘要生成\n- 多轮对话\n- 零样本分类\n- 情感分析\n- 命名实体识别\n- 词性标注\n- 问答\n- 语言生成\n- 掩码语言模型\n- 句子嵌入\n- 关键词提取\n\n\u003Cdetails>\n\u003Csummary> \u003Cb>展开以查看支持的模型\u002F任务矩阵 \u003C\u002Fb> \u003C\u002Fsummary>\n\n|              | **序列分类** | **标记分类** | **问答** | **文本生成** | **摘要生成** | **翻译** | **掩码语言模型** | **句子嵌入** |\n|:------------:|:---------------------------:|:------------------------:|:----------------------:|:-------------------:|:-----------------:|:---------------:|:-------------:|:-----------------------:|\n|  DistilBERT  |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |            ✅            |\n|  MobileBERT  |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |                         |\n|   DeBERTa    |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |                         |\n| DeBERTa (v2) |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |                         |\n|     FNet     |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |                         |\n|     BERT     |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |            ✅            |\n|   RoBERTa    |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |            ✅            |\n|     GPT      |                             |                          |                        |          ✅          |                   |                 |               |                         |\n|     GPT2     |                             |                          |                        |          ✅          |                   |                 |               |                         |\n|   GPT-Neo    |                             |                          |                        |          ✅          |                   |                 |               |                         |\n|    GPT-J     |                             |                          |                        |          ✅          |                   |                 |               |                         |\n|     BART     |              ✅              |                          |                        |          ✅          |         ✅         |                 |               |                         |\n|    Marian    |                             |                          |                        |                     |                   |        ✅        |               |                         |\n|    MBart     |              ✅              |                          |                        |          ✅          |                   |                 |               |                         |\n|    M2M100    |                             |                          |                        |          ✅          |                   |                 |               |                         |\n|     NLLB     |                             |                          |                        |          ✅          |                   |                 |               |                         |\n|   Electra    |                             |            ✅             |                        |                     |                   |                 |       ✅       |                         |\n|    ALBERT    |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |            ✅            |\n|      T5      |                             |                          |                        |          ✅          |         ✅         |        ✅        |               |            ✅            |\n|    LongT5    |                             |                          |                        |          ✅          |         ✅         |                 |               |                         |\n|    XLNet     |              ✅              |            ✅             |           ✅            |          ✅          |                   |                 |       ✅       |                         |\n|   Reformer   |              ✅              |                          |           ✅            |          ✅          |                   |                 |       ✅       |                         |\n|  ProphetNet  |                             |                          |                        |          ✅          |         ✅         |                 |               |                         |\n|  Longformer  |              ✅              |            ✅             |           ✅            |                     |                   |                 |       ✅       |                         |\n|   Pegasus    |                             |                          |                        |                     |         ✅         |                 |               |                         |\n\n\u003C\u002Fdetails>\n\n\n\n## 入门指南\n\n本库依赖于 [tch](https:\u002F\u002Fgithub.com\u002FLaurentMazare\u002Ftch-rs) crate 来实现与 C++ Libtorch API 的绑定。Libtorch 库是必需的，可以通过自动或手动方式下载。以下提供了关于如何设置环境以使用这些绑定的参考，请参阅 [tch](https:\u002F\u002Fgithub.com\u002FLaurentMazare\u002Ftch-rs) 获取详细信息或支持。\n\n此外，本库会使用一个缓存文件夹来下载预训练模型。该缓存位置默认为 `~\u002F.cache\u002F.rustbert`，但可以通过设置 `RUSTBERT_CACHE` 环境变量来更改。请注意，本库所使用的语言模型大小通常在数百兆到数GB之间。\n\n### 手动安装（推荐）\n\n1. 从 https:\u002F\u002Fpytorch.org\u002Fget-started\u002Flocally\u002F 下载 `libtorch`。此包需要 `v2.4` 版本；如果“开始使用”页面上不再提供该版本，可以通过修改目标链接来访问文件，例如对于带有 CUDA12 的 Linux 版本：`https:\u002F\u002Fdownload.pytorch.org\u002Flibtorch\u002Fcu124\u002Flibtorch-cxx11-abi-shared-with-deps-2.4.0%2Bcu124.zip`。**注意：** 当您从 [crates.io](https:\u002F\u002Fcrates.io) 将 `rust-bert` 作为依赖项使用时，请务必查看已发布的包的 [readme](https:\u002F\u002Fcrates.io\u002Fcrates\u002Frust-bert)，因为所需的 `LIBTORCH` 版本可能与此处文档中记录的版本不同（此处文档适用于当前仓库版本）。\n2. 将库解压到您选择的位置。\n3. 设置以下环境变量：\n\n##### Linux:\n\n```bash\nexport LIBTORCH=\u002Fpath\u002Fto\u002Flibtorch\nexport LD_LIBRARY_PATH=${LIBTORCH}\u002Flib:$LD_LIBRARY_PATH\n```\n\n##### Windows\n\n```powershell\n$Env:LIBTORCH = \"X:\\path\\to\\libtorch\"\n$Env:Path += \";X:\\path\\to\\libtorch\\lib\"\n```\n\n#### macOS + Homebrew\n\n```bash\nbrew install pytorch jq\nexport LIBTORCH=$(brew --cellar pytorch)\u002F$(brew info --json pytorch | jq -r '.[0].installed[0].version')\nexport LD_LIBRARY_PATH=${LIBTORCH}\u002Flib:$LD_LIBRARY_PATH\n```\n\n### 自动安装\n\n或者，您可以让 `build` 脚本自动为您下载 `libtorch` 库。为此，需要启用 `download-libtorch` 功能标志。默认情况下会下载 CPU 版本的 libtorch。要下载 CUDA 版本，请将环境变量 `TORCH_CUDA_VERSION` 设置为 `cu124`。请注意，libtorch 库体积较大（启用 CUDA 的版本约为几 GB），因此首次构建可能需要几分钟才能完成。\n\n### 验证安装\n\n通过在您的 `Cargo.toml` 中添加 `rust-bert` 依赖项，或克隆 `rust-bert` 源代码并运行示例来验证安装是否成功以及是否正确链接了 `libtorch`：\n\n```bash\ngit clone git@github.com:guillaume-be\u002Frust-bert.git\ncd rust-bert\ncargo run --example sentence_embeddings\n```\n\n## ONNX 支持（可选）\n\n可通过可选的 `onnx` 功能启用 ONNX 支持。此时，该 crate 会利用 [ort](https:\u002F\u002Fgithub.com\u002Fpykeio\u002Fort) crate，它提供了对 onnxruntime C++ 库的绑定。有关进一步的安装说明和支持，请参阅该项目的主页。\n\n1. 启用可选的 `onnx` 功能。`rust-bert` crate 并不包含 `ort` 的任何可选依赖项，最终用户应根据需要选择合适的功能集，以拉取所需的 `onnxruntime` C++ 库。\n2. 目前推荐的安装方式是使用动态链接，并指向现有的库位置。请为 `ort` 使用 `load-dynamic` cargo 功能。\n3. 将 `ORT_DYLIB_PATH` 设置为指向已下载的 onnxruntime 库的位置（根据操作系统不同，可能是 `onnxruntime.dll`、`libonnxruntime.so` 或 `libonnxruntime.dylib`）。这些库可以从 onnxruntime 项目的 [发布页面](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fonnxruntime\u002Freleases) 下载。\n\n大多数架构（包括编码器、解码器和编码器-解码器）都受支持。该库旨在保持与使用 [Optimum](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Foptimum) 库导出的模型兼容。关于如何使用 Optimum 将 Transformer 模型导出为 ONNX 的详细指南可在 https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Foptimum\u002Fmain\u002Fen\u002Fexporters\u002Fonnx\u002Fusage_guides\u002Fexport_a_model 上找到。创建 ONNX 模型所使用的资源与基于 PyTorch 的模型类似，只是将 PyTorch 替换为 ONNX 模型而已。由于 ONNX 模型在处理可选参数方面不如其 PyTorch 对应物灵活，因此将解码器或编码器-解码器模型导出为 ONNX 通常会产生多个文件。这些文件是预期的（但并非全部必要），可用于本库中的相应功能，具体如下表所示：\n\n| 架构                | 编码器文件 | 不带历史信息的解码器文件 | 带有历史信息的解码器文件 |\n|-----------------------------|--------------|---------------------------|------------------------|\n| 编码器（如 BERT）         | 必需     | 不使用                  | 不使用               |\n| 解码器（如 GPT2）         | 不使用     | 必需                  | 可选               |\n| 编码器-解码器（如 BART） | 必需     | 必需                  | 可选               |\n\n请注意，当“带历史信息的解码器文件”是可选但未提供时，计算效率会下降，因为模型无法使用缓存的历史键值来进行注意力机制，从而导致大量冗余计算。Optimum 库提供了导出选项，以确保生成这样的“带历史信息的解码器文件”。基础的编码器和解码器模型架构分别在 `encoder` 和 `decoder` 模块中提供，以便于使用。\n\n生成模型（纯解码器或编码器\u002F解码器架构）可在 `models` 模块中找到。针对 ONNX 模型检查点，提供了多种流水线，包括序列分类、零样本分类、标记分类（包括命名实体识别和词性标注）、问答、文本生成、摘要和翻译。这些模型在流水线中使用时，采用与 PyTorch 对应模型相同的配置和分词器文件。ONNX 模型的示例位于 `.\u002Fexamples` 目录中。\n\n## 即用型流水线\n\n基于 Hugging Face 的流水线，本 crate 提供了即用型端到端 NLP 流水线。目前可用的功能如下：\n\n**免责声明** 本仓库的贡献者不对第三方使用此处提供的预训练系统所产生的任何结果负责。\n\n\u003Cdetails>\n\u003Csummary> \u003Cb>1. 问答\u003C\u002Fb> \u003C\u002Fsummary>\n\n从给定的问题和上下文中提取答案。基于 SQuAD（斯坦福问答数据集）微调的 DistilBERT 模型。\n\n```rust\n    let qa_model = QuestionAnsweringModel::new(Default::default ()) ?;\n\nlet question = String::from(\"Amy住在哪儿？\");\nlet context = String::from(\"Amy住在阿姆斯特丹\");\n\nlet answers = qa_model.predict( & [QaInput { question, context }], 1, 32);\n```\n\n输出：\n\n```\n[Answer { score: 0.9976, start: 13, end: 21, answer: \"阿姆斯特丹\" }]\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>2. 翻译 \u003C\u002Fb> \u003C\u002Fsummary>\n\n支持广泛源语言和目标语言的翻译流水线。利用两种主要架构进行翻译任务：\n\n- 基于 Marian 的模型，适用于特定的源语言\u002F目标语言组合；\n- M2M100 模型允许在 100 种语言之间直接翻译（但对于某些选定的语言，计算成本较高且性能较低）。\n\n该库中提供了以下语言对的 Marian 预训练模型，用户也可以导入任何基于 PyTorch 的模型来进行预测：\n\n- 英语 \u003C-> 法语\n- 英语 \u003C-> 西班牙语\n- 英语 \u003C-> 葡萄牙语\n- 英语 \u003C-> 意大利语\n- 英语 \u003C-> 加泰罗尼亚语\n- 英语 \u003C-> 德语\n- 英语 \u003C-> 俄语\n- 英语 \u003C-> 中文\n- 英语 \u003C-> 荷兰语\n- 英语 \u003C-> 瑞典语\n- 英语 \u003C-> 阿拉伯语\n- 英语 \u003C-> 希伯来语\n- 英语 \u003C-> 印地语\n- 法语 \u003C-> 德语\n\n对于所提出的 Marian 预训练模型不支持的语言，用户可以使用 M2M100 模型，该模型支持 100 种语言之间的直接翻译（无需经过英语中转）。完整支持的语言列表可在[crate 文档](https:\u002F\u002Fdocs.rs\u002Frust-bert\u002Flatest\u002Frust_bert\u002Fpipelines\u002Ftranslation\u002Fenum.Language.html)中找到。\n\n```rust\nuse rust_bert::pipelines::translation::{Language, TranslationModelBuilder};\nfn main() -> anyhow::Result\u003C()> {\n    let model = TranslationModelBuilder::new()\n        .with_source_languages(vec![Language::English])\n        .with_target_languages(vec![Language::Spanish, Language::French, Language::Italian])\n        .create_model()?;\n    let input_text = \"This is a sentence to be translated\";\n    let output = model.translate(&[input_text], None, Language::French)?;\n    for sentence in output {\n        println!(\"{}\", sentence);\n    }\n    Ok(())\n}\n```\n\n输出：\n\n```\nIl s'agit d'une phrase à traduire\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>3. 摘要生成 \u003C\u002Fb> \u003C\u002Fsummary>\n\n使用预训练的 BART 模型进行摘要生成。\n\n```rust\n    let summarization_model = SummarizationModel::new(Default::default ()) ?;\n\nlet input = [\"In findings published Tuesday in Cornell University's arXiv by a team of scientists \\\nfrom the University of Montreal and a separate report published Wednesday in Nature Astronomy by a team \\\nfrom University College London (UCL), the presence of water vapour was confirmed in the atmosphere of K2-18b, \\\na planet circling a star in the constellation Leo. This is the first such discovery in a planet in its star's \\\nhabitable zone — not too hot and not too cold for liquid water to exist. The Montreal team, led by Björn Benneke, \\\nused data from the NASA's Hubble telescope to assess changes in the light coming from K2-18b's star as the planet \\\npassed between it and Earth. They found that certain wavelengths of light, which are usually absorbed by water, \\\nweakened when the planet was in the way, indicating not only does K2-18b have an atmosphere, but the atmosphere \\\ncontains water in vapour form. The team from UCL then analyzed the Montreal team's data using their own software \\\nand confirmed their conclusion. This was not the first time scientists have found signs of water on an exoplanet, \\\nbut previous discoveries were made on planets with high temperatures or other pronounced differences from Earth. \\\n\\\"This is the first potentially habitable planet where the temperature is right and where we now know there is water,\\\" \\\nsaid UCL astronomer Angelos Tsiaras. \\\"It's the best candidate for habitability right now.\\\" \\\"It's a good sign\\\", \\\nsaid Ryan Cloutier of the Harvard–Smithsonian Center for Astrophysics, who was not one of either study's authors. \\\n\\\"Overall,\\\" he continued, \\\"the presence of water in its atmosphere certainly improves the prospect of K2-18b being \\\na potentially habitable planet, but further observations will be required to say for sure. \\\"\nK2-18b was first identified in 2015 by the Kepler space telescope. It is about 110 light-years from Earth and larger \\\nbut less dense. Its star, a red dwarf, is cooler than the Sun, but the planet's orbit is much closer, such that a year \\\non K2-18b lasts 33 Earth days. According to The Guardian, astronomers were optimistic that NASA's James Webb space \\\ntelescope — scheduled for launch in 2021 — and the European Space Agency's 2028 ARIEL program, could reveal more \\\nabout exoplanets like K2-18b.\"];\n\nlet output = summarization_model.summarize( & input);\n```\n\n（示例来自：\n[WikiNews](https:\u002F\u002Fen.wikinews.org\u002Fwiki\u002FAstronomers_find_water_vapour_in_atmosphere_of_exoplanet_K2-18b))\n\n输出：\n\n```\n\"科学家在距离地球110光年的行星K2-18b上发现了水蒸气。这是首次在恒星宜居带内的行星上发现此类现象。该行星的温度既不太热也不太冷，适合液态水存在。\"\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>4. 对话模型 \u003C\u002Fb> \u003C\u002Fsummary>\n\n基于微软的\n[DialoGPT](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002FDialoGPT) 的对话模型。此管道允许生成人类与模型之间的一轮或多轮对话。DialoGPT 的页面指出：\n\n> 人类评估结果显示，在单轮对话图灵测试中，DialoGPT 生成的回复质量可与人类回复相媲美。\n> ([DialoGPT 仓库](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002FDialoGPT))\n\n该模型使用 `ConversationManager` 来跟踪正在进行的对话并为其生成回复。\n\n```rust\nuse rust_bert::pipelines::conversation::{ConversationModel, ConversationManager};\n\nlet conversation_model = ConversationModel::new(Default::default ());\nlet mut conversation_manager = ConversationManager::new();\n\nlet conversation_id = conversation_manager.create(\"Going to the movies tonight - any suggestions?\");\nlet output = conversation_model.generate_responses( & mut conversation_manager);\n```\n\n示例输出：\n\n```\n\"The Big Lebowski.\"\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>5. 自然语言生成 \u003C\u002Fb> \u003C\u002Fsummary>\n\n根据提示生成语言。提供 GPT2 和 GPT 作为基础模型。包含束搜索、top-k 采样、核采样、温度设置和重复惩罚等技术。支持从多个提示批量生成句子。如果模型有填充标记，则序列将用填充标记左填充；否则将用未知标记填充。这可能会影响结果，建议提交长度相近的提示以获得最佳效果。\n\n```rust\n    let model = GPT2Generator::new(Default::default ()) ?;\n\nlet input_context_1 = \"The dog\";\nlet input_context_2 = \"The cat was\";\n\nlet generate_options = GenerateOptions {\nmax_length: 30,\n..Default::default ()\n};\n\nlet output = model.generate(Some( & [input_context_1, input_context_2]), generate_options);\n```\n\n示例输出：\n\n```\n[\n    \"然而，这只狗的主人不愿透露姓名。根据诉讼文件，该动物的主人是一位29岁的\"\n    \"这只狗一直都是家庭的一员。‘他一直都会是我的狗，也一直都在照顾我’\"\n    \"这只狗现在已经能在家里待了三个多月了。‘它是一只非常好的狗。她’\"\n    \"这只猫本月早些时候在死者的一位亲属家中被发现。猫的主人希望匿名，\"\n    \"这只猫是由两岁的贾兹敏从街上救回来的。‘我不知道该怎么办，’她说\"\n    \"这只猫遭到两只流浪狗的袭击，随后被送往医院。另外两只猫也在此次袭击中受伤，正在接受治疗。\"\n]\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>6. 零样本分类 \u003C\u002Fb> \u003C\u002Fsummary>\n\n使用针对自然语言推理任务微调的模型，对输入句子进行零样本分类，并提供标签。\n\n```rust\n    let sequence_classification_model = ZeroShotClassificationModel::new(Default::default ()) ?;\n\nlet input_sentence = \"您在2020年会投票给谁？\";\nlet input_sequence_2 = \"首相宣布了一项刺激计划，该计划受到了反对派的广泛批评。\";\nlet candidate_labels = & [\"政治\", \"公共卫生\", \"经济\", \"体育\"];\n\nlet output = sequence_classification_model.predict_multilabel(\n& [input_sentence, input_sequence_2],\ncandidate_labels,\nNone,\n128,\n);\n```\n\n输出：\n\n```\n[\n  [ Label { \"政治\", score: 0.972 }, Label { \"公共卫生\", score: 0.032 }, Label {\"经济\", score: 0.006 }, Label {\"体育\", score: 0.004 } ],\n  [ Label { \"政治\", score: 0.975 }, Label { \"公共卫生\", score: 0.0818 }, Label {\"经济\", score: 0.852 }, Label {\"体育\", score: 0.001 } ],\n]\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>7. 情感分析 \u003C\u002Fb> \u003C\u002Fsummary>\n\n预测句子的二元情感。基于SST-2数据集微调的DistilBERT模型。\n\n```rust\n    let sentiment_classifier = SentimentModel::new(Default::default ()) ?;\n\nlet input = [\n\"这可能是我最喜欢的电影，讲述的是无私、牺牲和为崇高事业奉献的故事，但它并不说教或枯燥。\",\n\"这部电影试图同时成为太多东西：辛辣的政治讽刺、好莱坞大片、俗气的爱情喜剧、宣扬家庭价值观的宣传片……\",\n\"如果你喜欢原汁原味、直击心灵的笑声，你一定会喜欢这部电影。无论你年轻还是年长，都会爱上这部电影，甚至连我妈妈都喜欢。\",\n];\n\nlet output = sentiment_classifier.predict( & input);\n```\n\n（示例来自[IMDb](http:\u002F\u002Fwww.imdb.com)）\n\n输出：\n\n```\n[\n    Sentiment { polarity: Positive, score: 0.9981985493795946 },\n    Sentiment { polarity: Negative, score: 0.9927982091903687 },\n    Sentiment { polarity: Positive, score: 0.9997248985164333 }\n]\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>8. 命名实体识别 \u003C\u002Fb> \u003C\u002Fsummary>\n\n从文本中提取实体（人名、地点、组织、其他）。基于CoNNL03数据集微调的BERT cased large模型，由巴伐利亚州立图书馆的[MDZ数字图书馆团队](https:\u002F\u002Fgithub.com\u002Fdbmdz)贡献。目前提供英语、德语、西班牙语和荷兰语版本。\n\n```rust\n    let ner_model = NERModel::new( default::default ()) ?;\n\nlet input = [\n\"我叫艾米。我住在巴黎。\",\n\"巴黎是法国的一座城市。\"\n];\n\nlet output = ner_model.predict( & input);\n```\n\n输出：\n\n```\n[\n  [\n    Entity { word: \"艾米\", score: 0.9986, label: \"I-PER\" }\n    Entity { word: \"巴黎\", score: 0.9985, label: \"I-LOC\" }\n  ],\n  [\n    Entity { word: \"巴黎\", score: 0.9988, label: \"I-LOC\" }\n    Entity { word: \"法国\", score: 0.9993, label: \"I-LOC\" }\n  ]\n]\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>9. 关键词\u002F关键短语提取\u003C\u002Fb> \u003C\u002Fsummary>\n\n从输入文档中提取关键词和关键短语。\n\n```rust\nfn main() -> anyhow::Result\u003C()> {\n    let keyword_extraction_model = KeywordExtractionModel::new(Default::default())?;\n\n    let input = \"Rust是一种多范式、通用的编程语言。\\\n       Rust强调性能、类型安全和并发性。Rust通过强制内存安全——即确保所有引用都指向有效的内存——来实现这一点，而无需像其他内存安全语言那样使用垃圾回收器或引用计数。为了同时保证内存安全并防止并发数据竞争，Rust的借用检查器会在编译时跟踪程序中所有引用的对象生命周期和变量作用域。Rust在系统编程领域很受欢迎，同时也提供了包括函数式编程结构在内的高级特性。\";\n\n    let output = keyword_extraction_model.predict(&[input])?;\n}\n```\n\n输出：\n\n```\n\"rust\" - 0.50910604\n\"programming\" - 0.35731024\n\"concurrency\" - 0.33825397\n\"concurrent\" - 0.31229728\n\"program\" - 0.29115444\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>10. 词性标注 \u003C\u002Fb> \u003C\u002Fsummary>\n\n从文本中提取词性标签（名词、动词、形容词等）。\n\n```rust\n    let pos_model = POSModel::new( default::default ()) ?;\n\nlet input = [\"我的名字是鲍勃\"];\n\nlet output = pos_model.predict( & input);\n```\n\n输出：\n\n```\n[\n    Entity { word: \"My\", score: 0.1560, label: \"PRP\" }\n    Entity { word: \"name\", score: 0.6565, label: \"NN\" }\n    Entity { word: \"is\", score: 0.3697, label: \"VBZ\" }\n    Entity { word: \"Bob\", score: 0.7460, label: \"NNP\" }\n]\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>11. 句子嵌入 \u003C\u002Fb> \u003C\u002Fsummary>\n\n生成句子嵌入（向量表示）。这些可以用于密集信息检索等应用。\n\n```rust\n    let model = SentenceEmbeddingsBuilder::remote(\nSentenceEmbeddingsModelType::AllMiniLmL12V2\n).create_model() ?;\n\nlet sentences = [\n\"这是一个示例句子\",\n\"每个句子都会被转换\"\n];\n\nlet output = model.encode( & sentences) ?;\n```\n\n输出：\n\n```\n[\n    [-0.000202666, 0.08148022, 0.03136178, 0.002920636 ...],\n    [0.064757116, 0.048519745, -0.01786038, -0.0479775 ...]\n]\n```\n\n\u003C\u002Fdetails>\n \n\u003Cdetails>\n\u003Csummary> \u003Cb>12. 掩码语言模型 \u003C\u002Fb> \u003C\u002Fsummary>\n\n预测输入句子中的掩码词汇。\n\n```rust\n    let model = MaskedLanguageModel::new(Default::default ()) ?;\n\nlet sentences = [\n\"你好，我是一名\u003Cmask>学生\",\n\"巴黎是法国的\u003Cmask>。它位于欧洲的\u003Cmask>。\"\n];\n\nlet output = model.predict( & sentences);\n```\n\n输出：\n\n```\n[\n    [MaskedToken { text: \"college\", id: 2267, score: 8.091}],\n    [\n        MaskedToken { text: \"capital\", id: 3007, score: 16.7249}, \n        MaskedToken { text: \"located\", id: 2284, score: 9.0452}\n    ]\n]\n```\n\n\u003C\u002Fdetails>\n\n## 基准测试\n\n对于简单的流水线任务（序列分类、标记分类、问答），Python 和 Rust 之间的性能预计是相当的。这是因为这些流水线中最耗时的部分是语言模型本身，而它们在 Torch 后端中采用了相同的实现。论文 [Rust 中的端到端 NLP 流水线](https:\u002F\u002Fwww.aclweb.org\u002Fanthology\u002F2020.nlposs-1.4\u002F) 提供了一个涵盖所有流水线的基准测试部分。\n\n对于文本生成任务（摘要、翻译、对话、自由文本生成），则可以预期显著的优势（处理速度可快 2 到 4 倍，具体取决于输入和应用场景）。文章 [使用 Rust 加速文本生成](https:\u002F\u002Fguillaume-be.github.io\u002F2020-11-21\u002Fgeneration_benchmarks) 专门讨论了这些文本生成应用，并提供了与 Python 性能对比的更多细节。\n\n## 加载预训练及自定义模型权重\n\n基础模型和特定任务的头部也适用于希望公开自己基于 Transformer 的模型的用户。在 `.\u002Fexamples` 目录下提供了使用原生 Rust 分词器库准备数据的示例，涵盖 BERT、DistilBERT、RoBERTa、GPT、GPT2 和 BART 等模型。需要注意的是，从 PyTorch 导入模型时，参数命名规范需要与 Rust 模式保持一致。如果权重文件中找不到任何模型参数的权重，加载预训练权重将会失败。若需跳过这一质量检查，可以从变量存储中调用替代方法 `load_partial`。\n\n预训练模型可在 Hugging Face 的 [模型中心](https:\u002F\u002Fhuggingface.co\u002Fmodels?filter=rust) 上找到，并可使用本库中定义的 `RemoteResources` 进行加载。\n\n`.\u002Futils` 目录中包含一个转换工具脚本，用于将 PyTorch 权重转换为与此库兼容的权重集。该脚本需要安装 Python 和 `torch`，使用方法如下：`python .\u002Futils\u002Fconvert_model.py path\u002Fto\u002Fpytorch_model.bin`，其中 `path\u002Fto\u002Fpytorch_model.bin` 是原始 PyTorch 权重文件的路径。\n\n```bash\npython3 -m venv .venv\nsource .venv\u002Fbin\u002Factivate\n\npip install -r requirements.txt\n\npython utils\u002Fconvert_model.py path\u002Fto\u002Fpytorch_model.bin\n```\n\n## 引用\n\n如果您在工作中使用了 `rust-bert`，请引用 [Rust 中的端到端 NLP 流水线](https:\u002F\u002Fwww.aclweb.org\u002Fanthology\u002F2020.nlposs-1.4\u002F)：\n\n```bibtex\n@inproceedings{becquin-2020-end,\n    title = \"End-to-end {NLP} Pipelines in Rust\",\n    author = \"Becquin, Guillaume\",\n    booktitle = \"Second Workshop for NLP Open Source Software (NLP-OSS) Proceedings\",\n    year = \"2020\",\n    publisher = \"Association for Computational Linguistics\",\n    url = \"https:\u002F\u002Fwww.aclweb.org\u002Fanthology\u002F2020.nlposs-1.4\",\n    pages = \"20--25\",\n}\n```\n\n## 致谢\n\n感谢 [Hugging Face](https:\u002F\u002Fhuggingface.co) 托管了一组与本 Rust 库兼容的权重。现成的预训练模型列表可在 [https:\u002F\u002Fhuggingface.co\u002Fmodels?filter=rust](https:\u002F\u002Fhuggingface.co\u002Fmodels?filter=rust) 查看。","# rust-bert 快速上手指南\n\n`rust-bert` 是一个原生的 Rust 自然语言处理（NLP）库，提供了 Hugging Face Transformers 库的 Rust 实现。它支持多线索分词和 GPU 推理，涵盖问答、翻译、摘要、文本生成等多种任务。\n\n## 环境准备\n\n### 系统要求\n- **操作系统**: Linux, Windows, macOS\n- **Rust 版本**: 建议使用最新稳定版\n- **LibTorch 版本**: 需要 `v2.4` (若官网无此版本，可通过修改下载链接获取，例如 CUDA 12.4 版本)\n\n### 前置依赖\n该库依赖 C++ 的 LibTorch API。你可以选择**手动安装**（推荐，构建更快）或**自动安装**。\n\n> **注意**：预训练模型较大（几百 MB 到几 GB），首次运行会自动下载到缓存目录（默认 `~\u002F.cache\u002F.rustbert`）。可通过设置 `RUSTBERT_CACHE` 环境变量修改缓存路径。\n\n## 安装步骤\n\n### 方案一：手动安装 LibTorch（推荐）\n\n1. **下载 LibTorch**\n   访问 [PyTorch 官网](https:\u002F\u002Fpytorch.org\u002Fget-started\u002Flocally\u002F) 下载 v2.4 版本。\n   *示例链接 (Linux + CUDA 12.4)*:\n   `https:\u002F\u002Fdownload.pytorch.org\u002Flibtorch\u002Fcu124\u002Flibtorch-cxx11-abi-shared-with-deps-2.4.0%2Bcu124.zip`\n\n2. **解压库文件**\n   将下载的压缩包解压到你选择的目录（例如 `\u002Fopt\u002Flibtorch` 或 `X:\\libtorch`）。\n\n3. **配置环境变量**\n\n   **Linux:**\n   ```bash\n   export LIBTORCH=\u002Fpath\u002Fto\u002Flibtorch\n   export LD_LIBRARY_PATH=${LIBTORCH}\u002Flib:$LD_LIBRARY_PATH\n   ```\n\n   **Windows (PowerShell):**\n   ```powershell\n   $Env:LIBTORCH = \"X:\\path\\to\\libtorch\"\n   $Env:Path += \";X:\\path\\to\\libtorch\\lib\"\n   ```\n\n   **macOS (Homebrew):**\n   ```bash\n   brew install pytorch jq\n   export LIBTORCH=$(brew --cellar pytorch)\u002F$(brew info --json pytorch | jq -r '.[0].installed[0].version')\n   export LD_LIBRARY_PATH=${LIBTORCH}\u002Flib:$LD_LIBRARY_PATH\n   ```\n\n4. **添加依赖**\n   在项目的 `Cargo.toml` 中添加：\n   ```toml\n   [dependencies]\n   rust-bert = \"0.20\" # 请根据 crates.io 最新版本调整\n   ```\n\n### 方案二：自动安装（构建时下载）\n\n如果你不想手动配置 LibTorch，可以启用 `download-libtorch` 特性让构建脚本自动下载（首次构建较慢）。\n\n在 `Cargo.toml` 中配置：\n```toml\n[dependencies]\nrust-bert = { version = \"0.20\", features = [\"download-libtorch\"] }\n```\n\n*如需下载 CUDA 版本，请在构建前设置环境变量*:\n```bash\nexport TORCH_CUDA_VERSION=cu124\n```\n\n## 基本使用\n\n以下是一个最简单的**问答任务**示例。代码会自动下载所需的预训练模型。\n\n```rust\nuse rust_bert::pipelines::question_answering::{QaInput, QuestionAnsweringModel};\n\nfn main() -> Result\u003C(), Box\u003Cdyn std::error::Error>> {\n    \u002F\u002F 初始化问答模型\n    let qa_model = QuestionAnsweringModel::new(Default::default())?;\n\n    let question = String::from(\"Where does Amy live ?\");\n    let context = String::from(\"Amy lives in Amsterdam\");\n\n    \u002F\u002F 执行预测\n    let answers = qa_model.predict(&[QaInput { question, context }], 1, 32);\n\n    \u002F\u002F 输出结果\n    println!(\"{:?}\", answers);\n    \n    Ok(())\n}\n```\n\n**预期输出：**\n```text\n[Answer { score: 0.9976, start: 13, end: 21, answer: \"Amsterdam\" }]\n```\n\n### 验证安装\n你可以克隆仓库并运行官方示例来验证环境是否配置成功：\n```bash\ngit clone git@github.com:guillaume-be\u002Frust-bert.git\ncd rust-bert\ncargo run --example sentence_embeddings\n```\n\n> **提示**：国内开发者若遇到模型下载缓慢问题，可尝试配置代理或手动下载模型文件至 `RUSTBERT_CACHE` 指定目录。","某电商初创团队正在开发一款实时客服辅助系统，需要在用户咨询瞬间自动提取关键实体并分析情绪，以便快速路由工单。\n\n### 没有 rust-bert 时\n- **部署沉重**：团队必须捆绑庞大的 Python 环境和 PyTorch 依赖，导致容器镜像体积超过 1GB，启动缓慢且占用大量内存。\n- **并发瓶颈**：由于 Python 全局解释器锁（GIL）的限制，在处理高并发请求时，多线程分词和推理效率低下，响应延迟经常突破 500ms。\n- **运维复杂**：生产环境需同时维护 Rust 业务代码和 Python AI 服务两套运行时，跨语言调用增加了接口不稳定风险和调试难度。\n- **资源浪费**：为了维持低延迟，不得不为每个微服务实例分配独立的 GPU 资源，导致云端算力成本居高不下。\n\n### 使用 rust-bert 后\n- **轻量原生**：直接利用 rust-bert 的原生特性，将 BERT 和 DistilBERT 模型编译为单一二进制文件，镜像体积缩减至 50MB 以内，实现秒级启动。\n- **高性能并发**：借助 rust-bert 支持的多线程分词与 GPU 推理加速，系统在同等硬件下吞吐量提升 3 倍，平均响应时间稳定在 50ms 以内。\n- **架构统一**：整个链路仅使用 Rust 一种语言，消除了跨进程通信开销，开发人员可在同一项目中无缝调试从数据预处理到模型预测的全流程。\n- **成本优化**：凭借高效的内存管理和计算性能，团队成功将原本需要 GPU 支撑的任务迁移至高配 CPU 实例运行，每月云账单降低 60%。\n\nrust-bert 通过将顶尖 NLP 能力无缝融入 Rust 生态，让高性能、低成本的实时智能应用开发变得触手可及。","https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-be_rust-bert_24f35592.png","guillaume-be",null,"https:\u002F\u002Foss.gittoolsai.com\u002Favatars\u002Fguillaume-be_91e12d0e.png","https:\u002F\u002Fguillaume-be.github.io","https:\u002F\u002Fgithub.com\u002Fguillaume-be",[79,83],{"name":80,"color":81,"percentage":82},"Rust","#dea584",99.7,{"name":84,"color":85,"percentage":86},"Python","#3572A5",0.3,3059,245,"2026-04-05T18:10:08","Apache-2.0",4,"Linux, macOS, Windows","非必需。支持 GPU 推理（通过 LibTorch），若启用需安装对应 CUDA 版本的 LibTorch（文档示例提及 cu124\u002FCUDA 12.4）。CPU 版本默认可用。显存大小取决于具体模型（从几百 MB 到几 GB 不等）。","未说明（取决于模型大小，模型文件范围从几百 MB 到几 GB）",{"notes":96,"python":97,"dependencies":98},"1. 该工具是纯 Rust 实现，不需要 Python 环境。2. 必须手动或自动下载 LibTorch C++ 库（CPU 或 CUDA 版本），推荐使用 v2.4。3. 预训练模型会下载到缓存目录（默认 ~\u002F.cache\u002F.rustbert），可通过 RUSTBERT_CACHE 环境变量修改。4. 若使用 ONNX 后端，需单独配置 onnxruntime 动态链接库。5. 首次构建若开启自动下载 LibTorch CUDA 版本，可能需要较长时间且占用数 GB 空间。","不需要 (这是一个 Rust 原生库)",[99,100,101,102,103],"Rust (工具链)","LibTorch (v2.4+)","tch-rs","rust-tokenizers","ort (可选，用于 ONNX 支持)",[35,105,14,15,60,13],"音频",[107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123],"deep-learning","nlp","transformer","bert","rust-lang","rust","machine-learning","ner","sentiment-analysis","question-answering","language-generation","gpt-2","roberta","gpt","bart","electra","translation","2026-03-27T02:49:30.150509","2026-04-06T19:58:14.309171",[127,132,137,142,147,151],{"id":128,"question_zh":129,"answer_zh":130,"source_url":131},19954,"在使用对话模型生成第 4 条回复时出现 Panic 错误（sequence_history.drain），如何解决？","该问题通常由训练数据中的填充（padding）设置不当引起。如果在使用 `torch.nn.utils.rnn.pad_sequence` 时，tokenizer 的 `_pad_token` 为 None，默认会使用 0 作为填充值，这可能与模型配置的 `pad_token_id` 不匹配。\n解决方案：\n1. 确保训练时的填充值与模型配置一致。建议显式指定 `padding_value`，例如使用 `tokenizer.eos_token_id` 而不是默认的 0。\n2. 修改 Python 训练代码中的 padding 逻辑：\n```python\nif tokenizer._eos_token is None:\n    return pad_sequence(examples, batch_first=True)\nreturn pad_sequence(examples, batch_first=True, padding_value=tokenizer.eos_token_id)\n```\n3. 重新训练模型以应用正确的填充策略。该 Panic 问题已在 rust-bert 后续版本中通过 PR #86 修复。","https:\u002F\u002Fgithub.com\u002Fguillaume-be\u002Frust-bert\u002Fissues\u002F85",{"id":133,"question_zh":134,"answer_zh":135,"source_url":136},19955,"如何在 rust-bert 中为 GPT2 模型使用非默认的 Tokenizer（如 T5Tokenizer）？","早期版本不支持为 Generator 指定自定义 Tokenizer，导致加载词汇表时报错。维护者已推出新功能支持灵活配置。\n解决方案：\n1. 使用支持该特性的分支或最新版本（如 `flexible_tokenizer_generator` 分支）。\n2. 在代码中显式传入自定义的词汇表和合并文件资源：\n```rust\nlet generate_config = GenerateConfig {\n    model_resource,\n    config_resource,\n    vocab_resource: custom_vocab_resource, \u002F\u002F 指向 spiece.model 等资源\n    merges_resource: custom_merges_resource,\n    device: Device::Cpu, \u002F\u002F 或 Device::cuda_if_available\n    ..Default::default()\n};\n```\n3. 确保 `vocab_resource` 和 `merges_resource` 指向正确的文件路径，以匹配非默认 Tokenizer 的格式。","https:\u002F\u002Fgithub.com\u002Fguillaume-be\u002Frust-bert\u002Fissues\u002F221",{"id":138,"question_zh":139,"answer_zh":140,"source_url":141},19956,"在 gRPC 或多线程环境中加载一次模型并重复使用时，遇到 '*mut C_tensor cannot be shared between threads safely' 错误怎么办？","这是因为 rust-bert 的某些生成器（Generator）内部状态是可变的（Mutable），无法直接在多线程间共享引用。\n解决方案：\n1. **推荐方案**：使用维护者提供的 `non_mutable_generators` 特性或更新到支持不可变生成器的版本。这将允许通过共享引用（&Model）安全地调用生成方法。\n2. **替代方案**：如果必须使用旧版本，可以考虑在每个线程中加载模型副本（不推荐，耗内存），或者使用互斥锁（Mutex）包裹模型实例，但这会降低并发性能。\n3. 代码示例（使用新版不可变生成器）：\n```rust\n\u002F\u002F 定义静态模型实例\nstatic MODEL: Lazy\u003CModel> = Lazy::new(|| Model::load());\n\n\u002F\u002F 在 handler 中直接通过引用调用，无需可变借用\nlet output = MODEL.sst.predict(&input);\n```","https:\u002F\u002Fgithub.com\u002Fguillaume-be\u002Frust-bert\u002Fissues\u002F47",{"id":143,"question_zh":144,"answer_zh":145,"source_url":146},19957,"如何在 Rust 中加载并使用 Hugging Face 上托管的自定义 GPT2 模型？","需要先将 PyTorch 模型转换为 rust-bert 格式，然后正确配置资源路径。\n步骤：\n1. **转换模型**：使用 rust-bert 提供的工具将 `pytorch_model.bin` 转换为 `rust_model.ot`。\n2. **准备资源**：确保拥有 `config.json`、词汇表文件（如 `vocab.json`, `merges.txt` 或 `spiece.model`）以及转换后的 `rust_model.ot`。\n3. **代码加载**：使用 `LocalResource` 指向本地文件路径构建 `GenerateConfig`：\n```rust\nlet model_resource = LocalResource { local_path: PathBuf::from(\"path\u002Fto\u002Frust_model.ot\") };\nlet config_resource = LocalResource { local_path: PathBuf::from(\"path\u002Fto\u002Fconfig.json\") };\n\u002F\u002F 配置生成器...\nlet generator = GPT2Generator::new(model_resource, config_resource, vocab_resource, merges_resource)?;\n```\n4. 如果涉及特殊 Tokenizer（如日语模型），需参考 FAQ 中关于自定义 Tokenizer 的配置方法。","https:\u002F\u002Fgithub.com\u002Fguillaume-be\u002Frust-bert\u002Fissues\u002F159",{"id":148,"question_zh":149,"answer_zh":150,"source_url":136},19958,"使用 GPT2 模型时，如何正确配置设备（CPU\u002FGPU）以避免运行错误？","在创建 `GenerateConfig` 时，必须显式指定设备类型，否则可能导致模型加载失败或推理错误。\n配置方法：\n1. **CPU 模式**：设置 `device: Device::Cpu`。\n2. **GPU 模式**：设置 `device: Device::cuda_if_available` 或具体的 CUDA 设备 ID。\n示例代码：\n```rust\nlet generate_config = GenerateConfig {\n    model_resource,\n    config_resource,\n    vocab_resource,\n    merges_resource,\n    device: Device::Cpu, \u002F\u002F 关键配置项\n    ..Default::default()\n};\n```\n注意：如果使用半精度（half precision）模型，确保设备配置与模型精度兼容，部分用户反馈在 CPU 上半精度模型可能表现异常，建议全精度模型或在 GPU 上测试。",{"id":152,"question_zh":153,"answer_zh":154,"source_url":131},19959,"训练数据中的填充值（Padding Value）设置错误会导致什么后果，如何避免？","如果训练时使用的填充值（如默认的 0）与模型配置文件中的 `pad_token_id` 不一致，会导致推理时出现全零输出或 Panic 错误（如 `sequence_history.drain` 越界）。\n避免方法：\n1. 检查 Tokenizer 是否定义了 `_pad_token`。如果未定义，`pad_sequence` 会默认用 0 填充。\n2. 强制指定填充值为 `eos_token_id` 或其他与模型配置一致的 ID：\n```python\nreturn pad_sequence(examples, batch_first=True, padding_value=tokenizer.eos_token_id)\n```\n3. 验证转换后的 `rust_model.ot` 和 `config.json` 中的 `pad_token_id` 是否与训练数据实际使用的填充值匹配。如果不匹配，需重新训练或微调数据处理脚本。",[]]