[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"similar-ahkarami--Deep-Learning-in-Production":3,"tool-ahkarami--Deep-Learning-in-Production":61},[4,18,26,36,44,53],{"id":5,"name":6,"github_repo":7,"description_zh":8,"stars":9,"difficulty_score":10,"last_commit_at":11,"category_tags":12,"status":17},4358,"openclaw","openclaw\u002Fopenclaw","OpenClaw 是一款专为个人打造的本地化 AI 助手，旨在让你在自己的设备上拥有完全可控的智能伙伴。它打破了传统 AI 助手局限于特定网页或应用的束缚，能够直接接入你日常使用的各类通讯渠道，包括微信、WhatsApp、Telegram、Discord、iMessage 等数十种平台。无论你在哪个聊天软件中发送消息，OpenClaw 都能即时响应，甚至支持在 macOS、iOS 和 Android 设备上进行语音交互，并提供实时的画布渲染功能供你操控。\n\n这款工具主要解决了用户对数据隐私、响应速度以及“始终在线”体验的需求。通过将 AI 部署在本地，用户无需依赖云端服务即可享受快速、私密的智能辅助，真正实现了“你的数据，你做主”。其独特的技术亮点在于强大的网关架构，将控制平面与核心助手分离，确保跨平台通信的流畅性与扩展性。\n\nOpenClaw 非常适合希望构建个性化工作流的技术爱好者、开发者，以及注重隐私保护且不愿被单一生态绑定的普通用户。只要具备基础的终端操作能力（支持 macOS、Linux 及 Windows WSL2），即可通过简单的命令行引导完成部署。如果你渴望拥有一个懂你",349277,3,"2026-04-06T06:32:30",[13,14,15,16],"Agent","开发框架","图像","数据工具","ready",{"id":19,"name":20,"github_repo":21,"description_zh":22,"stars":23,"difficulty_score":10,"last_commit_at":24,"category_tags":25,"status":17},3808,"stable-diffusion-webui","AUTOMATIC1111\u002Fstable-diffusion-webui","stable-diffusion-webui 是一个基于 Gradio 构建的网页版操作界面，旨在让用户能够轻松地在本地运行和使用强大的 Stable Diffusion 图像生成模型。它解决了原始模型依赖命令行、操作门槛高且功能分散的痛点，将复杂的 AI 绘图流程整合进一个直观易用的图形化平台。\n\n无论是希望快速上手的普通创作者、需要精细控制画面细节的设计师，还是想要深入探索模型潜力的开发者与研究人员，都能从中获益。其核心亮点在于极高的功能丰富度：不仅支持文生图、图生图、局部重绘（Inpainting）和外绘（Outpainting）等基础模式，还独创了注意力机制调整、提示词矩阵、负向提示词以及“高清修复”等高级功能。此外，它内置了 GFPGAN 和 CodeFormer 等人脸修复工具，支持多种神经网络放大算法，并允许用户通过插件系统无限扩展能力。即使是显存有限的设备，stable-diffusion-webui 也提供了相应的优化选项，让高质量的 AI 艺术创作变得触手可及。",162132,"2026-04-05T11:01:52",[14,15,13],{"id":27,"name":28,"github_repo":29,"description_zh":30,"stars":31,"difficulty_score":32,"last_commit_at":33,"category_tags":34,"status":17},1381,"everything-claude-code","affaan-m\u002Feverything-claude-code","everything-claude-code 是一套专为 AI 编程助手（如 Claude Code、Codex、Cursor 等）打造的高性能优化系统。它不仅仅是一组配置文件，而是一个经过长期实战打磨的完整框架，旨在解决 AI 代理在实际开发中面临的效率低下、记忆丢失、安全隐患及缺乏持续学习能力等核心痛点。\n\n通过引入技能模块化、直觉增强、记忆持久化机制以及内置的安全扫描功能，everything-claude-code 能显著提升 AI 在复杂任务中的表现，帮助开发者构建更稳定、更智能的生产级 AI 代理。其独特的“研究优先”开发理念和针对 Token 消耗的优化策略，使得模型响应更快、成本更低，同时有效防御潜在的攻击向量。\n\n这套工具特别适合软件开发者、AI 研究人员以及希望深度定制 AI 工作流的技术团队使用。无论您是在构建大型代码库，还是需要 AI 协助进行安全审计与自动化测试，everything-claude-code 都能提供强大的底层支持。作为一个曾荣获 Anthropic 黑客大奖的开源项目，它融合了多语言支持与丰富的实战钩子（hooks），让 AI 真正成长为懂上",150720,2,"2026-04-11T11:33:10",[14,13,35],"语言模型",{"id":37,"name":38,"github_repo":39,"description_zh":40,"stars":41,"difficulty_score":32,"last_commit_at":42,"category_tags":43,"status":17},2271,"ComfyUI","Comfy-Org\u002FComfyUI","ComfyUI 是一款功能强大且高度模块化的视觉 AI 引擎，专为设计和执行复杂的 Stable Diffusion 图像生成流程而打造。它摒弃了传统的代码编写模式，采用直观的节点式流程图界面，让用户通过连接不同的功能模块即可构建个性化的生成管线。\n\n这一设计巧妙解决了高级 AI 绘图工作流配置复杂、灵活性不足的痛点。用户无需具备编程背景，也能自由组合模型、调整参数并实时预览效果，轻松实现从基础文生图到多步骤高清修复等各类复杂任务。ComfyUI 拥有极佳的兼容性，不仅支持 Windows、macOS 和 Linux 全平台，还广泛适配 NVIDIA、AMD、Intel 及苹果 Silicon 等多种硬件架构，并率先支持 SDXL、Flux、SD3 等前沿模型。\n\n无论是希望深入探索算法潜力的研究人员和开发者，还是追求极致创作自由度的设计师与资深 AI 绘画爱好者，ComfyUI 都能提供强大的支持。其独特的模块化架构允许社区不断扩展新功能，使其成为当前最灵活、生态最丰富的开源扩散模型工具之一，帮助用户将创意高效转化为现实。",108322,"2026-04-10T11:39:34",[14,15,13],{"id":45,"name":46,"github_repo":47,"description_zh":48,"stars":49,"difficulty_score":32,"last_commit_at":50,"category_tags":51,"status":17},6121,"gemini-cli","google-gemini\u002Fgemini-cli","gemini-cli 是一款由谷歌推出的开源 AI 命令行工具，它将强大的 Gemini 大模型能力直接集成到用户的终端环境中。对于习惯在命令行工作的开发者而言，它提供了一条从输入提示词到获取模型响应的最短路径，无需切换窗口即可享受智能辅助。\n\n这款工具主要解决了开发过程中频繁上下文切换的痛点，让用户能在熟悉的终端界面内直接完成代码理解、生成、调试以及自动化运维任务。无论是查询大型代码库、根据草图生成应用，还是执行复杂的 Git 操作，gemini-cli 都能通过自然语言指令高效处理。\n\n它特别适合广大软件工程师、DevOps 人员及技术研究人员使用。其核心亮点包括支持高达 100 万 token 的超长上下文窗口，具备出色的逻辑推理能力；内置 Google 搜索、文件操作及 Shell 命令执行等实用工具；更独特的是，它支持 MCP（模型上下文协议），允许用户灵活扩展自定义集成，连接如图像生成等外部能力。此外，个人谷歌账号即可享受免费的额度支持，且项目基于 Apache 2.0 协议完全开源，是提升终端工作效率的理想助手。",100752,"2026-04-10T01:20:03",[52,13,15,14],"插件",{"id":54,"name":55,"github_repo":56,"description_zh":57,"stars":58,"difficulty_score":32,"last_commit_at":59,"category_tags":60,"status":17},4721,"markitdown","microsoft\u002Fmarkitdown","MarkItDown 是一款由微软 AutoGen 团队打造的轻量级 Python 工具，专为将各类文件高效转换为 Markdown 格式而设计。它支持 PDF、Word、Excel、PPT、图片（含 OCR）、音频（含语音转录）、HTML 乃至 YouTube 链接等多种格式的解析，能够精准提取文档中的标题、列表、表格和链接等关键结构信息。\n\n在人工智能应用日益普及的今天，大语言模型（LLM）虽擅长处理文本，却难以直接读取复杂的二进制办公文档。MarkItDown 恰好解决了这一痛点，它将非结构化或半结构化的文件转化为模型“原生理解”且 Token 效率极高的 Markdown 格式，成为连接本地文件与 AI 分析 pipeline 的理想桥梁。此外，它还提供了 MCP（模型上下文协议）服务器，可无缝集成到 Claude Desktop 等 LLM 应用中。\n\n这款工具特别适合开发者、数据科学家及 AI 研究人员使用，尤其是那些需要构建文档检索增强生成（RAG）系统、进行批量文本分析或希望让 AI 助手直接“阅读”本地文件的用户。虽然生成的内容也具备一定可读性，但其核心优势在于为机器",93400,"2026-04-06T19:52:38",[52,14],{"id":62,"github_repo":63,"name":64,"description_en":65,"description_zh":66,"ai_summary_zh":67,"readme_en":68,"readme_zh":69,"quickstart_zh":70,"use_case_zh":71,"hero_image_url":72,"owner_login":73,"owner_name":74,"owner_avatar_url":75,"owner_bio":76,"owner_company":77,"owner_location":77,"owner_email":77,"owner_twitter":77,"owner_website":78,"owner_url":79,"languages":77,"stars":80,"forks":81,"last_commit_at":82,"license":77,"difficulty_score":83,"env_os":84,"env_gpu":85,"env_ram":85,"env_deps":86,"category_tags":89,"github_topics":90,"view_count":32,"oss_zip_url":77,"oss_zip_packed_at":77,"status":17,"created_at":111,"updated_at":112,"faqs":113,"releases":124},6611,"ahkarami\u002FDeep-Learning-in-Production","Deep-Learning-in-Production","In this repository, I will share some useful notes and references about deploying deep learning-based models in production.","Deep-Learning-in-Production 是一个专注于深度学习模型落地实战的资源库，旨在帮助开发者解决从实验环境到生产部署的“最后一公里”难题。许多研究人员在训练出高精度模型后，往往面临如何将其高效、稳定地集成到实际业务系统中的挑战，例如如何处理高并发请求、如何实现跨语言调用或如何进行服务端性能优化。\n\n本资源库系统性地整理了将 PyTorch 模型投入生产所需的关键笔记与技术参考。其核心内容涵盖模型服务化架构（如 TorchServe、LitServe）、REST API 构建（基于 Flask）、云端无服务器部署（AWS Lambda）以及关键的模型转换技术，特别是如何将 PyTorch 模型追踪并转换为 C++ 接口（LibTorch）以实现高性能推理。此外，它还涉及 ONNX 运行时优化、线程安全处理及浏览器端部署等前沿话题。\n\n这份指南特别适合人工智能工程师、后端开发人员以及需要将算法原型转化为可靠产品的科研团队。对于希望深入理解 PyTorch 内部机制、掌握工业级模型部署最佳实践的技术人员而言，Deep-Learning-in-Production 提供了一","Deep-Learning-in-Production 是一个专注于深度学习模型落地实战的资源库，旨在帮助开发者解决从实验环境到生产部署的“最后一公里”难题。许多研究人员在训练出高精度模型后，往往面临如何将其高效、稳定地集成到实际业务系统中的挑战，例如如何处理高并发请求、如何实现跨语言调用或如何进行服务端性能优化。\n\n本资源库系统性地整理了将 PyTorch 模型投入生产所需的关键笔记与技术参考。其核心内容涵盖模型服务化架构（如 TorchServe、LitServe）、REST API 构建（基于 Flask）、云端无服务器部署（AWS Lambda）以及关键的模型转换技术，特别是如何将 PyTorch 模型追踪并转换为 C++ 接口（LibTorch）以实现高性能推理。此外，它还涉及 ONNX 运行时优化、线程安全处理及浏览器端部署等前沿话题。\n\n这份指南特别适合人工智能工程师、后端开发人员以及需要将算法原型转化为可靠产品的科研团队。对于希望深入理解 PyTorch 内部机制、掌握工业级模型部署最佳实践的技术人员而言，Deep-Learning-in-Production 提供了一条清晰的学习路径和丰富的实战案例，是连接学术研究与工业应用的宝贵桥梁。","# Deep-Learning-in-Production\nIn this repository, I will share some useful notes and references about deploying deep learning-based models in production.\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fahkarami_Deep-Learning-in-Production_readme_7c03a4d80257.jpg\" alt=\"Logo\"\u002F>\n\u003C\u002Fp>\n\n## Convert PyTorch Models in Production:\n- [PyTorch Production Level Tutorials](https:\u002F\u002Fpytorch.org\u002Ftutorials\u002F#production-usage) [_Fantastic_]  \n- [The road to 1.0: production ready PyTorch](https:\u002F\u002Fpytorch.org\u002F2018\u002F05\u002F02\u002Froad-to-1.0.html)\n- [PyTorch 1.0 tracing JIT and LibTorch C++ API to integrate PyTorch into NodeJS](http:\u002F\u002Fblog.christianperone.com\u002F2018\u002F10\u002Fpytorch-1-0-tracing-jit-and-libtorch-c-api-to-integrate-pytorch-into-nodejs\u002F) [_Good Article_]\n- [Model Serving in PyTorch](https:\u002F\u002Fpytorch.org\u002Fblog\u002Fmodel-serving-in-pyorch\u002F)\n- [PyTorch Summer Hackathon](https:\u002F\u002Fpytorch.devpost.com\u002F) [_Very Important_]\n- [Deploying PyTorch and Building a REST API using Flask](https:\u002F\u002Fpytorch.org\u002Ftutorials\u002Fintermediate\u002Fflask_rest_api_tutorial.html) [_Important_]\n- [PyTorch model recognizing hotdogs and not-hotdogs deployed on flask](https:\u002F\u002Fgithub.com\u002Fjaroslaw-weber\u002Fhotdog-not-hotdog)\n- [Serving PyTorch 1.0 Models as a Web Server in C++ ](https:\u002F\u002Fgithub.com\u002FWizaron\u002Fpytorch-cpp-inference) [_Useful Example_]\n- [PyTorch Internals](http:\u002F\u002Fblog.ezyang.com\u002F2019\u002F05\u002Fpytorch-internals\u002F)  [_Interesting & Useful Article_]  \n- [Flask application to support pytorch model prediction](https:\u002F\u002Fgithub.com\u002Fcraigsidcarlson\u002FPytorchFlaskApp)\n- [Serving PyTorch Model on Flask Thread-Safety](https:\u002F\u002Fdiscuss.pytorch.org\u002Ft\u002Fserving-pytorch-model-on-flask-thread-safety\u002F13921)\n- [Serving PyTorch Models on AWS Lambda with Caffe2 & ONNX](https:\u002F\u002Fmachinelearnings.co\u002Fserving-pytorch-models-on-aws-lambda-with-caffe2-onnx-7b096806cfac)\n- [Serving PyTorch Models on AWS Lambda with Caffe2 & ONNX (Another Version)](https:\u002F\u002Fblog.waya.ai\u002Fdeploy-deep-machine-learning-in-production-the-pythonic-way-a17105f1540e)\n- [Deep Dive into ONNX Runtime](https:\u002F\u002Fmedium.com\u002F@mohsen.mahmoodzadeh\u002Fa-deep-dive-into-onnx-onnx-runtime-part-1-874517c66ffc)  \n- [EuclidesDB - _multi-model machine learning feature database with PyTorch_](https:\u002F\u002Feuclidesdb.readthedocs.io\u002Fen\u002Flatest\u002F)\n- [EuclidesDB - GitHub](https:\u002F\u002Fgithub.com\u002Fperone\u002Feuclidesdb\u002F)\n- [WebDNN: Fastest DNN Execution Framework on Web Browser](https:\u002F\u002Fgithub.com\u002Fmil-tokyo\u002Fwebdnn)\n- [FastAI PyTorch Serverless API (with AWS Lambda)](https:\u002F\u002Fgithub.com\u002Falecrubin\u002Fpytorch-serverless\u002F)\n- [FastAI PyTorch in Production (discussion)](http:\u002F\u002Fforums.fast.ai\u002Ft\u002Ffastai-pytorch-in-production\u002F16928)   \n- [OpenMMLab Model Deployment Framework](https:\u002F\u002Fgithub.com\u002Fopen-mmlab\u002Fmmdeploy)\n- [PyTorch Native Architecture Optimization: torchao](https:\u002F\u002Fpytorch.org\u002Fblog\u002Fpytorch-native-architecture-optimization\u002F?utm_content=309679619&utm_medium=social&utm_source=twitter&hss_channel=tw-776585502606721024)  \n- [TorchServe](https:\u002F\u002Fgithub.com\u002Fpytorch\u002Fserve) [Great Tool]    \n- [TorchServe Video Tutorial](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=XlO7iQMV3Ik)\n- [LitServe: Easily serve AI models Lightning fast](https:\u002F\u002Fgithub.com\u002FLightning-AI\u002Flitserve)  \n\n## Convert PyTorch Models to C++:\n- [**Loading a PyTorch Model in C++**](https:\u002F\u002Fpytorch.org\u002Ftutorials\u002Fadvanced\u002Fcpp_export.html) [_**Fantastic**_]\n- [**PyTorch C++ API**](https:\u002F\u002Fpytorch.org\u002Fcppdocs\u002Findex.html) [_Bravo_]\n- [An Introduction To Torch (Pytorch) C++ Front-End](https:\u002F\u002Fradicalrafi.github.io\u002Fposts\u002Fpytorch-cpp-intro\u002F) [_Very Good_]\n- [Blogs on using PyTorch C++ API](https:\u002F\u002Fdiscuss.pytorch.org\u002Ft\u002Fa-series-of-blogs-on-pytorch-c-api-transfer-learning-jupyter-notebook-with-libtorch-xeus-cling-and-more\u002F54628) [_Good_]\n- [ATen: A TENsor library](https:\u002F\u002Fgithub.com\u002Fpytorch\u002Fpytorch\u002Ftree\u002Fmaster\u002Faten)\n- [Important Issue about PyTorch-like C++ interface](https:\u002F\u002Fgithub.com\u002Fpytorch\u002Fpytorch\u002Fissues\u002F3335)\n- [PyTorch C++ API Test](https:\u002F\u002Fgithub.com\u002Fpytorch\u002Fpytorch\u002Ftree\u002Fmaster\u002Ftest\u002Fcpp\u002Fapi)\n- [PyTorch via C++](https:\u002F\u002Fdiscuss.pytorch.org\u002Ft\u002Fpytorch-via-c\u002F19234) [_Useful Notes_]\n- [AUTOGRADPP](https:\u002F\u002Fgithub.com\u002Fpytorch\u002Fpytorch\u002Ftree\u002Fmaster\u002Ftorch\u002Fcsrc\u002Fapi)\n- [PyTorch C++ Library](https:\u002F\u002Fgithub.com\u002Fwarmspringwinds\u002Fpytorch-cpp)\n- [Direct C++ Interface to PyTorch](https:\u002F\u002Fgithub.com\u002Febetica\u002Fautogradpp)\n- [A Python module for compiling PyTorch graphs to C](https:\u002F\u002Fgithub.com\u002Flantiga\u002Fpytorch2c)\n\n## Deploy TensorFlow Models in Production:\n- [How to deploy Machine Learning models with TensorFlow - _Part1_](https:\u002F\u002Ftowardsdatascience.com\u002Fhow-to-deploy-machine-learning-models-with-tensorflow-part-1-make-your-model-ready-for-serving-776a14ec3198)\n- [How to deploy Machine Learning models with TensorFlow - _Part2_](https:\u002F\u002Ftowardsdatascience.com\u002Fhow-to-deploy-machine-learning-models-with-tensorflow-part-2-containerize-it-db0ad7ca35a7)\n- [How to deploy Machine Learning models with TensorFlow - _Part3_](https:\u002F\u002Ftowardsdatascience.com\u002Fhow-to-deploy-machine-learning-models-with-tensorflow-part-3-into-the-cloud-7115ff774bb6)\n- [Neural Structured Learning (NSL) in TensorFlow](https:\u002F\u002Fmedium.com\u002Ftensorflow\u002Fintroducing-neural-structured-learning-in-tensorflow-5a802efd7afd) [_Great_]\n- [Building Robust Production-Ready Deep Learning Vision Models](https:\u002F\u002Fmedium.com\u002Fgoogle-developer-experts\u002Fbuilding-robust-production-ready-deep-learning-vision-models-in-minutes-acd716f6450a)  \n- [Creating REST API for TensorFlow models](https:\u002F\u002Fbecominghuman.ai\u002Fcreating-restful-api-to-tensorflow-models-c5c57b692c10)\n- [\"How to Deploy a Tensorflow Model in Production\" by _Siraj Raval_ on YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=T_afaArR0E8)\n- [Code for the \"How to Deploy a Tensorflow Model in Production\" by _Siraj Raval_ on YouTube](https:\u002F\u002Fgithub.com\u002FllSourcell\u002FHow-to-Deploy-a-Tensorflow-Model-in-Production)\n- [How to deploy an Object Detection Model with TensorFlow serving](https:\u002F\u002Fmedium.freecodecamp.org\u002Fhow-to-deploy-an-object-detection-model-with-tensorflow-serving-d6436e65d1d9) [_Very Good Tutorial_]\n- [Freeze Tensorflow models and serve on web](http:\u002F\u002Fcv-tricks.com\u002Fhow-to\u002Ffreeze-tensorflow-models\u002F) [_Very Good Tutorial_]\n- [How to deploy TensorFlow models to production using TF Serving](https:\u002F\u002Fmedium.freecodecamp.org\u002Fhow-to-deploy-tensorflow-models-to-production-using-tf-serving-4b4b78d41700) [_Good_]\n- [How Zendesk Serves TensorFlow Models in Production](https:\u002F\u002Fmedium.com\u002Fzendesk-engineering\u002Fhow-zendesk-serves-tensorflow-models-in-production-751ee22f0f4b)\n- [TensorFlow Serving Example Projects](https:\u002F\u002Fgithub.com\u002FVetal1977\u002Ftf_serving_example)\n- [Serving Models in Production with TensorFlow Serving](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=q_IkJcPyNl0) [_TensorFlow Dev Summit 2017 Video_]\n- [Building TensorFlow as a Standalone Project](https:\u002F\u002Ftuatini.me\u002Fbuilding-tensorflow-as-a-standalone-project\u002F)\n- [TensorFlow C++ API Example](https:\u002F\u002Fgithub.com\u002Fjhjin\u002Ftensorflow-cpp)\n- [TensorFlow.js](https:\u002F\u002Fjs.tensorflow.org\u002F)\n- [Introducing TensorFlow.js: Machine Learning in Javascript](https:\u002F\u002Fmedium.com\u002Ftensorflow\u002Fintroducing-tensorflow-js-machine-learning-in-javascript-bf3eab376db)\n\n## Convert Keras Models in Production:\n- [Deep learning in production with Keras, Redis, Flask, and Apache](https:\u002F\u002Fwww.pyimagesearch.com\u002F2018\u002F02\u002F05\u002Fdeep-learning-production-keras-redis-flask-apache\u002F) [_Rank: 1st & General Usefult Tutorial_]\n- [Deploying a Keras Deep Learning Model as a Web Application in Python](https:\u002F\u002Ftowardsdatascience.com\u002Fdeploying-a-keras-deep-learning-model-as-a-web-application-in-p-fc0f2354a7ff) [_Very Good_]\n- [Deploying a Python Web App on AWS](https:\u002F\u002Ftowardsdatascience.com\u002Fdeploying-a-python-web-app-on-aws-57ed772b2319) [_Very Good_]\n- [Deploying Deep Learning Models Part 1: Preparing the Model](https:\u002F\u002Fblog.paperspace.com\u002Fdeploying-deep-learning-models-flask-web-python\u002F)  \n- [Deploying your Keras model](https:\u002F\u002Fmedium.com\u002F@burgalon\u002Fdeploying-your-keras-model-35648f9dc5fb)\n- [Deploying your Keras model using Keras.JS](https:\u002F\u002Fbecominghuman.ai\u002Fdeploying-your-keras-model-using-keras-js-2e5a29589ad8)\n- [\"How to Deploy a Keras Model to Production\" by _Siraj Raval_ on Youtube](https:\u002F\u002Fgithub.com\u002FllSourcell\u002Fhow_to_deploy_a_keras_model_to_production)\n- [Deploy Keras Model with Flask as Web App in 10 Minutes](https:\u002F\u002Fgithub.com\u002Fmtobeiyf\u002Fkeras-flask-deploy-webapp) [Good Repository]\n- [Deploying Keras Deep Learning Models with Flask](https:\u002F\u002Ftowardsdatascience.com\u002Fdeploying-keras-deep-learning-models-with-flask-5da4181436a2)\n- [keras2cpp](https:\u002F\u002Fgithub.com\u002Fpplonski\u002Fkeras2cpp)\n\n## Deploy MXNet Models in Production:\n- [Model Server for Apache MXNet](https:\u002F\u002Fgithub.com\u002Fawslabs\u002Fmxnet-model-server)\n- [Running the Model Server](https:\u002F\u002Fgithub.com\u002Fawslabs\u002Fmxnet-model-server\u002Fblob\u002Fmaster\u002Fdocs\u002Fserver.md)\n- [Multi Model Server (MMS) Documentation](https:\u002F\u002Fgithub.com\u002Fawslabs\u002Fmulti-model-server\u002Ftree\u002Fmaster\u002Fdocs)  \n- [Introducing Model Server for Apache MXNet](https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fmachine-learning\u002Fintroducing-model-server-for-apache-mxnet\u002F)  \n- [Single Shot Multi Object Detection Inference Service](https:\u002F\u002Fgithub.com\u002Fawslabs\u002Fmxnet-model-server\u002Fblob\u002Fmaster\u002Fexamples\u002Fssd\u002FREADME.md)\n- [Amazon SageMaker](https:\u002F\u002Faws.amazon.com\u002Fsagemaker\u002F)\n- [How can we serve MXNet models built with gluon api](https:\u002F\u002Fdiscuss.mxnet.io\u002Ft\u002Fhow-can-we-serve-mxnet-models-built-with-gluon-api\u002F684)\n- [MXNet C++ Package](https:\u002F\u002Fgithub.com\u002Fapache\u002Fincubator-mxnet\u002Ftree\u002Fmaster\u002Fcpp-package)\n- [MXNet C++ Package Examples](https:\u002F\u002Fgithub.com\u002Fapache\u002Fincubator-mxnet\u002Ftree\u002Fmaster\u002Fcpp-package\u002Fexample)\n- [MXNet Image Classification Example of C++](https:\u002F\u002Fgithub.com\u002Fapache\u002Fincubator-mxnet\u002Ftree\u002Fmaster\u002Fexample\u002Fimage-classification\u002Fpredict-cpp)\n- [MXNet C++ Tutorial](http:\u002F\u002Fmxnet.incubator.apache.org\u002Ftutorials\u002Fc%2B%2B\u002Fbasics.html)\n- [An introduction to the MXNet API](https:\u002F\u002Fbecominghuman.ai\u002Fan-introduction-to-the-mxnet-api-part-1-848febdcf8ab) [Very Good Tutorial for Learning MXNet]\n- [GluonCV](https:\u002F\u002Fgluon-cv.mxnet.io\u002F)\n- [GluonNLP](http:\u002F\u002Fgluon-nlp.mxnet.io\u002F)\n- [Model Quantization for Production-Level Neural Network Inference](https:\u002F\u002Fmedium.com\u002Fapache-mxnet\u002Fmodel-quantization-for-production-level-neural-network-inference-f54462ebba05) [_Excellent_]\n\n## Deploy Machine Learning Models with Go:\n- [Cortex: Deploy machine learning models in production](https:\u002F\u002Fgithub.com\u002Fcortexlabs\u002Fcortex)  \n- [Cortex - Main Page](https:\u002F\u002Fwww.cortex.dev\u002F)  \n- [Why we deploy machine learning models with Go — not Python](https:\u002F\u002Ftowardsdatascience.com\u002Fwhy-we-deploy-machine-learning-models-with-go-not-python-a4e35ec16deb)  \n- [Go-Torch](https:\u002F\u002Fgithub.com\u002Forktes\u002Fgo-torch)  \n- [Gotch - Go API for PyTorch](https:\u002F\u002Fgithub.com\u002Fsugarme\u002Fgotch)  \n- [TensorFlow Go Lang](https:\u002F\u002Fwww.tensorflow.org\u002Finstall\u002Flang_go)  \n- [Go-onnx](https:\u002F\u002Fgithub.com\u002Fdhdanie\u002Fgoonnx)  \n\n## General Deep Learning Deployment Toolkits:\n- [OpenVINO Toolkit - Deep Learning Deployment Toolkit repository](https:\u002F\u002Fgithub.com\u002Fopenvinotoolkit\u002Fopenvino) [_Great_]   \n- [ClearML - ML\u002FDL development and production suite](https:\u002F\u002Fgithub.com\u002Fallegroai\u002Fclearml)  \n- [Model Deployment Using Heroku: A Complete Guide on Heroku](https:\u002F\u002Fwww.analyticsvidhya.com\u002Fblog\u002F2021\u002F10\u002Fa-complete-guide-on-machine-learning-model-deployment-using-heroku\u002F) [Good]   \n- [NVIDIA Triton Inference Server](https:\u002F\u002Fdeveloper.nvidia.com\u002Fnvidia-triton-inference-server) [**Great**]      \n- [NVIDIA Triton Inference Server - GitHub](https:\u002F\u002Fgithub.com\u002Ftriton-inference-server\u002Fserver) [**Great**]   \n- [Cohere Boosts Inference Speed With NVIDIA Triton Inference Server](https:\u002F\u002Ftxt.cohere.ai\u002Fnvidia-boosts-inference-speed-with-cohere\u002F)  \n- [NVIDIA Deep Learning Examples for Tensor Cores](https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FDeepLearningExamples) [Interesting]  \n- [Deploying the Jasper Inference model using Triton Inference Server](https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FDeepLearningExamples\u002Ftree\u002Fmaster\u002FPyTorch\u002FSpeechRecognition\u002FJasper\u002Ftriton) [Useful]   \n- [Nvidia MLOPs Course via Triton](https:\u002F\u002Fanalyticsindiamag.com\u002Fnvidia-is-offering-a-four-hour-self-paced-course-on-mlops\u002F)  \n- [Awesome Production Machine Learning](https:\u002F\u002Fgithub.com\u002FEthicalML\u002Fawesome-production-machine-learning) [Great]\n- [BentoML Model Serving](https:\u002F\u002Fwww.linkedin.com\u002Fposts\u002Faxelmdz_mlops-machinelearning-datascience-activity-7194637600861618176-iQaF?utm_source=share&utm_medium=member_desktop)  \n\n## Huawei Deep Learning Framework:\n- [MindSpore - Huawei Deep Learning Framework](https:\u002F\u002Fgithub.com\u002Fmindspore-ai\u002Fmindspore)  \n- [MindSpore - Tutorial](https:\u002F\u002Fwww.mindspore.cn\u002Ftutorial\u002Fen\u002F0.1.0-alpha\u002Fquick_start\u002Fquick_start.html)  \n\n## General Deep Learning Compiler Stack:\n- [TVM Stack](https:\u002F\u002Ftvm.ai\u002F)\n\n## Model Conversion between Deep Learning Frameworks:\n- [ONNX (Open Neural Network Exchange)](https:\u002F\u002Fonnx.ai\u002F)\n- [Tutorials for using ONNX](https:\u002F\u002Fgithub.com\u002Fonnx\u002Ftutorials)\n- [MMdnn](https:\u002F\u002Fgithub.com\u002FMicrosoft\u002FMMdnn) [_Fantastic_]  \n- [Convert Full ImageNet Pre-trained Model from MXNet to PyTorch](https:\u002F\u002Fblog.paperspace.com\u002Fconvert-full-imagenet-pre-trained-model-from-mxnet-to-pytorch\u002F) [_Fantastic_, & Full ImageNet model means the model trained on ~ 14M images] \n\n## Some Caffe2 Tutorials:\n- [Mnist using caffe2](http:\u002F\u002Fvast.uccs.edu\u002F~adhamija\u002Fblog\u002FMNIST_singleGPU.html)\n- [Caffe2 C++ Tutorials and Examples](https:\u002F\u002Fgithub.com\u002Fleonardvandriel\u002Fcaffe2_cpp_tutorial)\n- [Make Transfer Learning of SqueezeNet on Caffe2](https:\u002F\u002Fmedium.com\u002F@KazamiXHayato\u002Fmake-transfer-learning-in-caffe2-21d96c47ba0e)\n- [Build Basic program by using Caffe2 framework in C++](https:\u002F\u002Fmedium.com\u002F@KazamiXHayato\u002Fwrite-caffe2-program-in-c-5519e2646382)\n\n## Some Useful Resources for Designing UI (Front-End Development):\n- [ReactJS vs Angular5 vs Vue.js](https:\u002F\u002Fmedium.com\u002F@TechMagic\u002Freactjs-vs-angular5-vs-vue-js-what-to-choose-in-2018-b91e028fa91d)\n- [A comparison between Angular and React and their core languages](https:\u002F\u002Fmedium.freecodecamp.org\u002Fa-comparison-between-angular-and-react-and-their-core-languages-9de52f485a76)\n- [A Guide to Becoming a Full-Stack Developer](https:\u002F\u002Fmedium.com\u002Fcoderbyte\u002Fa-guide-to-becoming-a-full-stack-developer-in-2017-5c3c08a1600c) [_Very Good Tutorial_]\n- [Roadmap to becoming a web developer in 2018](https:\u002F\u002Fgithub.com\u002Fkamranahmedse\u002Fdeveloper-roadmap) [_Very Good Repository_]\n- [Modern Frontend Developer in 2018](https:\u002F\u002Fmedium.com\u002Ftech-tajawal\u002Fmodern-frontend-developer-in-2018-4c2072fa2b9c)\n- [Roadmap to becoming a React developer in 2018](https:\u002F\u002Fgithub.com\u002Fadam-golab\u002Freact-developer-roadmap)\n- [2019 UI and UX Design Trends](https:\u002F\u002Fuxplanet.org\u002F2019-ui-and-ux-design-trends-92dfa8323225) [_Good_]\n- [Streamlit](https:\u002F\u002Fstreamlit.io\u002F) [_The fastest way to build custom ML tools_]  \n- [Gradio](https:\u002F\u002Fwww.gradio.app\u002F) [**Good**]   \n- [Web Developer Monthly](https:\u002F\u002Fmedium.com\u002F@andreineagoie\u002Fweb-developer-monthly-july-2018-513e02f15fb6)\n- [23 Best React UI Component Frameworks](https:\u002F\u002Fhackernoon.com\u002F23-best-react-ui-component-libraries-and-frameworks-250a81b2ac42)\n- [9 React Styled-Components UI Libraries for 2018](https:\u002F\u002Fblog.bitsrc.io\u002F9-react-styled-components-ui-libraries-for-2018-4e1a0bd3e179)\n- [35 New Tools for UI Design](https:\u002F\u002Fblog.prototypr.io\u002F35-new-tools-for-ui-design-412cf1d701fd)\n- [5 Tools To Speed Up Your App Development](https:\u002F\u002Fmedium.com\u002Fswlh\u002F5-tools-to-speed-up-your-app-development-6979d0e49e34) [_Very Good_]\n- [How to use ReactJS with Webpack 4, Babel 7, and Material Design](https:\u002F\u002Fmedium.freecodecamp.org\u002Fhow-to-use-reactjs-with-webpack-4-babel-7-and-material-design-ff754586f618)\n- [Adobe Typekit](https:\u002F\u002Ftypekit.com\u002F) [_Great fonts, where you need them_]\n- [Build A Real World Beautiful Web APP with Angular 6](https:\u002F\u002Fmedium.com\u002F@hamedbaatour\u002Fbuild-a-real-world-beautiful-web-app-with-angular-6-a-to-z-ultimate-guide-2018-part-i-e121dd1d55e)\n- [You Don't Know JS](https:\u002F\u002Fgithub.com\u002Fgetify\u002FYou-Dont-Know-JS)\n- [JavaScript Top 10 Articles](https:\u002F\u002Fmedium.mybridge.co\u002Fjavascript-top-10-articles-for-the-past-month-v-sep-2018-8f27a300d6c5)\n- [Web Design with Adobe XD](https:\u002F\u002Fmedium.freecodecamp.org\u002Fa-developers-guide-to-web-design-for-non-designers-1f64ce28c38d)\n- [INSPINIA Bootstrap Web Theme](https:\u002F\u002Fwrapbootstrap.com\u002Ftheme\u002Finspinia-responsive-admin-theme-WB0R5L90S)\n- [A Learning Tracker for Front-End Developers](https:\u002F\u002Fgithub.com\u002FSyknapse\u002FMy-Learning-Tracker-first-ten-months)\n- [The best front-end hacking cheatsheets — all in one place](https:\u002F\u002Fmedium.freecodecamp.org\u002Fmodern-frontend-hacking-cheatsheets-df9c2566c72a) [_Useful & Interesting_]\n- [GUI-fying the Machine Learning Workflow (Machine Flow)](https:\u002F\u002Ftowardsdatascience.com\u002Fgui-fying-the-machine-learning-workflow-towards-rapid-discovery-of-viable-pipelines-cab2552c909f)\n- [Electron - Build cross platform desktop apps with JavaScript](https:\u002F\u002Felectronjs.org\u002F) [_Very Good_]\n- [Opyrator - Turns Python functions into microservices with web API](https:\u002F\u002Fgithub.com\u002Fml-tooling\u002Fopyrator) [**Great**]    \n- [A First Look at PyScript: Python in the Web Browser](https:\u002F\u002Frealpython.com\u002Fpyscript-python-in-browser\u002F) [**Interesting**]  \n\n## Mobile & Embedded Devices Development:\n- [PyTorch Mobile](https:\u002F\u002Fpytorch.org\u002Fmobile\u002Fhome\u002F) [_Excellent_]  \n- [Mobile UI Design Trends In 2018](https:\u002F\u002Fuxplanet.org\u002Fmobile-ui-design-trends-in-2018-ccd26031dfd8)  \n- [ncnn - high-performance neural network inference framework optimized for the mobile platform](https:\u002F\u002Fgithub.com\u002FTencent\u002Fncnn) [_Useful_]  \n- [Alibaba - MNN](https:\u002F\u002Fgithub.com\u002Falibaba\u002FMNN)  \n- [Awesome Mobile Machine Learning](https:\u002F\u002Fgithub.com\u002Ffritzlabs\u002FAwesome-Mobile-Machine-Learning)  \n- [EMDL - Embedded and Mobile Deep Learning](https:\u002F\u002Fgithub.com\u002FEMDL\u002Fawesome-emdl)  \n- [Fritz - machine learning platform for iOS and Android](https:\u002F\u002Fwww.fritz.ai\u002F)  \n- [TensorFlow Lite](https:\u002F\u002Fwww.tensorflow.org\u002Fmobile\u002Ftflite\u002F)  \n- [Tiny Machine Learning: The Next AI Revolution](https:\u002F\u002Fmedium.com\u002F@matthew_stewart\u002Ftiny-machine-learning-the-next-ai-revolution-495c26463868)  \n- [TLT - NVIDIA Transfer Learning Toolkit](https:\u002F\u002Fdeveloper.nvidia.com\u002Ftransfer-learning-toolkit)  \n- [NVIDIA Jetson Inference](https:\u002F\u002Fgithub.com\u002Fdusty-nv\u002Fjetson-inference)  [_Great_]\n- [EfficientML.ai Lecture (MIT Course)](https:\u002F\u002Fwww.youtube.com\u002Fplaylist?list=PL80kAHvQbh-pT4lCkDT53zT8DKmhE0idB)\n- [PyTorch Edge: Enabling On-Device Inference Across Mobile and Edge Devices with ExecuTorch](https:\u002F\u002Fpytorch.org\u002Fblog\u002Fpytorch-edge\u002F?utm_content=268296369&utm_medium=social&utm_source=linkedin&hss_channel=lcp-78618366)  \n\n## Back-End Development Part:\n- [Modern Backend Developer in 2018](https:\u002F\u002Fmedium.com\u002Ftech-tajawal\u002Fmodern-backend-developer-in-2018-6b3f7b5f8b9)\n- [Deploying frontend applications — the fun way](https:\u002F\u002Fhackernoon.com\u002Fdeploying-frontend-applications-the-fun-way-bc3f69e15331) [_Very Good_]\n- [RabbitMQ](https:\u002F\u002Fwww.rabbitmq.com\u002F) [_Message Broker Software_]\n- [Celery](http:\u002F\u002Fwww.celeryproject.org\u002F) [_Distributed Task Queue_]\n- [Kafka](https:\u002F\u002Fkafka.apache.org\u002F) [_Distributed Streaming Platform_]\n- [Docker training with DockerMe](https:\u002F\u002Fgithub.com\u002FAhmadRafiee\u002FDocker_training_with_DockerMe)  \n- [Kubernetes - GitHub](https:\u002F\u002Fgithub.com\u002Fkubernetes\u002Fkubernetes)\n- [Deploy Machine Learning Pipeline on Google Kubernetes Engine](https:\u002F\u002Ftowardsdatascience.com\u002Fdeploy-machine-learning-model-on-google-kubernetes-engine-94daac85108b)  \n- [An introduction to Kubernetes for Data Scientists](https:\u002F\u002Fwww.jeremyjordan.me\u002Fkubernetes\u002F)  \n- [Jenkins and Kubernetes with Docker Desktop](https:\u002F\u002Fmedium.com\u002F@garunski\u002Fjenkins-and-kubernetes-with-docker-desktop-53a853486f7c)\n- [Helm: The package manager for Kubernetes](https:\u002F\u002Fhelm.sh\u002F)  \n- [Create Cluster using docker swarm](https:\u002F\u002Fmedium.com\u002Ftech-tajawal\u002Fcreate-cluster-using-docker-swarm-94d7b2a10c43)  \n- [deepo - Docker Image for all DL Framewors](https:\u002F\u002Fgithub.com\u002Fufoym\u002Fdeepo)  \n- [Kubeflow](https:\u002F\u002Fwww.kubeflow.org\u002F)  [_deployments of ML workflows on Kubernetes_]  \n- [kubespray - Deploy a Production Ready Kubernetes Cluster](https:\u002F\u002Fgithub.com\u002Fkubernetes-sigs\u002Fkubespray)  \n- [KFServing - Kubernetes for Serving ML Models](https:\u002F\u002Fgithub.com\u002Fkubeflow\u002Fkfserving)  \n- [Deploying a HuggingFace NLP Model with KFServing](http:\u002F\u002Fwww.pattersonconsultingtn.com\u002Fblog\u002Fdeploying_huggingface_with_kfserving.html) [_Interesting_]  \n- [Seldon Core - Deploying Machine Learning Models on Kubernetes](https:\u002F\u002Fwww.seldon.io\u002Ftech\u002Fproducts\u002Fcore\u002F)  \n- [Seldon Core - GitHub](https:\u002F\u002Fgithub.com\u002FSeldonIO\u002Fseldon-core)  \n- [Machine Learning: serving models with Kubeflow on Ubuntu, Part 1](https:\u002F\u002Fubuntu.com\u002Fblog\u002Fml-serving-models-with-kubeflow-on-ubuntu-part-1)  \n- [CoreWeave Kubernetes Cloud](https:\u002F\u002Fgithub.com\u002Fcoreweave\u002Fkubernetes-cloud\u002Ftree\u002Fmaster\u002Fonline-inference\u002F)  \n- [MLOps References](https:\u002F\u002Fgithub.com\u002Fvisenger\u002Fmlops-references)  [_DevOps for ML_]\n- [Data Version Control - DVC](https:\u002F\u002Fdvc.org\u002F)  [_Great_]  \n- [MLEM: package and deploy machine learning models](https:\u002F\u002Fgithub.com\u002Fiterative\u002Fmlem)  \n- [PySyft - A library for encrypted, privacy preserving deep learning](https:\u002F\u002Fgithub.com\u002FOpenMined\u002FPySyft)  \n- [LocalStack - A fully functional local AWS cloud stack](https:\u002F\u002Fgithub.com\u002Flocalstack\u002Flocalstack)  \n- [poetry: Python packaging and dependency management](https:\u002F\u002Fpython-poetry.org\u002F)\n- [Netflix's Powerful Database Selections for Streaming at Scale](https:\u002F\u002Fwww.linkedin.com\u002Fposts\u002Fchandreshdesai_cloudcomputing-devops-softwareengineering-activity-7108414221482115072-HGil?utm_source=share&utm_medium=member_desktop)\n- [Ray: a unified framework for scaling AI and Python applications](https:\u002F\u002Fgithub.com\u002Fray-project\u002Fray)  \n\n## GPU Management Libraries:\n- [GPUtil](https:\u002F\u002Fgithub.com\u002Fanderskm\u002Fgputil)\n- [py3nvml](https:\u002F\u002Fgithub.com\u002Ffbcotter\u002Fpy3nvml) [_Python 3 binding to the NVIDIA Management Library_]\n- [PyCUDA - GitHub](https:\u002F\u002Fgithub.com\u002Finducer\u002Fpycuda)\n- [PyCUDA](https:\u002F\u002Fmathema.tician.de\u002Fsoftware\u002Fpycuda\u002F)\n- [PyCUDA Tutorial](https:\u002F\u002Fdocumen.tician.de\u002Fpycuda\u002F)\n- [setGPU](https:\u002F\u002Fgithub.com\u002Fbamos\u002FsetGPU)\n- [Monitor your GPUs](https:\u002F\u002Fgithub.com\u002Fmsalvaris\u002Fgpu_monitor) [**Excellent**]  \n- [GPU-Burn - Multi-GPU CUDA stress test](https:\u002F\u002Fgithub.com\u002Fwilicc\u002Fgpu-burn) [_Useful_]   \n- [Grafana - Monitoring and Observability](https:\u002F\u002Fgithub.com\u002Fgrafana\u002Fgrafana) [**Excellent**]  \n- [Prometheus](https:\u002F\u002Fprometheus.io\u002F) [_Excellent for monitoring solution & extract required metrics_]  \n- [OpenAI Triton: Open-Source GPU Programming for Neural Networks](https:\u002F\u002Fopenai.com\u002Fblog\u002Ftriton\u002F)  \n\n## Speed-up & Scalabale Python Codes:\n- [Numba - makes Python code fast](http:\u002F\u002Fnumba.pydata.org\u002F)\n- [Dask - natively scales Python](https:\u002F\u002Fdask.org\u002F)\n- [What is Dask](https:\u002F\u002Fmedium.com\u002Fbetter-programming\u002Fwhat-is-dask-and-how-can-it-help-you-as-a-data-scientist-72adec7cec57)  \n- [Ray - running distributed applications](https:\u002F\u002Fgithub.com\u002Fray-project\u002Fray)  \n- [Neural Network Distiller](https:\u002F\u002Fgithub.com\u002FNervanaSystems\u002Fdistiller\u002F) [_Distillation & Quantization of Deep Learning Models in PyTorch_]\n- [Autodistill](https:\u002F\u002Fgithub.com\u002Fautodistill\u002Fautodistill)  \n- [PyTorch Pruning Tutorial](https:\u002F\u002Fpytorch.org\u002Ftutorials\u002Fintermediate\u002Fpruning_tutorial.html)  \n- [Can you remove 99% of a neural network without losing accuracy? - An introduction to weight pruning](https:\u002F\u002Ftowardsdatascience.com\u002Fcan-you-remove-99-of-a-neural-network-without-losing-accuracy-915b1fab873b)  \n- [PocketFlow - An Automatic Model Compression (AutoMC) framework](https:\u002F\u002Fgithub.com\u002FTencent\u002FPocketFlow) [**Great**]  \n- [Introducing the Model Optimization Toolkit for TensorFlow](https:\u002F\u002Fmedium.com\u002Ftensorflow\u002Fintroducing-the-model-optimization-toolkit-for-tensorflow-254aca1ba0a3)  \n- [TensorFlow Model Optimization Toolkit — Post-Training Integer Quantization](https:\u002F\u002Fmedium.com\u002Ftensorflow\u002Ftensorflow-model-optimization-toolkit-post-training-integer-quantization-b4964a1ea9ba)  \n- [TensorFlow Post-training Quantization](https:\u002F\u002Fwww.tensorflow.org\u002Flite\u002Fperformance\u002Fpost_training_quantization)  \n- [Dynamic Quantization in PyTorch](https:\u002F\u002Fpytorch.org\u002Ftutorials\u002Fadvanced\u002Fdynamic_quantization_tutorial.html)  \n- [Static Quantization in PyTorch](https:\u002F\u002Fpytorch.org\u002Ftutorials\u002Fadvanced\u002Fstatic_quantization_tutorial.html)  \n- [NVIDIA DALI - highly optimized data pre-processing in deep learning](https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fdali)  \n- [Horovod - Distributed training framework](https:\u002F\u002Fgithub.com\u002Fhorovod\u002Fhorovod)  \n- [ONNX Float32 to Float16](https:\u002F\u002Fgithub.com\u002Fonnx\u002Fonnx-docker\u002Fblob\u002Fmaster\u002Fonnx-ecosystem\u002Fconverter_scripts\u002Ffloat32_float16_onnx.ipynb)  \n- [Speeding Up Deep Learning Inference Using TensorRT](https:\u002F\u002Fdevblogs.nvidia.com\u002Fspeeding-up-deep-learning-inference-using-tensorrt\u002F)  \n- [Speed up Training](https:\u002F\u002Fai.googleblog.com\u002F2020\u002F05\u002Fspeeding-up-neural-network-training.html)  \n- [Native PyTorch automatic mixed precision for faster training on NVIDIA GPUs](https:\u002F\u002Fpytorch.org\u002Fblog\u002Faccelerating-training-on-nvidia-gpus-with-pytorch-automatic-mixed-precision\u002F)  \n- [JAX - Composable transformations of Python+NumPy programs](https:\u002F\u002Fgithub.com\u002Fgoogle\u002Fjax)  \n- [TensorRTx - popular DL networks with tensorrt](https:\u002F\u002Fgithub.com\u002Fwang-xinyu\u002Ftensorrtx)  \n- [Speeding up Deep Learning Inference Using TensorFlow, ONNX, and TensorRT](https:\u002F\u002Fdevblogs.nvidia.com\u002Fspeeding-up-deep-learning-inference-using-tensorflow-onnx-and-tensorrt\u002F)  \n- [TensorRT Developer Guide](https:\u002F\u002Fdocs.nvidia.com\u002Fdeeplearning\u002Fsdk\u002Ftensorrt-developer-guide\u002Findex.html)  \n- [How to Convert a Model from PyTorch to TensorRT and Speed Up Inference](https:\u002F\u002Fwww.learnopencv.com\u002Fhow-to-convert-a-model-from-pytorch-to-tensorrt-and-speed-up-inference\u002F) [_Good_]\n- [cuDF: GPU DataFrame Library](https:\u002F\u002Fgithub.com\u002Frapidsai\u002Fcudf)\n- [cuDF example](https:\u002F\u002Fcolab.research.google.com\u002Fdrive\u002F12tCzP94zFG2BRduACucn5Q_OcX1TUKY3)  \n\n## Hardware Notes for Deep Learning:  \n- [Hardware for Deep Learning](https:\u002F\u002Fblog.inten.to\u002Fhardware-for-deep-learning-part-3-gpu-8906c1644664)  \n\n## MLOPs Courses & Resources:  \n- [MLOps-Basics](https:\u002F\u002Fgithub.com\u002Fgraviraja\u002FMLOps-Basics) [Great]  \n- [MLOPs-Zoomcamp](https:\u002F\u002Fgithub.com\u002FDataTalksClub\u002Fmlops-zoomcamp) [Great]   \n- [A collection of resources to learn about MLOPs](https:\u002F\u002Fgithub.com\u002Fdair-ai\u002FMLOPs-Primer) [Great]  \n- [Awesome MLOPs](https:\u002F\u002Fgithub.com\u002Fvisenger\u002Fawesome-mlops) [Great]  \n- [Data Science Topics & MLOPs](https:\u002F\u002Fgithub.com\u002Fkhuyentran1401\u002FData-science#mlops) [Great]  \n- [MLEM: package and deploy machine learning models](https:\u002F\u002Fgithub.com\u002Fiterative\u002Fmlem)  \n- [DevOps Exercises](https:\u002F\u002Fgithub.com\u002Fbregman-arie\u002Fdevops-exercises)  \n- [MlOPs Sample Project](https:\u002F\u002Fgithub.com\u002FAntonisCSt\u002FMlops_project_semicon)  \n- [prefect: Orchestrate and observe all of your workflows](https:\u002F\u002Fwww.prefect.io\u002F)  \n- [DataTalks Club: The place to talk about data](https:\u002F\u002Fdatatalks.club\u002F)  \n- [OpenNMT CTranslate2: Fast inference engine for Transformer models](https:\u002F\u002Fgithub.com\u002FOpenNMT\u002FCTranslate2\u002F)  \n\n## Other:\n- [A Guide to Production Level Deep Learning](https:\u002F\u002Fgithub.com\u002Falirezadir\u002FProduction-Level-Deep-Learning)  \n- [Facebook Says Developers Will Love PyTorch 1.0](https:\u002F\u002Fmedium.com\u002Fsyncedreview\u002Ffacebook-says-developers-will-love-pytorch-1-0-ba2f89ebc9cc)\n- [Some PyTorch Workflow Changes](https:\u002F\u002Fgithub.com\u002Fpytorch\u002Fpytorch\u002Fissues\u002F6032)\n- [wandb - A tool for visualizing and tracking your machine learning experiments](https:\u002F\u002Fgithub.com\u002Fwandb\u002Fclient)  \n- [PyTorch and Caffe2 repos getting closer together](https:\u002F\u002Fgithub.com\u002Fcaffe2\u002Fcaffe2\u002Fissues\u002F2439#issuecomment-391155017)\n- [PyTorch or TensorFlow?](https:\u002F\u002Fawni.github.io\u002Fpytorch-tensorflow\u002F)\n- [Choosing a Deep Learning Framework in 2018: Tensorflow or Pytorch?](http:\u002F\u002Fcv-tricks.com\u002Fdeep-learning-2\u002Ftensorflow-or-pytorch\u002F)\n- [Deep Learning War between PyTorch & TensorFlow](https:\u002F\u002Fhub.packtpub.com\u002Fcan-a-production-ready-pytorch-1-0-give-tensorflow-a-tough-time\u002F)\n- [Embedding Machine Learning Models to Web Apps (Part-1)](https:\u002F\u002Ftowardsdatascience.com\u002Fembedding-machine-learning-models-to-web-apps-part-1-6ab7b55ee428)\n- [Deploying deep learning models: Part 1 an overview](https:\u002F\u002Ftowardsdatascience.com\u002Fdeploying-deep-learning-models-part-1-an-overview-77b4d01dd6f7)\n- [Machine Learning in Production](https:\u002F\u002Fmedium.com\u002Fcontentsquare-engineering-blog\u002Fmachine-learning-in-production-c53b43283ab1)\n- [how you can get a 2–6x speed-up on your data pre-processing with Python](https:\u002F\u002Ftowardsdatascience.com\u002Fheres-how-you-can-get-a-2-6x-speed-up-on-your-data-pre-processing-with-python-847887e63be5)\n- [Making your C library callable from Python](https:\u002F\u002Fmedium.com\u002F@shamir.stav_83310\u002Fmaking-your-c-library-callable-from-python-by-wrapping-it-with-cython-b09db35012a3)\n- [MIL WebDNN](https:\u002F\u002Fmil-tokyo.github.io\u002Fwebdnn\u002F)\n- [Multi-GPU Framework Comparisons](https:\u002F\u002Fmedium.com\u002F@iliakarmanov\u002Fmulti-gpu-rosetta-stone-d4fa96162986) [_Great_]  \n\n","# 生产环境中的深度学习\n在这个仓库中，我将分享一些关于在生产环境中部署基于深度学习的模型的实用笔记和参考资料。\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fahkarami_Deep-Learning-in-Production_readme_7c03a4d80257.jpg\" alt=\"Logo\"\u002F>\n\u003C\u002Fp>\n\n## 将 PyTorch 模型转换为生产级：\n- [PyTorch 生产级教程](https:\u002F\u002Fpytorch.org\u002Ftutorials\u002F#production-usage) [_非常棒_]  \n- [通往 1.0 的道路：生产就绪的 PyTorch](https:\u002F\u002Fpytorch.org\u002F2018\u002F05\u002F02\u002Froad-to-1.0.html)\n- [PyTorch 1.0 追踪 JIT 和 LibTorch C++ API，用于将 PyTorch 集成到 NodeJS 中](http:\u002F\u002Fblog.christianperone.com\u002F2018\u002F10\u002Fpytorch-1-0-tracing-jit-and-libtorch-c-api-to-integrate-pytorch-into-nodejs\u002F) [_好文章_]\n- [PyTorch 中的模型服务](https:\u002F\u002Fpytorch.org\u002Fblog\u002Fmodel-serving-in-pyorch\u002F)\n- [PyTorch 夏季黑客马拉松](https:\u002F\u002Fpytorch.devpost.com\u002F) [_非常重要_]\n- [使用 Flask 部署 PyTorch 并构建 REST API](https:\u002F\u002Fpytorch.org\u002Ftutorials\u002Fintermediate\u002Fflask_rest_api_tutorial.html) [_重要_]\n- [在 Flask 上部署的识别热狗与非热狗的 PyTorch 模型](https:\u002F\u002Fgithub.com\u002Fjaroslaw-weber\u002Fhotdog-not-hotdog)\n- [以 C++ 作为 Web 服务器来服务 PyTorch 1.0 模型](https:\u002F\u002Fgithub.com\u002FWizaron\u002Fpytorch-cpp-inference) [_有用的示例_]\n- [PyTorch 内部机制](http:\u002F\u002Fblog.ezyang.com\u002F2019\u002F05\u002Fpytorch-internals\u002F)  [_有趣且有用的文章_]  \n- [支持 PyTorch 模型预测的 Flask 应用程序](https:\u002F\u002Fgithub.com\u002Fcraigsidcarlson\u002FPytorchFlaskApp)\n- [在 Flask 上安全地服务 PyTorch 模型](https:\u002F\u002Fdiscuss.pytorch.org\u002Ft\u002Fserving-pytorch-model-on-flask-thread-safety\u002F13921)\n- [使用 Caffe2 和 ONNX 在 AWS Lambda 上服务 PyTorch 模型](https:\u002F\u002Fmachinelearnings.co\u002Fserving-pytorch-models-on-aws-lambda-with-caffe2-onnx-7b096806cfac)\n- [使用 Caffe2 和 ONNX 在 AWS Lambda 上服务 PyTorch 模型（另一个版本）](https:\u002F\u002Fblog.waya.ai\u002Fdeploy-deep-machine-learning-in-production-the-pythonic-way-a17105f1540e)\n- [深入 ONNX Runtime](https:\u002F\u002Fmedium.com\u002F@mohsen.mahmoodzadeh\u002Fa-deep-dive-into-onnx-onnx-runtime-part-1-874517c66ffc)  \n- [EuclidesDB - _多模型机器学习特征数据库，支持 PyTorch_](https:\u002F\u002Feuclidesdb.readthedocs.io\u002Fen\u002Flatest\u002F)\n- [EuclidesDB - GitHub](https:\u002F\u002Fgithub.com\u002Fperone\u002Feuclidesdb\u002F)\n- [WebDNN：Web 浏览器上最快的 DNN 执行框架](https:\u002F\u002Fgithub.com\u002Fmil-tokyo\u002Fwebdnn)\n- [FastAI PyTorch 无服务器 API（使用 AWS Lambda）](https:\u002F\u002Fgithub.com\u002Falecrubin\u002Fpytorch-serverless\u002F)\n- [FastAI PyTorch 在生产中的讨论](http:\u002F\u002Fforums.fast.ai\u002Ft\u002Ffastai-pytorch-in-production\u002F16928)   \n- [OpenMMLab 模型部署框架](https:\u002F\u002Fgithub.com\u002Fopen-mmlab\u002Fmmdeploy)\n- [PyTorch 原生架构优化：torchao](https:\u002F\u002Fpytorch.org\u002Fblog\u002Fpytorch-native-architecture-optimization\u002F?utm_content=309679619&utm_medium=social&utm_source=twitter&hss_channel=tw-776585502606721024)  \n- [TorchServe](https:\u002F\u002Fgithub.com\u002Fpytorch\u002Fserve) [很棒的工具]    \n- [TorchServe 视频教程](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=XlO7iQMV3Ik)\n- [LitServe：闪电般快速地轻松服务 AI 模型](https:\u002F\u002Fgithub.com\u002FLightning-AI\u002Flitserve)  \n\n## 将 PyTorch 模型转换为 C++：\n- [**在 C++ 中加载 PyTorch 模型**](https:\u002F\u002Fpytorch.org\u002Ftutorials\u002Fadvanced\u002Fcpp_export.html) [_**非常棒**_]  \n- [**PyTorch C++ API**](https:\u002F\u002Fpytorch.org\u002Fcppdocs\u002Findex.html) [_太棒了_]  \n- [Torch（PyTorch）C++ 前端简介](https:\u002F\u002Fradicalrafi.github.io\u002Fposts\u002Fpytorch-cpp-intro\u002F) [_非常好_]  \n- [关于使用 PyTorch C++ API 的博客](https:\u002F\u002Fdiscuss.pytorch.org\u002Ft\u002Fa-series-of-blogs-on-pytorch-c-api-transfer-learning-jupyter-notebook-with-libtorch-xeus-cling-and-more\u002F54628) [_不错_]  \n- [ATen：一个张量库](https:\u002F\u002Fgithub.com\u002Fpytorch\u002Fpytorch\u002Ftree\u002Fmaster\u002Faten)\n- [关于类似 PyTorch 的 C++ 接口的重要问题](https:\u002F\u002Fgithub.com\u002Fpytorch\u002Fpytorch\u002Fissues\u002F3335)\n- [PyTorch C++ API 测试](https:\u002F\u002Fgithub.com\u002Fpytorch\u002Fpytorch\u002Ftree\u002Fmaster\u002Ftest\u002Fcpp\u002Fapi)\n- [通过 C++ 使用 PyTorch](https:\u002F\u002Fdiscuss.pytorch.org\u002Ft\u002Fpytorch-via-c\u002F19234) [_有用的笔记_]\n- [AUTOGRADPP](https:\u002F\u002Fgithub.com\u002Fpytorch\u002Fpytorch\u002Ftree\u002Fmaster\u002Ftorch\u002Fcsrc\u002Fapi)\n- [PyTorch C++ 库](https:\u002F\u002Fgithub.com\u002Fwarmspringwinds\u002Fpytorch-cpp)\n- [PyTorch 的直接 C++ 接口](https:\u002F\u002Fgithub.com\u002Febetica\u002Fautogradpp)\n- [用于将 PyTorch 图编译为 C 的 Python 模块](https:\u002F\u002Fgithub.com\u002Flantiga\u002Fpytorch2c)\n\n## 将 TensorFlow 模型部署到生产环境：\n- [如何使用 TensorFlow 部署机器学习模型 - _第1部分_](https:\u002F\u002Ftowardsdatascience.com\u002Fhow-to-deploy-machine-learning-models-with-tensorflow-part-1-make-your-model-ready-for-serving-776a14ec3198)\n- [如何使用 TensorFlow 部署机器学习模型 - _第2部分_](https:\u002F\u002Ftowardsdatascience.com\u002Fhow-to-deploy-machine-learning-models-with-tensorflow-part-2-containerize-it-db0ad7ca35a7)\n- [如何使用 TensorFlow 部署机器学习模型 - _第3部分_](https:\u002F\u002Ftowardsdatascience.com\u002Fhow-to-deploy-machine-learning-models-with-tensorflow-part-3-into-the-cloud-7115ff774bb6)\n- [TensorFlow 中的神经结构化学习 (NSL)](https:\u002F\u002Fmedium.com\u002Ftensorflow\u002Fintroducing-neural-structured-learning-in-tensorflow-5a802efd7afd) [_很棒_]  \n- [构建健壮的生产就绪深度学习视觉模型](https:\u002F\u002Fmedium.com\u002Fgoogle-developer-experts\u002Fbuilding-robust-production-ready-deep-learning-vision-models-in-minutes-acd716f6450a)  \n- [为 TensorFlow 模型创建 REST API](https:\u002F\u002Fbecominghuman.ai\u002Fcreating-restful-api-to-tensorflow-models-c5c57b692c10)\n- [Siraj Raval 在 YouTube 上讲解的“如何在生产环境中部署 TensorFlow 模型”](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=T_afaArR0E8)\n- [Siraj Raval 在 YouTube 上讲解的“如何在生产环境中部署 TensorFlow 模型”的代码](https:\u002F\u002Fgithub.com\u002FllSourcell\u002FHow-to-Deploy-a-Tensorflow-Model-in-Production)\n- [如何使用 TensorFlow Serving 部署目标检测模型](https:\u002F\u002Fmedium.freecodecamp.org\u002Fhow-to-deploy-an-object-detection-model-with-tensorflow-serving-d6436e65d1d9) [_非常好的教程_]  \n- [冻结 TensorFlow 模型并在网页上提供服务](http:\u002F\u002Fcv-tricks.com\u002Fhow-to\u002Ffreeze-tensorflow-models\u002F) [_非常好的教程_]  \n- [如何使用 TF Serving 将 TensorFlow 模型部署到生产环境](https:\u002F\u002Fmedium.freecodecamp.org\u002Fhow-to-deploy-tensorflow-models-to-production-using-tf-serving-4b4b78d41700) [_不错_]  \n- [Zendesk 如何在生产环境中服务 TensorFlow 模型](https:\u002F\u002Fmedium.com\u002Fzendesk-engineering\u002Fhow-zendesk-serves-tensorflow-models-in-production-751ee22f0f4b)\n- [TensorFlow Serving 示例项目](https:\u002F\u002Fgithub.com\u002FVetal1977\u002Ftf_serving_example)\n- [使用 TensorFlow Serving 在生产环境中服务模型](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=q_IkJcPyNl0) [_2017 年 TensorFlow 开发者峰会视频_]\n- [将 TensorFlow 构建为独立项目](https:\u002F\u002Ftuatini.me\u002Fbuilding-tensorflow-as-a-standalone-project\u002F)\n- [TensorFlow C++ API 示例](https:\u002F\u002Fgithub.com\u002Fjhjin\u002Ftensorflow-cpp)\n- [TensorFlow.js](https:\u002F\u002Fjs.tensorflow.org\u002F)\n- [介绍 TensorFlow.js：JavaScript 中的机器学习](https:\u002F\u002Fmedium.com\u002Ftensorflow\u002Fintroducing-tensorflow-js-machine-learning-in-javascript-bf3eab376db)\n\n## 将 Keras 模型部署到生产环境：\n- [使用 Keras、Redis、Flask 和 Apache 进行深度学习生产部署](https:\u002F\u002Fwww.pyimagesearch.com\u002F2018\u002F02\u002F05\u002Fdeep-learning-production-keras-redis-flask-apache\u002F) [_排名第一且通用实用教程_]\n- [在 Python 中将 Keras 深度学习模型部署为 Web 应用程序](https:\u002F\u002Ftowardsdatascience.com\u002Fdeploying-a-keras-deep-learning-model-as-a-web-application-in-p-fc0f2354a7ff) [_非常好_]\n- [在 AWS 上部署 Python Web 应用程序](https:\u002F\u002Ftowardsdatascience.com\u002Fdeploying-a-python-web-app-on-aws-57ed772b2319) [_非常好_]\n- [部署深度学习模型第 1 部分：准备模型](https:\u002F\u002Fblog.paperspace.com\u002Fdeploying-deep-learning-models-flask-web-python\u002F)\n- [部署你的 Keras 模型](https:\u002F\u002Fmedium.com\u002F@burgalon\u002Fdeploying-your-keras-model-35648f9dc5fb)\n- [使用 Keras.js 部署你的 Keras 模型](https:\u002F\u002Fbecominghuman.ai\u002Fdeploying-your-keras-model-using-keras-js-2e5a29589ad8)\n- _Siraj Raval_ 在 YouTube 上发布的《如何将 Keras 模型部署到生产环境》[https:\u002F\u002Fgithub.com\u002FllSourcell\u002Fhow_to_deploy_a_keras_model_to_production]\n- [10 分钟内使用 Flask 将 Keras 模型部署为 Web 应用程序](https:\u002F\u002Fgithub.com\u002Fmtobeiyf\u002Fkeras-flask-deploy-webapp) [优秀的代码库]\n- [使用 Flask 部署 Keras 深度学习模型](https:\u002F\u002Ftowardsdatascience.com\u002Fdeploying-keras-deep-learning-models-with-flask-5da4181436a2)\n- [keras2cpp](https:\u002F\u002Fgithub.com\u002Fpplonski\u002Fkeras2cpp)\n\n## 将 MXNet 模型部署到生产环境：\n- [Apache MXNet 的模型服务器](https:\u002F\u002Fgithub.com\u002Fawslabs\u002Fmxnet-model-server)\n- [运行模型服务器](https:\u002F\u002Fgithub.com\u002Fawslabs\u002Fmxnet-model-server\u002Fblob\u002Fmaster\u002Fdocs\u002Fserver.md)\n- [多模型服务器 (MMS) 文档](https:\u002F\u002Fgithub.com\u002Fawslabs\u002Fmulti-model-server\u002Ftree\u002Fmaster\u002Fdocs)\n- [介绍 Apache MXNet 的模型服务器](https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Fmachine-learning\u002Fintroducing-model-server-for-apache-mxnet\u002F)\n- [单次多目标检测推理服务](https:\u002F\u002Fgithub.com\u002Fawslabs\u002Fmxnet-model-server\u002Fblob\u002Fmaster\u002Fexamples\u002Fssd\u002FREADME.md)\n- [Amazon SageMaker](https:\u002F\u002Faws.amazon.com\u002Fsagemaker\u002F)\n- [如何服务使用 Gluon API 构建的 MXNet 模型](https:\u002F\u002Fdiscuss.mxnet.io\u002Ft\u002Fhow-can-we-serve-mxnet-models-built-with-gluon-api\u002F684)\n- [MXNet C++ 包](https:\u002F\u002Fgithub.com\u002Fapache\u002Fincubator-mxnet\u002Ftree\u002Fmaster\u002Fcpp-package)\n- [MXNet C++ 包示例](https:\u002F\u002Fgithub.com\u002Fapache\u002Fincubator-mxnet\u002Ftree\u002Fmaster\u002Fcpp-package\u002Fexample)\n- [MXNet C++ 图像分类示例](https:\u002F\u002Fgithub.com\u002Fapache\u002Fincubator-mxnet\u002Ftree\u002Fmaster\u002Fexample\u002Fimage-classification\u002Fpredict-cpp)\n- [MXNet C++ 教程](http:\u002F\u002Fmxnet.incubator.apache.org\u002Ftutorials\u002Fc%2B%2B\u002Fbasics.html)\n- [MXNet API 简介](https:\u002F\u002Fbecominghuman.ai\u002Fan-introduction-to-the-mxnet-api-part-1-848febdcf8ab) [_非常好的学习 MXNet 的教程_]\n- [GluonCV](https:\u002F\u002Fgluon-cv.mxnet.io\u002F)\n- [GluonNLP](http:\u002F\u002Fgluon-nlp.mxnet.io\u002F)\n- [用于生产级神经网络推理的模型量化](https:\u002F\u002Fmedium.com\u002Fapache-mxnet\u002Fmodel-quantization-for-production-level-neural-network-inference-f54462ebba05) [_非常优秀_]\n\n## 使用 Go 部署机器学习模型：\n- [Cortex：在生产环境中部署机器学习模型](https:\u002F\u002Fgithub.com\u002Fcortexlabs\u002Fcortex)\n- [Cortex 主页](https:\u002F\u002Fwww.cortex.dev\u002F)\n- [为什么我们使用 Go 而不是 Python 来部署机器学习模型](https:\u002F\u002Ftowardsdatascience.com\u002Fwhy-we-deploy-machine-learning-models-with-go-not-python-a4e35ec16deb)\n- [Go-Torch](https:\u002F\u002Fgithub.com\u002Forktes\u002Fgo-torch)\n- [Gotch — PyTorch 的 Go API](https:\u002F\u002Fgithub.com\u002Fsugarme\u002Fgotch)\n- [TensorFlow Go Lang](https:\u002F\u002Fwww.tensorflow.org\u002Finstall\u002Flang_go)\n- [Go-onnx](https:\u002F\u002Fgithub.com\u002Fdhdanie\u002Fgoonnx)\n\n## 通用深度学习部署工具包：\n- [OpenVINO 工具包 — 深度学习部署工具包仓库](https:\u002F\u002Fgithub.com\u002Fopenvinotoolkit\u002Fopenvino) [_很棒_]  \n- [ClearML — 机器学习\u002F深度学习开发与生产套件](https:\u002F\u002Fgithub.com\u002Fallegroai\u002Fclearml)  \n- [使用 Heroku 部署模型：Heroku 完整指南](https:\u002F\u002Fwww.analyticsvidhya.com\u002Fblog\u002F2021\u002F10\u002Fa-complete-guide-on-machine-learning-model-deployment-using-heroku\u002F) [不错]  \n- [NVIDIA Triton 推理服务器](https:\u002F\u002Fdeveloper.nvidia.com\u002Fnvidia-triton-inference-server) [**很棒**]  \n- [NVIDIA Triton 推理服务器 — GitHub](https:\u002F\u002Fgithub.com\u002Ftriton-inference-server\u002Fserver) [**很棒**]  \n- [Cohere 利用 NVIDIA Triton 推理服务器提升推理速度](https:\u002F\u002Ftxt.cohere.ai\u002Fnvidia-boosts-inference-speed-with-cohere\u002F)  \n- [NVIDIA 深度学习 Tensor Core 示例](https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FDeepLearningExamples) [很有意思]  \n- [使用 Triton 推理服务器部署 Jasper 推理模型](https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FDeepLearningExamples\u002Ftree\u002Fmaster\u002FPyTorch\u002FSpeechRecognition\u002FJasper\u002Ftriton) [很有用]  \n- [NVIDIA 通过 Triton 提供的 MLOps 课程](https:\u002F\u002Fanalyticsindiamag.com\u002Fnvidia-is-offering-a-four-hour-self-paced-course-on-mlops\u002F)  \n- [Awesome Production Machine Learning](https:\u002F\u002Fgithub.com\u002FEthicalML\u002Fawesome-production-machine-learning) [很棒]\n- [BentoML 模型服务](https:\u002F\u002Fwww.linkedin.com\u002Fposts\u002Faxelmdz_mlops-machinelearning-datascience-activity-7194637600861618176-iQaF?utm_source=share&utm_medium=member_desktop)\n\n## 华为深度学习框架：\n- [MindSpore — 华为深度学习框架](https:\u002F\u002Fgithub.com\u002Fmindspore-ai\u002Fmindspore)\n- [MindSpore 教程](https:\u002F\u002Fwww.mindspore.cn\u002Ftutorial\u002Fen\u002F0.1.0-alpha\u002Fquick_start\u002Fquick_start.html)\n\n## 通用深度学习编译器栈：\n- [TVM 栈](https:\u002F\u002Ftvm.ai\u002F)\n\n## 不同深度学习框架之间的模型转换：\n- [ONNX（开放神经网络交换）](https:\u002F\u002Fonnx.ai\u002F)\n- [使用 ONNX 的教程](https:\u002F\u002Fgithub.com\u002Fonnx\u002Ftutorials)\n- [MMdnn](https:\u002F\u002Fgithub.com\u002FMicrosoft\u002FMMdnn) [_非常棒_]  \n- [将完整的 ImageNet 预训练模型从 MXNet 转换为 PyTorch](https:\u002F\u002Fblog.paperspace.com\u002Fconvert-full-imagenet-pre-trained-model-from-mxnet-to-pytorch\u002F) [_非常棒_，这里的“完整 ImageNet 模型”指的是在约 1400 万张图像上训练的模型]\n\n## 一些 Caffe2 教程：\n- [使用 Caffe2 的 Mnist 示例](http:\u002F\u002Fvast.uccs.edu\u002F~adhamija\u002Fblog\u002FMNIST_singleGPU.html)\n- [Caffe2 C++ 教程和示例](https:\u002F\u002Fgithub.com\u002Fleonardvandriel\u002Fcaffe2_cpp_tutorial)\n- [在 Caffe2 上进行 SqueezeNet 的迁移学习](https:\u002F\u002Fmedium.com\u002F@KazamiXHayato\u002Fmake-transfer-learning-in-caffe2-21d96c47ba0e)\n- [使用 Caffe2 框架在 C++ 中构建基础程序](https:\u002F\u002Fmedium.com\u002F@KazamiXHayato\u002Fwrite-caffe2-program-in-c-5519e2646382)\n\n## 一些用于 UI 设计（前端开发）的实用资源：\n- [ReactJS vs Angular5 vs Vue.js](https:\u002F\u002Fmedium.com\u002F@TechMagic\u002Freactjs-vs-angular5-vs-vue-js-what-to-choose-in-2018-b91e028fa91d)\n- [Angular 与 React 及其核心语言的比较](https:\u002F\u002Fmedium.freecodecamp.org\u002Fa-comparison-between-angular-and-react-and-their-core-languages-9de52f485a76)\n- [成为全栈开发者的指南](https:\u002F\u002Fmedium.coderbyte.com\u002Fa-guide-to-becoming-a-full-stack-developer-in-2017-5c3c08a1600c) [_非常不错的教程_]\n- [2018 年成为 Web 开发者的路线图](https:\u002F\u002Fgithub.com\u002Fkamranahmedse\u002Fdeveloper-roadmap) [_非常好的代码仓库_]\n- [2018 年的现代前端开发者](https:\u002F\u002Fmedium.com\u002Ftech-tajawal\u002Fmodern-frontend-developer-in-2018-4c2072fa2b9c)\n- [2018 年成为 React 开发者的路线图](https:\u002F\u002Fgithub.com\u002Fadam-golab\u002Freact-developer-roadmap)\n- [2019 年 UI 和 UX 设计趋势](https:\u002F\u002Fuxplanet.org\u002F2019-ui-and-ux-design-trends-92dfa8323225) [_不错_]\n- [Streamlit](https:\u002F\u002Fstreamlit.io\u002F) [_构建自定义机器学习工具的最快方式_]\n- [Gradio](https:\u002F\u002Fwww.gradio.app\u002F) [**不错**]\n- [Web Developer Monthly](https:\u002F\u002Fmedium.com\u002F@andreineagoie\u002Fweb-developer-monthly-july-2018-513e02f15fb6)\n- [23 款最佳 React UI 组件框架](https:\u002F\u002Fhackernoon.com\u002F23-best-react-ui-component-libraries-and-frameworks-250a81b2ac42)\n- [2018 年 9 款使用 Styled Components 的 React UI 库](https:\u002F\u002Fblog.bitsrc.io\u002F9-react-styled-components-ui-libraries-for-2018-4e1a0bd3e179)\n- [35 款新的 UI 设计工具](https:\u002F\u002Fblog.prototypr.io\u002F35-new-tools-for-ui-design-412cf1d701fd)\n- [5 款加速应用开发的工具](https:\u002F\u002Fmedium.com\u002Fswlh\u002F5-tools-to-speed-up-your-app-development-6979d0e49e34) [_非常不错_]\n- [如何将 ReactJS 与 Webpack 4、Babel 7 和 Material Design 结合使用](https:\u002F\u002Fmedium.freecodecamp.org\u002Fhow-to-use-reactjs-with-webpack-4-babel-7-and-material-design-ff754586f618)\n- [Adobe Typekit](https:\u002F\u002Ftypekit.com\u002F) [_出色的字体，随需而至_]\n- [使用 Angular 6 构建一个真实的精美 Web 应用程序](https:\u002F\u002Fmedium.com\u002F@hamedbaatour\u002Fbuild-a-real-world-beautiful-web-app-with-angular-6-a-to-z-ultimate-guide-2018-part-i-e121dd1d55e)\n- [你不懂 JavaScript](https:\u002F\u002Fgithub.com\u002Fgetify\u002FYou-Dont-Know-JS)\n- [JavaScript 最佳文章 Top 10](https:\u002F\u002Fmedium.mybridge.co\u002Fjavascript-top-10-articles-for-the-past-month-v-sep-2018-8f27a300d6c5)\n- [使用 Adobe XD 进行网页设计](https:\u002F\u002Fmedium.freecodecamp.org\u002Fa-developers-guide-to-web-design-for-non-designers-1f64ce28c38d)\n- [INSPINIA Bootstrap Web 主题](https:\u002F\u002Fwrapbootstrap.com\u002Ftheme\u002Finspinia-responsive-admin-theme-WB0R5L90S)\n- [面向前端开发者的学习跟踪器](https:\u002F\u002Fgithub.com\u002FSyknapse\u002FMy-Learning-Tracker-first-ten-months)\n- [最全的现代前端黑客技巧速查表](https:\u002F\u002Fmedium.freecodecamp.org\u002Fmodern-frontend-hacking-cheatsheets-df9c2566c72a) [_实用且有趣_]\n- [为机器学习工作流 GUI 化（Machine Flow）](https:\u002F\u002Ftowardsdatascience.com\u002Fgui-fying-the-machine-learning-workflow-towards-rapid-discovery-of-viable-pipelines-cab2552c909f)\n- [Electron - 使用 JavaScript 构建跨平台桌面应用](https:\u002F\u002Felectronjs.org\u002F) [_非常好_]\n- [Opyrator - 将 Python 函数转换为带有 Web API 的微服务](https:\u002F\u002Fgithub.com\u002Fml-tooling\u002Fopyrator) [**很棒**]\n- [初探 PyScript：在浏览器中运行 Python](https:\u002F\u002Frealpython.com\u002Fpyscript-python-in-browser\u002F) [**有趣**]\n\n## 移动与嵌入式设备开发：\n- [PyTorch Mobile](https:\u002F\u002Fpytorch.org\u002Fmobile\u002Fhome\u002F) [_非常优秀_]  \n- [2018 年移动 UI 设计趋势](https:\u002F\u002Fuxplanet.org\u002Fmobile-ui-design-trends-in-2018-ccd26031dfd8)  \n- [ncnn - 高性能神经网络推理框架，专为移动平台优化](https:\u002F\u002Fgithub.com\u002FTencent\u002Fncnn) [_实用_]  \n- [阿里巴巴 - MNN](https:\u002F\u002Fgithub.com\u002Falibaba\u002FMNN)  \n- [Awesome Mobile Machine Learning](https:\u002F\u002Fgithub.com\u002Ffritzlabs\u002FAwesome-Mobile-Machine-Learning)  \n- [EMDL - 嵌入式与移动深度学习](https:\u002F\u002Fgithub.com\u002FEMDL\u002Fawesome-emdl)  \n- [Fritz - 面向 iOS 和 Android 的机器学习平台](https:\u002F\u002Fwww.fritz.ai\u002F)  \n- [TensorFlow Lite](https:\u002F\u002Fwww.tensorflow.org\u002Fmobile\u002Ftflite\u002F)  \n- [微型机器学习：下一场 AI 革命](https:\u002F\u002Fmedium.com\u002F@matthew_stewart\u002Ftiny-machine-learning-the-next-ai-revolution-495c26463868)  \n- [TLT - NVIDIA 转移学习工具包](https:\u002F\u002Fdeveloper.nvidia.com\u002Ftransfer-learning-toolkit)  \n- [NVIDIA Jetson Inference](https:\u002F\u002Fgithub.com\u002Fdusty-nv\u002Fjetson-inference)  [_很棒_]\n- [EfficientML.ai 讲座（MIT 课程）](https:\u002F\u002Fwww.youtube.com\u002Fplaylist?list=PL80kAHvQbh-pT4lCkDT53zT8DKmhE0idB)\n- [PyTorch Edge：通过 ExecuTorch 实现跨移动和边缘设备的本地推理](https:\u002F\u002Fpytorch.org\u002Fblog\u002Fpytorch-edge\u002F?utm_content=268296369&utm_medium=social&utm_source=linkedin&hss_channel=lcp-78618366)\n\n## 后端开发部分：\n- [2018年的现代后端开发者](https:\u002F\u002Fmedium.com\u002Ftech-tajawal\u002Fmodern-backend-developer-in-2018-6b3f7b5f8b9)\n- [以有趣的方式部署前端应用](https:\u002F\u002Fhackernoon.com\u002Fdeploying-frontend-applications-the-fun-way-bc3f69e15331) [_非常好_]\n- [RabbitMQ](https:\u002F\u002Fwww.rabbitmq.com\u002F) [_消息代理软件_]\n- [Celery](http:\u002F\u002Fwww.celeryproject.org\u002F) [_分布式任务队列_]\n- [Kafka](https:\u002F\u002Fkafka.apache.org\u002F) [_分布式流处理平台_]\n- [使用DockerMe进行Docker培训](https:\u002F\u002Fgithub.com\u002FAhmadRafiee\u002FDocker_training_with_DockerMe)\n- [Kubernetes - GitHub](https:\u002F\u002Fgithub.com\u002Fkubernetes\u002Fkubernetes)\n- [在Google Kubernetes Engine上部署机器学习流水线](https:\u002F\u002Ftowardsdatascience.com\u002Fdeploy-machine-learning-model-on-google-kubernetes-engine-94daac85108b)\n- [面向数据科学家的Kubernetes入门](https:\u002F\u002Fwww.jeremyjordan.me\u002Fkubernetes\u002F)\n- [使用Docker Desktop的Jenkins和Kubernetes](https:\u002F\u002Fmedium.com\u002F@garunski\u002Fjenkins-and-kubernetes-with-docker-desktop-53a853486f7c)\n- [Helm：Kubernetes的包管理器](https:\u002F\u002Fhelm.sh\u002F)\n- [使用Docker Swarm创建集群](https:\u002F\u002Fmedium.com\u002Ftech-tajawal\u002Fcreate-cluster-using-docker-swarm-94d7b2a10c43)\n- [deepo - 适用于所有深度学习框架的Docker镜像](https:\u002F\u002Fgithub.com\u002Fufoym\u002Fdeepo)\n- [Kubeflow](https:\u002F\u002Fwww.kubeflow.org\u002F) [_在Kubernetes上部署机器学习工作流_]\n- [kubespray - 部署生产就绪的Kubernetes集群](https:\u002F\u002Fgithub.com\u002Fkubernetes-sigs\u002Fkubespray)\n- [KFServing - 用于服务机器学习模型的Kubernetes](https:\u002F\u002Fgithub.com\u002Fkubeflow\u002Fkfserving)\n- [使用KFServing部署HuggingFace NLP模型](http:\u002F\u002Fwww.pattersonconsultingtn.com\u002Fblog\u002Fdeploying_huggingface_with_kfserving.html) [_有趣_]\n- [Seldon Core - 在Kubernetes上部署机器学习模型](https:\u002F\u002Fwww.seldon.io\u002Ftech\u002Fproducts\u002Fcore\u002F)\n- [Seldon Core - GitHub](https:\u002F\u002Fgithub.com\u002FSeldonIO\u002Fseldon-core)\n- [机器学习：在Ubuntu上使用Kubeflow服务模型，第一部分](https:\u002F\u002Fubuntu.com\u002Fblog\u002Fml-serving-models-with-kubeflow-on-ubuntu-part-1)\n- [CoreWeave Kubernetes云](https:\u002F\u002Fgithub.com\u002Fcoreweave\u002Fkubernetes-cloud\u002Ftree\u002Fmaster\u002Fonline-inference\u002F)\n- [MLOps参考资料](https:\u002F\u002Fgithub.com\u002Fvisenger\u002Fmlops-references) [_面向机器学习的DevOps_]\n- [数据版本控制 - DVC](https:\u002F\u002Fdvc.org\u002F) [_很棒_]\n- [MLEM：打包和部署机器学习模型](https:\u002F\u002Fgithub.com\u002Fiterative\u002Fmlem)\n- [PySyft - 用于加密、保护隐私的深度学习库](https:\u002F\u002Fgithub.com\u002FOpenMined\u002FPySyft)\n- [LocalStack - 功能齐全的本地AWS云栈](https:\u002F\u002Fgithub.com\u002Flocalstack\u002Flocalstack)\n- [poetry：Python包管理和依赖管理](https:\u002F\u002Fpython-poetry.org\u002F)\n- [Netflix为大规模流媒体选择的强大数据库](https:\u002F\u002Fwww.linkedin.com\u002Fposts\u002Fchandreshdesai_cloudcomputing-devops-softwareengineering-activity-7108414221482115072-HGil?utm_source=share&utm_medium=member_desktop)\n- [Ray：用于扩展AI和Python应用的统一框架](https:\u002F\u002Fgithub.com\u002Fray-project\u002Fray)\n\n## GPU管理库：\n- [GPUtil](https:\u002F\u002Fgithub.com\u002Fanderskm\u002Fgputil)\n- [py3nvml](https:\u002F\u002Fgithub.com\u002Ffbcotter\u002Fpy3nvml) [_NVIDIA管理库的Python 3绑定_]\n- [PyCUDA - GitHub](https:\u002F\u002Fgithub.com\u002Finducer\u002Fpycuda)\n- [PyCUDA](https:\u002F\u002Fmathema.tician.de\u002Fsoftware\u002Fpycuda\u002F)\n- [PyCUDA教程](https:\u002F\u002Fdocumen.tician.de\u002Fpycuda\u002F)\n- [setGPU](https:\u002F\u002Fgithub.com\u002Fbamos\u002FsetGPU)\n- [监控你的GPU](https:\u002F\u002Fgithub.com\u002Fmsalvaris\u002Fgpu_monitor) [**优秀**]\n- [GPU-Burn - 多GPU CUDA压力测试](https:\u002F\u002Fgithub.com\u002Fwilicc\u002Fgpu-burn) [_实用_]\n- [Grafana - 监控与可观测性](https:\u002F\u002Fgithub.com\u002Fgrafana\u002Fgrafana) [**优秀**]\n- [Prometheus](https:\u002F\u002Fprometheus.io\u002F) [_非常优秀的监控解决方案，可用于提取所需指标_]\n- [OpenAI Triton：面向神经网络的开源GPU编程](https:\u002F\u002Fopenai.com\u002Fblog\u002Ftriton\u002F)\n\n## 加速与可扩展的 Python 代码：\n- [Numba - 让 Python 代码运行得更快](http:\u002F\u002Fnumba.pydata.org\u002F)\n- [Dask - 原生支持 Python 的分布式计算](https:\u002F\u002Fdask.org\u002F)\n- [什么是 Dask](https:\u002F\u002Fmedium.com\u002Fbetter-programming\u002Fwhat-is-dask-and-how-can-it-help-you-as-a-data-scientist-72adec7cec57)  \n- [Ray - 运行分布式应用](https:\u002F\u002Fgithub.com\u002Fray-project\u002Fray)  \n- [Neural Network Distiller](https:\u002F\u002Fgithub.com\u002FNervanaSystems\u002Fdistiller\u002F) [_PyTorch 中深度学习模型的蒸馏与量化_]\n- [Autodistill](https:\u002F\u002Fgithub.com\u002Fautodistill\u002Fautodistill)  \n- [PyTorch 剪枝教程](https:\u002F\u002Fpytorch.org\u002Ftutorials\u002Fintermediate\u002Fpruning_tutorial.html)  \n- [能否在不损失精度的情况下移除神经网络中 99% 的参数？—— 权重剪枝简介](https:\u002F\u002Ftowardsdatascience.com\u002Fcan-you-remove-99-of-a-neural-network-without-losing-accuracy-915b1fab873b)  \n- [PocketFlow - 自动模型压缩（AutoMC）框架](https:\u002F\u002Fgithub.com\u002FTencent\u002FPocketFlow) [**优秀**]  \n- [TensorFlow 模型优化工具包介绍](https:\u002F\u002Fmedium.com\u002Ftensorflow\u002Fintroducing-the-model-optimization-toolkit-for-tensorflow-254aca1ba0a3)  \n- [TensorFlow 模型优化工具包 —— 训练后整数量化](https:\u002F\u002Fmedium.com\u002Ftensorflow\u002Ftensorflow-model-optimization-toolkit-post-training-integer-quantization-b4964a1ea9ba)  \n- [TensorFlow 训练后量化](https:\u002F\u002Fwww.tensorflow.org\u002Flite\u002Fperformance\u002Fpost_training_quantization)  \n- [PyTorch 中的动态量化](https:\u002F\u002Fpytorch.org\u002Ftutorials\u002Fadvanced\u002Fdynamic_quantization_tutorial.html)  \n- [PyTorch 中的静态量化](https:\u002F\u002Fpytorch.org\u002Ftutorials\u002Fadvanced\u002Fstatic_quantization_tutorial.html)  \n- [NVIDIA DALI - 高度优化的深度学习数据预处理](https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fdali)  \n- [Horovod - 分布式训练框架](https:\u002F\u002Fgithub.com\u002Fhorovod\u002Fhorovod)  \n- [ONNX 浮点32转浮点16](https:\u002F\u002Fgithub.com\u002Fonnx\u002Fonnx-docker\u002Fblob\u002Fmaster\u002Fonnx-ecosystem\u002Fconverter_scripts\u002Ffloat32_float16_onnx.ipynb)  \n- [使用 TensorRT 加速深度学习推理](https:\u002F\u002Fdevblogs.nvidia.com\u002Fspeeding-up-deep-learning-inference-using-tensorrt\u002F)  \n- [加速训练](https:\u002F\u002Fai.googleblog.com\u002F2020\u002F05\u002Fspeeding-up-neural-network-training.html)  \n- [原生 PyTorch 自动混合精度，用于在 NVIDIA GPU 上加速训练](https:\u002F\u002Fpytorch.org\u002Fblog\u002Faccelerating-training-on-nvidia-gpus-with-pytorch-automatic-mixed-precision\u002F)  \n- [JAX - 可组合的 Python+NumPy 程序变换工具](https:\u002F\u002Fgithub.com\u002Fgoogle\u002Fjax)  \n- [TensorRTx - 使用 TensorRT 实现流行的深度学习网络](https:\u002F\u002Fgithub.com\u002Fwang-xinyu\u002Ftensorrtx)  \n- [使用 TensorFlow、ONNX 和 TensorRT 加速深度学习推理](https:\u002F\u002Fdevblogs.nvidia.com\u002Fspeeding-up-deep-learning-inference-using-tensorflow-onnx-and-tensorrt\u002F)  \n- [TensorRT 开发者指南](https:\u002F\u002Fdocs.nvidia.com\u002Fdeeplearning\u002Fsdk\u002Ftensorrt-developer-guide\u002Findex.html)  \n- [如何将模型从 PyTorch 转换为 TensorRT 并加速推理](https:\u002F\u002Fwww.learnopencv.com\u002Fhow-to-convert-a-model-from-pytorch-to-tensorrt-and-speed-up-inference\u002F) [_不错_]  \n- [cuDF: GPU DataFrame 库](https:\u002F\u002Fgithub.com\u002Frapidsai\u002Fcudf)\n- [cuDF 示例](https:\u002F\u002Fcolab.research.google.com\u002Fdrive\u002F12tCzP94zFG2BRduACucn5Q_OcX1TUKY3)  \n\n## 深度学习硬件笔记：  \n- [深度学习硬件](https:\u002F\u002Fblog.inten.to\u002Fhardware-for-deep-learning-part-3-gpu-8906c1644664)  \n\n## MLOps 课程与资源：  \n- [MLOps 基础](https:\u002F\u002Fgithub.com\u002Fgraviraja\u002FMLOps-Basics) [优秀]  \n- [MLOPs-Zoomcamp](https:\u002F\u002Fgithub.com\u002FDataTalksClub\u002Fmlops-zoomcamp) [优秀]   \n- [学习 MLOps 的资源合集](https:\u002F\u002Fgithub.com\u002Fdair-ai\u002FMLOPs-Primer) [优秀]  \n- [Awesome MLOps](https:\u002F\u002Fgithub.com\u002Fvisenger\u002Fawesome-mlops) [优秀]  \n- [数据科学主题与 MLOps](https:\u002F\u002Fgithub.com\u002Fkhuyentran1401\u002FData-science#mlops) [优秀]  \n- [MLEM: 打包和部署机器学习模型](https:\u002F\u002Fgithub.com\u002Fiterative\u002Fmlem)  \n- [DevOps 练习](https:\u002F\u002Fgithub.com\u002Fbregman-arie\u002Fdevops-exercises)  \n- [MlOPs 示例项目](https:\u002F\u002Fgithub.com\u002FAntonisCSt\u002FMlops_project_semicon)  \n- [prefect: 编排和监控您的所有工作流](https:\u002F\u002Fwww.prefect.io\u002F)  \n- [DataTalks Club: 数据交流的场所](https:\u002F\u002Fdatatalks.club\u002F)  \n- [OpenNMT CTranslate2: Transformer 模型的快速推理引擎](https:\u002F\u002Fgithub.com\u002FOpenNMT\u002FCTranslate2\u002F)  \n\n## 其他：\n- [生产级深度学习指南](https:\u002F\u002Fgithub.com\u002Falirezadir\u002FProduction-Level-Deep-Learning)  \n- [Facebook 表示开发者会爱上 PyTorch 1.0](https:\u002F\u002Fmedium.com\u002Fsyncedreview\u002Ffacebook-says-developers-will-love-pytorch-1-0-ba2f89ebc9cc)\n- [PyTorch 工作流的一些变更](https:\u002F\u002Fgithub.com\u002Fpytorch\u002Fpytorch\u002Fissues\u002F6032)\n- [wandb - 用于可视化和跟踪机器学习实验的工具](https:\u002F\u002Fgithub.com\u002Fwandb\u002Fclient)  \n- [PyTorch 和 Caffe2 仓库逐渐融合](https:\u002F\u002Fgithub.com\u002Fcaffe2\u002Fcaffe2\u002Fissues\u002F2439#issuecomment-391155017)\n- [PyTorch 还是 TensorFlow？](https:\u002F\u002Fawni.github.io\u002Fpytorch-tensorflow\u002F)\n- [2018 年选择深度学习框架：TensorFlow 还是 PyTorch？](http:\u002F\u002Fcv-tricks.com\u002Fdeep-learning-2\u002Ftensorflow-or-pytorch\u002F)\n- [PyTorch 与 TensorFlow 之间的深度学习大战](https:\u002F\u002Fhub.packtpub.com\u002Fcan-a-production-ready-pytorch-1-0-give-tensorflow-a-tough-time\u002F)\n- [将机器学习模型嵌入 Web 应用程序（第 1 部分）](https:\u002F\u002Ftowardsdatascience.com\u002Fembedding-machine-learning-models-to-web-apps-part-1-6ab7b55ee428)\n- [部署深度学习模型：第 1 部分概述](https:\u002F\u002Ftowardsdatascience.com\u002Fdeploying-deep-learning-models-part-1-an-overview-77b4d01dd6f7)\n- [生产环境中的机器学习](https:\u002F\u002Fmedium.com\u002Fcontentsquare-engineering-blog\u002Fmachine-learning-in-production-c53b43283ab1)\n- [如何通过 Python 将数据预处理速度提升 2–6 倍](https:\u002F\u002Ftowardsdatascience.com\u002Fheres-how-you-can-get-a-2-6x-speed-up-on-your-data-pre-processing-with-python-847887e63be5)\n- [使您的 C 库可通过 Python 调用](https:\u002F\u002Fmedium.com\u002F@shamir.stav_83310\u002Fmaking-your-c-library-callable-from-python-by-wrapping-it-with-cython-b09db35012a3)\n- [MIL WebDNN](https:\u002F\u002Fmil-tokyo.github.io\u002Fwebdnn\u002F)\n- [多 GPU 框架比较](https:\u002F\u002Fmedium.com\u002F@iliakarmanov\u002Fmulti-gpu-rosetta-stone-d4fa96162986) [_优秀_]","# Deep-Learning-in-Production 快速上手指南\n\n本指南基于 `Deep-Learning-in-Production` 仓库资源整理，旨在帮助开发者将深度学习模型（PyTorch, TensorFlow, Keras, MXNet 等）高效部署至生产环境。由于该仓库主要汇集了大量框架特定的部署方案与工具链，以下指南将以最主流的 **PyTorch + TorchServe** 方案为例，展示从模型导出到服务部署的核心流程，并列出其他框架的关键路径。\n\n## 环境准备\n\n在开始之前，请确保您的开发环境满足以下基本要求：\n\n*   **操作系统**: Linux (推荐 Ubuntu 18.04\u002F20.04), macOS, 或 Windows (WSL2 推荐)。\n*   **Python 版本**: Python 3.8 - 3.10 (根据具体框架版本调整)。\n*   **前置依赖**:\n    *   Git\n    *   CUDA Toolkit (如需 GPU 加速，版本需与深度学习框架匹配)\n    *   Java JDK 11+ (TorchServe 运行需要)\n*   **国内加速建议**:\n    *   推荐使用清华源或阿里源安装 Python 包，以提升下载速度：\n        ```bash\n        pip install -i https:\u002F\u002Fpypi.tuna.tsinghua.edu.cn\u002Fsimple \u003Cpackage_name>\n        ```\n\n## 安装步骤\n\n本示例以部署 PyTorch 模型为例，使用官方推荐的 **TorchServe** 作为推理服务器。\n\n### 1. 安装基础深度学习框架\n首先安装 PyTorch。国内用户建议使用清华源加速安装：\n\n```bash\npip install torch torchvision torchaudio --index-url https:\u002F\u002Fdownload.pytorch.org\u002Fwhl\u002Fcu118\n# 若无需 GPU，使用 cpu 版本:\n# pip install torch torchvision torchaudio --index-url https:\u002F\u002Fdownload.pytorch.org\u002Fwhl\u002Fcpu\n```\n\n### 2. 安装 TorchServe 及处理器\n安装 TorchServe 核心组件及其默认的 PyTorch 处理器：\n\n```bash\npip install torchserve torch-model-archiver torch-workflow-archiver\n# 若使用 GPU 版本，需额外安装：\n# pip install torchserve-gpu torch-model-archiver-gpu torch-workflow-archiver-gpu\n```\n\n### 3. 验证安装\n检查安装是否成功：\n\n```bash\ntorchserve --version\nmodel-archiver --version\n```\n\n## 基本使用\n\n以下流程演示如何将一个训练好的 PyTorch 模型打包并启动为 REST API 服务。\n\n### 第一步：导出模型 (.pt\u002F.pth)\n假设你已有一个训练好的 PyTorch 模型 `model`，首先需要将其转换为 TorchScript 格式以便在生产环境中运行：\n\n```python\nimport torch\n\n# 假设 model 是你的神经网络实例，且已加载权重\nmodel.eval()\nexample_input = torch.rand(1, 3, 224, 224) # 根据实际输入形状调整\n\n# 追踪模式 (Tracing) 导出\nscripted_model = torch.jit.trace(model, example_input)\n\n# 保存模型\nscripted_model.save(\"resnet18.pt\")\n```\n\n### 第二步：打包模型 (.mar)\n使用 `model-archiver` 将模型文件、处理逻辑（可选，此处使用默认图像分类处理器）打包为 `.mar` 文件：\n\n```bash\nmodel-archiver --model-name resnet18 \\\n--version 1.0 \\\n--model-file resnet18.pt \\\n--handler image_classifier \\\n--export-path model-store\n```\n*注：`--handler` 可根据任务类型选择，如 `text_classification`, `object_detector` 等，也可编写自定义 handler。*\n\n### 第三步：启动服务\n启动 TorchServe 并加载模型：\n\n```bash\ntorchserve --start --model-store model-store --models resnet18.mar\n```\n\n### 第四步：调用推理接口\n服务启动后，默认监听 `8080` (管理) 和 `8081` (推理) 端口。使用 `curl` 发送图片进行预测：\n\n```bash\ncurl http:\u002F\u002Flocalhost:8081\u002Fpredictions\u002Fresnet18 -T dog.jpg\n```\n\n返回结果通常为 JSON 格式的类别概率分布。\n\n---\n\n### 其他框架部署关键路径参考\n\n如果您使用的是其他框架，请参考以下核心工具链进行部署：\n\n*   **TensorFlow**: 推荐使用 **TensorFlow Serving**。\n    *   核心命令：`tensorflow_model_server --model_base_path=\u002Fpath\u002Fto\u002Fmodel --rest_api_port=8501`\n    *   关键资源：Freeze 模型 (`freeze_graph`) -> Docker 容器化部署。\n*   **Keras**: 可复用 TensorFlow Serving，或使用 **Flask + Redis** 架构。\n    *   典型方案：利用 `keras-flask-deploy-webapp` 模板快速搭建 Web 服务。\n*   **MXNet**: 推荐使用 **Model Server for Apache MXNet (MMS)**。\n    *   核心命令：`mxnet-model-server --start --models my_model.mar`\n*   **C++ 高性能部署**:\n    *   **PyTorch**: 使用 LibTorch C++ API，参考 `Loading a PyTorch Model in C++` 教程。\n    *   **通用加速**: 使用 **NVIDIA Triton Inference Server** 或 **OpenVINO Toolkit** 进行多框架统一部署与硬件加速。\n*   **Go 语言部署**:\n    *   可使用 **Cortex** 平台或 **Gotch** (Go binding for PyTorch) 进行高性能后端开发。","某电商初创团队需要将研发好的 PyTorch 商品图像识别模型快速部署到线上，以支持实时的“以图搜图”功能。\n\n### 没有 Deep-Learning-in-Production 时\n- **技术选型迷茫**：面对 TorchServe、LibTorch C++ API、ONNX Runtime 等多种部署方案，团队缺乏权威指引，难以判断哪种最适合当前高并发场景。\n- **环境兼容陷阱**：在将 Python 训练模型迁移至 C++ 生产环境时，因算子不支持或版本差异导致推理失败，排查问题耗费数天。\n- **性能瓶颈难解**：直接封装 Flask 接口导致线程安全问题频发，且未进行架构优化，响应延迟高达数百毫秒，无法满足实时交互需求。\n- **资源浪费严重**：由于缺乏服务器less（如 AWS Lambda）或边缘端部署的最佳实践，不得不维持高昂的 GPU 服务器常开成本。\n\n### 使用 Deep-Learning-in-Production 后\n- **路径清晰明确**：借助仓库中整理的\"TorchServe\"和\"LitServe\"等精选教程，团队迅速锁定了适合的高性能服务框架，缩短了决策周期。\n- **平滑语言迁移**：参考\"Loading a PyTorch Model in C++\"等核心指南，顺利利用 LibTorch 完成模型转换，彻底解决了生产环境的依赖冲突。\n- **推理效率飞跃**：采纳关于线程安全及原生架构优化（torchao）的建议，重构后的接口响应时间降低至 50 毫秒以内，支撑起万级 QPS。\n- **成本显著优化**：依据 AWS Lambda 与 ONNX 结合的实战案例，成功实现弹性伸缩部署，使基础设施成本降低了 60%。\n\nDeep-Learning-in-Production 通过提供经过验证的部署路线图与避坑指南，帮助团队将模型从实验室原型高效转化为稳定、低成本的生产力工具。","https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fahkarami_Deep-Learning-in-Production_8bddba4e.png","ahkarami","Amir Hossein Karami","https:\u002F\u002Foss.gittoolsai.com\u002Favatars\u002Fahkarami_dfb0631d.jpg","Deep Learning Research Scientist",null,"https:\u002F\u002Fleanpub.com\u002Fconvolutions-for-deep-learning","https:\u002F\u002Fgithub.com\u002Fahkarami",4380,692,"2026-04-02T08:37:14",1,"","未说明",{"notes":87,"python":85,"dependencies":88},"该仓库是一个关于深度学习模型生产环境部署的笔记和参考资源合集，并非一个可直接运行的单一软件工具。内容涵盖了 PyTorch、TensorFlow、Keras、MXNet 等多种框架的部署教程、C++ 接口调用、服务器搭建（如 TorchServe, TensorFlow Serving, Triton）以及 Go 语言部署方案。具体的运行环境需求（如操作系统、GPU、内存、Python 版本等）取决于用户选择参考的特定教程或所部署的具体模型，需查阅文中列出的各个外部链接获取详细信息。",[],[52,14],[91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110],"deep-learning","deep-neural-networks","python","pytorch","tesnorflow","keras","mxnet","caffe2","production","serving","c-plus-plus","model-serving","tutorial","flask","rest-api","react","serving-pytorch-models","convert-pytorch-models","angularjs","tensorflow-models","2026-03-27T02:49:30.150509","2026-04-11T21:58:51.842503",[114,119],{"id":115,"question_zh":116,"answer_zh":117,"source_url":118},29866,"如何将现有的 PyTorch 图像描述模型部署到 Flask？","可以使用通用的 Flask 部署教程，其结构与 Keras 或 PyTorch 模型基本一致。推荐参考此教程：https:\u002F\u002Fwww.pyimagesearch.com\u002F2018\u002F02\u002F05\u002Fdeep-learning-production-keras-redis-flask-apache\u002F。该教程涵盖了使用 Redis、Flask 和 Apache 进行深度学习模型生产环境部署的完整流程。","https:\u002F\u002Fgithub.com\u002Fahkarami\u002FDeep-Learning-in-Production\u002Fissues\u002F2",{"id":120,"question_zh":121,"answer_zh":122,"source_url":123},29867,"如何通过 LibTorch C++ API 将 PyTorch 集成到 Node.js 中？","可以参考这篇关于使用发布的 LibTorch C++ API 将 PyTorch 集成到 Node.js 的文章：http:\u002F\u002Fblog.christianperone.com\u002F2018\u002F10\u002Fpytorch-1-0-tracing-jit-and-libtorch-c-api-to-integrate-pytorch-into-nodejs\u002F。文章详细介绍了利用 PyTorch 1.0 的追踪 JIT 功能实现集成的方法。","https:\u002F\u002Fgithub.com\u002Fahkarami\u002FDeep-Learning-in-Production\u002Fissues\u002F1",[]]