[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"similar-Peterande--D-FINE":3,"tool-Peterande--D-FINE":64},[4,17,26,40,48,56],{"id":5,"name":6,"github_repo":7,"description_zh":8,"stars":9,"difficulty_score":10,"last_commit_at":11,"category_tags":12,"status":16},3808,"stable-diffusion-webui","AUTOMATIC1111\u002Fstable-diffusion-webui","stable-diffusion-webui 是一个基于 Gradio 构建的网页版操作界面，旨在让用户能够轻松地在本地运行和使用强大的 Stable Diffusion 图像生成模型。它解决了原始模型依赖命令行、操作门槛高且功能分散的痛点，将复杂的 AI 绘图流程整合进一个直观易用的图形化平台。\n\n无论是希望快速上手的普通创作者、需要精细控制画面细节的设计师，还是想要深入探索模型潜力的开发者与研究人员，都能从中获益。其核心亮点在于极高的功能丰富度：不仅支持文生图、图生图、局部重绘（Inpainting）和外绘（Outpainting）等基础模式，还独创了注意力机制调整、提示词矩阵、负向提示词以及“高清修复”等高级功能。此外，它内置了 GFPGAN 和 CodeFormer 等人脸修复工具，支持多种神经网络放大算法，并允许用户通过插件系统无限扩展能力。即使是显存有限的设备，stable-diffusion-webui 也提供了相应的优化选项，让高质量的 AI 艺术创作变得触手可及。",162132,3,"2026-04-05T11:01:52",[13,14,15],"开发框架","图像","Agent","ready",{"id":18,"name":19,"github_repo":20,"description_zh":21,"stars":22,"difficulty_score":23,"last_commit_at":24,"category_tags":25,"status":16},2271,"ComfyUI","Comfy-Org\u002FComfyUI","ComfyUI 是一款功能强大且高度模块化的视觉 AI 引擎，专为设计和执行复杂的 Stable Diffusion 图像生成流程而打造。它摒弃了传统的代码编写模式，采用直观的节点式流程图界面，让用户通过连接不同的功能模块即可构建个性化的生成管线。\n\n这一设计巧妙解决了高级 AI 绘图工作流配置复杂、灵活性不足的痛点。用户无需具备编程背景，也能自由组合模型、调整参数并实时预览效果，轻松实现从基础文生图到多步骤高清修复等各类复杂任务。ComfyUI 拥有极佳的兼容性，不仅支持 Windows、macOS 和 Linux 全平台，还广泛适配 NVIDIA、AMD、Intel 及苹果 Silicon 等多种硬件架构，并率先支持 SDXL、Flux、SD3 等前沿模型。\n\n无论是希望深入探索算法潜力的研究人员和开发者，还是追求极致创作自由度的设计师与资深 AI 绘画爱好者，ComfyUI 都能提供强大的支持。其独特的模块化架构允许社区不断扩展新功能，使其成为当前最灵活、生态最丰富的开源扩散模型工具之一，帮助用户将创意高效转化为现实。",107662,2,"2026-04-03T11:11:01",[13,14,15],{"id":27,"name":28,"github_repo":29,"description_zh":30,"stars":31,"difficulty_score":23,"last_commit_at":32,"category_tags":33,"status":16},2268,"ML-For-Beginners","microsoft\u002FML-For-Beginners","ML-For-Beginners 是由微软推出的一套系统化机器学习入门课程，旨在帮助零基础用户轻松掌握经典机器学习知识。这套课程将学习路径规划为 12 周，包含 26 节精炼课程和 52 道配套测验，内容涵盖从基础概念到实际应用的完整流程，有效解决了初学者面对庞大知识体系时无从下手、缺乏结构化指导的痛点。\n\n无论是希望转型的开发者、需要补充算法背景的研究人员，还是对人工智能充满好奇的普通爱好者，都能从中受益。课程不仅提供了清晰的理论讲解，还强调动手实践，让用户在循序渐进中建立扎实的技能基础。其独特的亮点在于强大的多语言支持，通过自动化机制提供了包括简体中文在内的 50 多种语言版本，极大地降低了全球不同背景用户的学习门槛。此外，项目采用开源协作模式，社区活跃且内容持续更新，确保学习者能获取前沿且准确的技术资讯。如果你正寻找一条清晰、友好且专业的机器学习入门之路，ML-For-Beginners 将是理想的起点。",84991,"2026-04-05T10:45:23",[14,34,35,36,15,37,38,13,39],"数据工具","视频","插件","其他","语言模型","音频",{"id":41,"name":42,"github_repo":43,"description_zh":44,"stars":45,"difficulty_score":10,"last_commit_at":46,"category_tags":47,"status":16},3128,"ragflow","infiniflow\u002Fragflow","RAGFlow 是一款领先的开源检索增强生成（RAG）引擎，旨在为大语言模型构建更精准、可靠的上下文层。它巧妙地将前沿的 RAG 技术与智能体（Agent）能力相结合，不仅支持从各类文档中高效提取知识，还能让模型基于这些知识进行逻辑推理和任务执行。\n\n在大模型应用中，幻觉问题和知识滞后是常见痛点。RAGFlow 通过深度解析复杂文档结构（如表格、图表及混合排版），显著提升了信息检索的准确度，从而有效减少模型“胡编乱造”的现象，确保回答既有据可依又具备时效性。其内置的智能体机制更进一步，使系统不仅能回答问题，还能自主规划步骤解决复杂问题。\n\n这款工具特别适合开发者、企业技术团队以及 AI 研究人员使用。无论是希望快速搭建私有知识库问答系统，还是致力于探索大模型在垂直领域落地的创新者，都能从中受益。RAGFlow 提供了可视化的工作流编排界面和灵活的 API 接口，既降低了非算法背景用户的上手门槛，也满足了专业开发者对系统深度定制的需求。作为基于 Apache 2.0 协议开源的项目，它正成为连接通用大模型与行业专有知识之间的重要桥梁。",77062,"2026-04-04T04:44:48",[15,14,13,38,37],{"id":49,"name":50,"github_repo":51,"description_zh":52,"stars":53,"difficulty_score":10,"last_commit_at":54,"category_tags":55,"status":16},519,"PaddleOCR","PaddlePaddle\u002FPaddleOCR","PaddleOCR 是一款基于百度飞桨框架开发的高性能开源光学字符识别工具包。它的核心能力是将图片、PDF 等文档中的文字提取出来，转换成计算机可读取的结构化数据，让机器真正“看懂”图文内容。\n\n面对海量纸质或电子文档，PaddleOCR 解决了人工录入效率低、数字化成本高的问题。尤其在人工智能领域，它扮演着连接图像与大型语言模型（LLM）的桥梁角色，能将视觉信息直接转化为文本输入，助力智能问答、文档分析等应用场景落地。\n\nPaddleOCR 适合开发者、算法研究人员以及有文档自动化需求的普通用户。其技术优势十分明显：不仅支持全球 100 多种语言的识别，还能在 Windows、Linux、macOS 等多个系统上运行，并灵活适配 CPU、GPU、NPU 等各类硬件。作为一个轻量级且社区活跃的开源项目，PaddleOCR 既能满足快速集成的需求，也能支撑前沿的视觉语言研究，是处理文字识别任务的理想选择。",74913,"2026-04-05T10:44:17",[38,14,13,37],{"id":57,"name":58,"github_repo":59,"description_zh":60,"stars":61,"difficulty_score":23,"last_commit_at":62,"category_tags":63,"status":16},2471,"tesseract","tesseract-ocr\u002Ftesseract","Tesseract 是一款历史悠久且备受推崇的开源光学字符识别（OCR）引擎，最初由惠普实验室开发，后由 Google 维护，目前由全球社区共同贡献。它的核心功能是将图片中的文字转化为可编辑、可搜索的文本数据，有效解决了从扫描件、照片或 PDF 文档中提取文字信息的难题，是数字化归档和信息自动化的重要基础工具。\n\n在技术层面，Tesseract 展现了强大的适应能力。从版本 4 开始，它引入了基于长短期记忆网络（LSTM）的神经网络 OCR 引擎，显著提升了行识别的准确率；同时，为了兼顾旧有需求，它依然支持传统的字符模式识别引擎。Tesseract 原生支持 UTF-8 编码，开箱即用即可识别超过 100 种语言，并兼容 PNG、JPEG、TIFF 等多种常见图像格式。输出方面，它灵活支持纯文本、hOCR、PDF、TSV 等多种格式，方便后续数据处理。\n\nTesseract 主要面向开发者、研究人员以及需要构建文档处理流程的企业用户。由于它本身是一个命令行工具和库（libtesseract），不包含图形用户界面（GUI），因此最适合具备一定编程能力的技术人员集成到自动化脚本或应用程序中",73286,"2026-04-03T01:56:45",[13,14],{"id":65,"github_repo":66,"name":67,"description_en":68,"description_zh":69,"ai_summary_zh":69,"readme_en":70,"readme_zh":71,"quickstart_zh":72,"use_case_zh":73,"hero_image_url":74,"owner_login":75,"owner_name":76,"owner_avatar_url":77,"owner_bio":78,"owner_company":79,"owner_location":80,"owner_email":80,"owner_twitter":80,"owner_website":81,"owner_url":82,"languages":83,"stars":104,"forks":105,"last_commit_at":106,"license":107,"difficulty_score":10,"env_os":108,"env_gpu":109,"env_ram":110,"env_deps":111,"category_tags":125,"github_topics":126,"view_count":10,"oss_zip_url":80,"oss_zip_packed_at":80,"status":16,"created_at":130,"updated_at":131,"faqs":132,"releases":161},440,"Peterande\u002FD-FINE","D-FINE","D-FINE: Redefine Regression Task of DETRs as Fine-grained Distribution Refinement  [ICLR 2025 Spotlight]","D-FINE 是一个高效实时的目标检测模型，专为提升 DETR（Detection Transformer）系列模型的定位精度而设计。它将传统边界框回归任务重新定义为“细粒度分布优化”（Fine-grained Distribution Refinement, FDR），通过更精细地建模预测坐标的概率分布，显著提升了小目标和模糊边缘的检测准确性。同时，D-FINE 引入了“全局最优定位自蒸馏”（GO-LSD）机制，在不增加推理开销和训练成本的前提下，进一步优化模型性能。在 COCO 数据集上，D-FINE 在速度与精度之间取得了优异平衡，达到当前实时目标检测的领先水平。该模型特别适合计算机视觉领域的研究人员和算法工程师使用，尤其适用于对检测精度和推理效率有较高要求的场景，如自动驾驶、视频监控等。普通用户可通过 Hugging Face Spaces 快速体验其效果，但深度使用建议具备一定的深度学习基础。","\u003C!--# [D-FINE: Redefine Regression Task of DETRs as Fine-grained Distribution Refinement](https:\u002F\u002Farxiv.org\u002Fabs\u002Fxxxxxx) -->\n\nEnglish | [简体中文](README_cn.md) | [日本語](README_ja.md) | [English Blog](src\u002Fzoo\u002Fdfine\u002Fblog.md) | [中文博客](src\u002Fzoo\u002Fdfine\u002Fblog_cn.md)\n\n\u003Ch2 align=\"center\">\n  D-FINE: Redefine Regression Task of DETRs as Fine‑grained Distribution Refinement\n\u003C\u002Fh2>\n\n\n\n\u003Cp align=\"center\">\n    \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fdeveloper0hye\u002FD-FINE\">\n        \u003Cimg alt=\"hf\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F%F0%9F%A4%97%20Hugging%20Face-Spaces-blue\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\u002Fblob\u002Fmaster\u002FLICENSE\">\n        \u003Cimg alt=\"license\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLICENSE-Apache%202.0-blue\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\u002Fpulls\">\n        \u003Cimg alt=\"prs\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fissues-pr\u002FPeterande\u002FD-FINE\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\u002Fissues\">\n        \u003Cimg alt=\"issues\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fissues\u002FPeterande\u002FD-FINE?color=olive\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.13842\">\n        \u003Cimg alt=\"arXiv\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FarXiv-2410.13842-red\">\n    \u003C\u002Fa>\n\u003C!--     \u003Ca href=\"mailto: pengyansong@mail.ustc.edu.cn\">\n        \u003Cimg alt=\"email\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fcontact_me-email-yellow\">\n    \u003C\u002Fa> -->\n      \u003Ca href=\"https:\u002F\u002Fresults.pre-commit.ci\u002Flatest\u002Fgithub\u002FPeterande\u002FD-FINE\u002Fmaster\">\n        \u003Cimg alt=\"pre-commit.ci status\" src=\"https:\u002F\u002Fresults.pre-commit.ci\u002Fbadge\u002Fgithub\u002FPeterande\u002FD-FINE\u002Fmaster.svg\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\">\n        \u003Cimg alt=\"stars\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fstars\u002FPeterande\u002FD-FINE\">\n    \u003C\u002Fa>\n\u003C\u002Fp>\n\n\n\n\u003Cp align=\"center\">\n    📄 This is the official implementation of the paper:\n    \u003Cbr>\n    \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.13842\">D-FINE: Redefine Regression Task of DETRs as Fine-grained Distribution Refinement\u003C\u002Fa>\n\u003C\u002Fp>\n\n\n\n\u003Cp align=\"center\">\nYansong Peng, Hebei Li, Peixi Wu, Yueyi Zhang, Xiaoyan Sun, and Feng Wu\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\nUniversity of Science and Technology of China\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n    \u003Ca href=\"https:\u002F\u002Fpaperswithcode.com\u002Fsota\u002Freal-time-object-detection-on-coco?p=d-fine-redefine-regression-task-in-detrs-as\">\n        \u003Cimg alt=\"sota\" src=\"https:\u002F\u002Fimg.shields.io\u002Fendpoint.svg?url=https:\u002F\u002Fpaperswithcode.com\u002Fbadge\u002Fd-fine-redefine-regression-task-in-detrs-as\u002Freal-time-object-detection-on-coco\">\n    \u003C\u002Fa>\n\u003C\u002Fp>\n\n\u003C!-- \u003Ctable>\u003Ctr>\n\u003Ctd>\u003Cimg src=https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Fblob\u002Fmaster\u002Flatency.png border=0 width=333>\u003C\u002Ftd>\n\u003Ctd>\u003Cimg src=https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Fblob\u002Fmaster\u002Fparams.png border=0 width=333>\u003C\u002Ftd>\n\u003Ctd>\u003Cimg src=https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Fblob\u002Fmaster\u002Fflops.png border=0 width=333>\u003C\u002Ftd>\n\u003C\u002Ftr>\u003C\u002Ftable> -->\n\n\u003Cp align=\"center\">\n\u003Cstrong>If you like D-FINE, please give us a ⭐! Your support motivates us to keep improving!\u003C\u002Fstrong>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n    \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FPeterande_D-FINE_readme_367523e9164d.png\" width=\"1000\">\n\u003C\u002Fp>\n\nD-FINE is a powerful real-time object detector that redefines the bounding box regression task in DETRs as Fine-grained Distribution Refinement (FDR) and introduces Global Optimal Localization Self-Distillation (GO-LSD), achieving outstanding performance without introducing additional inference and training costs.\n\n\u003Cdetails open>\n\u003Csummary> Video \u003C\u002Fsummary>\n\nWe conduct object detection using D-FINE and YOLO11 on a complex street scene video from [YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=CfhEWj9sd9A). Despite challenging conditions such as backlighting, motion blur, and dense crowds, D-FINE-X successfully detects nearly all targets, including subtle small objects like backpacks, bicycles, and traffic lights. Its confidence scores and the localization precision for blurred edges are significantly higher than those of YOLO11.\n\n\u003C!-- We use D-FINE and YOLO11 on a street scene video from [YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=CfhEWj9sd9A). Despite challenges like backlighting, motion blur, and dense crowds, D-FINE-X outperforms YOLO11x, detecting more objects with higher confidence and better precision. -->\n\nhttps:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002Fe5933d8e-3c8a-400e-870b-4e452f5321d9\n\n\u003C\u002Fdetails>\n\n## 🚀 Updates\n- [x] **\\[2024.10.18\\]** Release D-FINE series.\n- [x] **\\[2024.10.25\\]** Add custom dataset finetuning configs ([#7](https:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\u002Fissues\u002F7)).\n- [x] **\\[2024.10.30\\]** Update D-FINE-L (E25) pretrained model, with performance improved by 2.0%.\n- [x] **\\[2024.11.07\\]** Release **D-FINE-N**, achiving 42.8% AP\u003Csup>val\u003C\u002Fsup> on COCO @ 472 FPS\u003Csup>T4\u003C\u002Fsup>!\n\n## Model Zoo\n\n### COCO\n| Model | Dataset | AP\u003Csup>val\u003C\u002Fsup> | #Params | Latency | GFLOPs | config | checkpoint | logs |\n| :---: | :---: | :---: |  :---: | :---: | :---: | :---: | :---: | :---: |\n**D‑FINE‑N** | COCO | **42.8** | 4M | 2.12ms | 7 | [yml](.\u002Fconfigs\u002Fdfine\u002Fdfine_hgnetv2_n_coco.yml) | [42.8](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_n_coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fcoco\u002Fdfine_n_coco_log.txt)\n**D‑FINE‑S** | COCO | **48.5** | 10M | 3.49ms | 25 | [yml](.\u002Fconfigs\u002Fdfine\u002Fdfine_hgnetv2_s_coco.yml) | [48.5](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_s_coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fcoco\u002Fdfine_s_coco_log.txt)\n**D‑FINE‑M** | COCO | **52.3** | 19M | 5.62ms | 57 | [yml](.\u002Fconfigs\u002Fdfine\u002Fdfine_hgnetv2_m_coco.yml) | [52.3](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_m_coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fcoco\u002Fdfine_m_coco_log.txt)\n**D‑FINE‑L** | COCO | **54.0** | 31M | 8.07ms | 91 | [yml](.\u002Fconfigs\u002Fdfine\u002Fdfine_hgnetv2_l_coco.yml) | [54.0](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_l_coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fcoco\u002Fdfine_l_coco_log.txt)\n**D‑FINE‑X** | COCO | **55.8** | 62M | 12.89ms | 202 | [yml](.\u002Fconfigs\u002Fdfine\u002Fdfine_hgnetv2_x_coco.yml) | [55.8](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_x_coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fcoco\u002Fdfine_x_coco_log.txt)\n\n\n### Objects365+COCO\n| Model | Dataset | AP\u003Csup>val\u003C\u002Fsup> | #Params | Latency | GFLOPs | config | checkpoint | logs |\n| :---: | :---: | :---: |  :---: | :---: | :---: | :---: | :---: | :---: |\n**D‑FINE‑S** | Objects365+COCO | **50.7** | 10M | 3.49ms | 25 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_s_obj2coco.yml) | [50.7](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_s_obj2coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj2coco\u002Fdfine_s_obj2coco_log.txt)\n**D‑FINE‑M** | Objects365+COCO | **55.1** | 19M | 5.62ms | 57 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_m_obj2coco.yml) | [55.1](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_m_obj2coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj2coco\u002Fdfine_m_obj2coco_log.txt)\n**D‑FINE‑L** | Objects365+COCO | **57.3** | 31M | 8.07ms | 91 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_l_obj2coco.yml) | [57.3](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_l_obj2coco_e25.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj2coco\u002Fdfine_l_obj2coco_log_e25.txt)\n**D‑FINE‑X** | Objects365+COCO | **59.3** | 62M | 12.89ms | 202 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_x_obj2coco.yml) | [59.3](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_x_obj2coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj2coco\u002Fdfine_x_obj2coco_log.txt)\n\n**We highly recommend that you use the Objects365 pre-trained model for fine-tuning:**\n\n⚠️ **Important**: Please note that this is generally beneficial for complex scene understanding. If your categories are very simple, it might lead to overfitting and suboptimal performance.\n\u003Cdetails>\n\u003Csummary>\u003Cstrong> 🔥 Pretrained Models on Objects365 (Best generalization) \u003C\u002Fstrong>\u003C\u002Fsummary>\n\n| Model | Dataset | AP\u003Csup>val\u003C\u002Fsup> | AP\u003Csup>5000\u003C\u002Fsup> | #Params | Latency | GFLOPs | config | checkpoint | logs |\n| :---: | :---: | :---: |  :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n**D‑FINE‑S** | Objects365 | **31.0** | **30.5** | 10M | 3.49ms | 25 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_s_obj365.yml) | [30.5](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_s_obj365.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj365\u002Fdfine_s_obj365_log.txt)\n**D‑FINE‑M** | Objects365 | **38.6** | **37.4** | 19M | 5.62ms | 57 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_m_obj365.yml) | [37.4](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_m_obj365.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj365\u002Fdfine_m_obj365_log.txt)\n**D‑FINE‑L** | Objects365 | - | **40.6** | 31M | 8.07ms | 91 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_l_obj365.yml) | [40.6](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_l_obj365.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj365\u002Fdfine_l_obj365_log.txt)\n**D‑FINE‑L (E25)** | Objects365 | **44.7** | **42.6** | 31M | 8.07ms | 91 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_l_obj365.yml) | [42.6](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_l_obj365_e25.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj365\u002Fdfine_l_obj365_log_e25.txt)\n**D‑FINE‑X** | Objects365 | **49.5** | **46.5** | 62M | 12.89ms | 202 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_x_obj365.yml) | [46.5](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_x_obj365.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj365\u002Fdfine_x_obj365_log.txt)\n- **E25**: Re-trained and extended the pretraining to 25 epochs.\n- **AP\u003Csup>val\u003C\u002Fsup>** is evaluated on *Objects365* full validation set.\n- **AP\u003Csup>5000\u003C\u002Fsup>** is evaluated on the first 5000 samples of the *Objects365* validation set.\n\u003C\u002Fdetails>\n\n**Notes:**\n- **AP\u003Csup>val\u003C\u002Fsup>** is evaluated on *MSCOCO val2017* dataset.\n- **Latency** is evaluated on a single T4 GPU with $batch\\\\_size = 1$, $fp16$, and $TensorRT==10.4.0$.\n- **Objects365+COCO** means finetuned model on *COCO* using pretrained weights trained on *Objects365*.\n\n\n\n## Quick start\n\n### Setup\n\n```shell\nconda create -n dfine python=3.11.9\nconda activate dfine\npip install -r requirements.txt\n```\n\n\n### Data Preparation\n\n\u003Cdetails>\n\u003Csummary> COCO2017 Dataset \u003C\u002Fsummary>\n\n1. Download COCO2017 from [OpenDataLab](https:\u002F\u002Fopendatalab.com\u002FOpenDataLab\u002FCOCO_2017) or [COCO](https:\u002F\u002Fcocodataset.org\u002F#download).\n1. Modify paths in [coco_detection.yml](.\u002Fconfigs\u002Fdataset\u002Fcoco_detection.yml)\n\n    ```yaml\n    train_dataloader:\n        img_folder: \u002Fdata\u002FCOCO2017\u002Ftrain2017\u002F\n        ann_file: \u002Fdata\u002FCOCO2017\u002Fannotations\u002Finstances_train2017.json\n    val_dataloader:\n        img_folder: \u002Fdata\u002FCOCO2017\u002Fval2017\u002F\n        ann_file: \u002Fdata\u002FCOCO2017\u002Fannotations\u002Finstances_val2017.json\n    ```\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Objects365 Dataset \u003C\u002Fsummary>\n\n1. Download Objects365 from [OpenDataLab](https:\u002F\u002Fopendatalab.com\u002FOpenDataLab\u002FObjects365).\n\n2. Set the Base Directory:\n```shell\nexport BASE_DIR=\u002Fdata\u002FObjects365\u002Fdata\n```\n\n3. Extract and organize the downloaded files, resulting directory structure:\n\n```shell\n${BASE_DIR}\u002Ftrain\n├── images\n│   ├── v1\n│   │   ├── patch0\n│   │   │   ├── 000000000.jpg\n│   │   │   ├── 000000001.jpg\n│   │   │   └── ... (more images)\n│   ├── v2\n│   │   ├── patchx\n│   │   │   ├── 000000000.jpg\n│   │   │   ├── 000000001.jpg\n│   │   │   └── ... (more images)\n├── zhiyuan_objv2_train.json\n```\n\n```shell\n${BASE_DIR}\u002Fval\n├── images\n│   ├── v1\n│   │   ├── patch0\n│   │   │   ├── 000000000.jpg\n│   │   │   └── ... (more images)\n│   ├── v2\n│   │   ├── patchx\n│   │   │   ├── 000000000.jpg\n│   │   │   └── ... (more images)\n├── zhiyuan_objv2_val.json\n```\n\n4. Create a New Directory to Store Images from the Validation Set:\n```shell\nmkdir -p ${BASE_DIR}\u002Ftrain\u002Fimages_from_val\n```\n\n5. Copy the v1 and v2 folders from the val directory into the train\u002Fimages_from_val directory\n```shell\ncp -r ${BASE_DIR}\u002Fval\u002Fimages\u002Fv1 ${BASE_DIR}\u002Ftrain\u002Fimages_from_val\u002F\ncp -r ${BASE_DIR}\u002Fval\u002Fimages\u002Fv2 ${BASE_DIR}\u002Ftrain\u002Fimages_from_val\u002F\n```\n\n6. Run remap_obj365.py to merge a subset of the validation set into the training set. Specifically, this script moves samples with indices between 5000 and 800000 from the validation set to the training set.\n```shell\npython tools\u002Fremap_obj365.py --base_dir ${BASE_DIR}\n```\n\n\n7. Run the resize_obj365.py script to resize any images in the dataset where the maximum edge length exceeds 640 pixels. Use the updated JSON file generated in Step 5 to process the sample data. Ensure that you resize images in both the train and val datasets to maintain consistency.\n```shell\npython tools\u002Fresize_obj365.py --base_dir ${BASE_DIR}\n```\n\n8. Modify paths in [obj365_detection.yml](.\u002Fconfigs\u002Fdataset\u002Fobj365_detection.yml)\n\n    ```yaml\n    train_dataloader:\n        img_folder: \u002Fdata\u002FObjects365\u002Fdata\u002Ftrain\n        ann_file: \u002Fdata\u002FObjects365\u002Fdata\u002Ftrain\u002Fnew_zhiyuan_objv2_train_resized.json\n    val_dataloader:\n        img_folder: \u002Fdata\u002FObjects365\u002Fdata\u002Fval\u002F\n        ann_file: \u002Fdata\u002FObjects365\u002Fdata\u002Fval\u002Fnew_zhiyuan_objv2_val_resized.json\n    ```\n\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>CrowdHuman\u003C\u002Fsummary>\n\nDownload COCO format dataset here: [url](https:\u002F\u002Faistudio.baidu.com\u002Fdatasetdetail\u002F231455)\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>Custom Dataset\u003C\u002Fsummary>\n\nTo train on your custom dataset, you need to organize it in the COCO format. Follow the steps below to prepare your dataset:\n\n1. **Set `remap_mscoco_category` to `False`:**\n\n    This prevents the automatic remapping of category IDs to match the MSCOCO categories.\n\n    ```yaml\n    remap_mscoco_category: False\n    ```\n\n2. **Organize Images:**\n\n    Structure your dataset directories as follows:\n\n    ```shell\n    dataset\u002F\n    ├── images\u002F\n    │   ├── train\u002F\n    │   │   ├── image1.jpg\n    │   │   ├── image2.jpg\n    │   │   └── ...\n    │   ├── val\u002F\n    │   │   ├── image1.jpg\n    │   │   ├── image2.jpg\n    │   │   └── ...\n    └── annotations\u002F\n        ├── instances_train.json\n        ├── instances_val.json\n        └── ...\n    ```\n\n    - **`images\u002Ftrain\u002F`**: Contains all training images.\n    - **`images\u002Fval\u002F`**: Contains all validation images.\n    - **`annotations\u002F`**: Contains COCO-formatted annotation files.\n\n3. **Convert Annotations to COCO Format:**\n\n    If your annotations are not already in COCO format, you'll need to convert them. You can use the following Python script as a reference or utilize existing tools:\n\n    ```python\n    import json\n\n    def convert_to_coco(input_annotations, output_annotations):\n        # Implement conversion logic here\n        pass\n\n    if __name__ == \"__main__\":\n        convert_to_coco('path\u002Fto\u002Fyour_annotations.json', 'dataset\u002Fannotations\u002Finstances_train.json')\n    ```\n\n4. **Update Configuration Files:**\n\n    Modify your [custom_detection.yml](.\u002Fconfigs\u002Fdataset\u002Fcustom_detection.yml).\n\n    ```yaml\n    task: detection\n\n    evaluator:\n      type: CocoEvaluator\n      iou_types: ['bbox', ]\n\n    num_classes: 777 # your dataset classes\n    remap_mscoco_category: False\n\n    train_dataloader:\n      type: DataLoader\n      dataset:\n        type: CocoDetection\n        img_folder: \u002Fdata\u002Fyourdataset\u002Ftrain\n        ann_file: \u002Fdata\u002Fyourdataset\u002Ftrain\u002Ftrain.json\n        return_masks: False\n        transforms:\n          type: Compose\n          ops: ~\n      shuffle: True\n      num_workers: 4\n      drop_last: True\n      collate_fn:\n        type: BatchImageCollateFunction\n\n    val_dataloader:\n      type: DataLoader\n      dataset:\n        type: CocoDetection\n        img_folder: \u002Fdata\u002Fyourdataset\u002Fval\n        ann_file: \u002Fdata\u002Fyourdataset\u002Fval\u002Fann.json\n        return_masks: False\n        transforms:\n          type: Compose\n          ops: ~\n      shuffle: False\n      num_workers: 4\n      drop_last: False\n      collate_fn:\n        type: BatchImageCollateFunction\n    ```\n\n\u003C\u002Fdetails>\n\n\n## Usage\n\u003Cdetails open>\n\u003Csummary> COCO2017 \u003C\u002Fsummary>\n\n\u003C!-- \u003Csummary>1. Training \u003C\u002Fsummary> -->\n1. Set Model\n```shell\nexport model=l  # n s m l x\n```\n\n2. Training\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml --use-amp --seed=0\n```\n\n\u003C!-- \u003Csummary>2. Testing \u003C\u002Fsummary> -->\n3. Testing\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml --test-only -r model.pth\n```\n\n\u003C!-- \u003Csummary>3. Tuning \u003C\u002Fsummary> -->\n4. Tuning\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml --use-amp --seed=0 -t model.pth\n```\n\u003C\u002Fdetails>\n\n\n\u003Cdetails>\n\u003Csummary> Objects365 to COCO2017 \u003C\u002Fsummary>\n\n1. Set Model\n```shell\nexport model=l  # n s m l x\n```\n\n2. Training on Objects365\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_${model}_obj365.yml --use-amp --seed=0\n```\n\n3. Tuning on COCO2017\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_${model}_obj2coco.yml --use-amp --seed=0 -t model.pth\n```\n\n\u003C!-- \u003Csummary>2. Testing \u003C\u002Fsummary> -->\n4. Testing\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml --test-only -r model.pth\n```\n\u003C\u002Fdetails>\n\n\n\u003Cdetails>\n\u003Csummary> Custom Dataset \u003C\u002Fsummary>\n\n1. Set Model\n```shell\nexport model=l  # n s m l x\n```\n\n2. Training on Custom Dataset\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fcustom\u002Fdfine_hgnetv2_${model}_custom.yml --use-amp --seed=0\n```\n\u003C!-- \u003Csummary>2. Testing \u003C\u002Fsummary> -->\n3. Testing\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fcustom\u002Fdfine_hgnetv2_${model}_custom.yml --test-only -r model.pth\n```\n\n4. Tuning on Custom Dataset\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fcustom\u002Fobjects365\u002Fdfine_hgnetv2_${model}_obj2custom.yml --use-amp --seed=0 -t model.pth\n```\n\n5. **[Optional]** Modify Class Mappings:\n\nWhen using the Objects365 pre-trained weights to train on your custom dataset, the example assumes that your dataset only contains the classes `'Person'` and `'Car'`. For faster convergence, you can modify `self.obj365_ids` in `src\u002Fsolver\u002F_solver.py` as follows:\n\n\n```python\nself.obj365_ids = [0, 5]  # Person, Cars\n```\nYou can replace these with any corresponding classes from your dataset. The list of Objects365 classes with their corresponding IDs:\nhttps:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\u002Fblob\u002F352a94ece291e26e1957df81277bef00fe88a8e3\u002Fsrc\u002Fsolver\u002F_solver.py#L330\n\nNew training command:\n\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fcustom\u002Fdfine_hgnetv2_${model}_custom.yml --use-amp --seed=0 -t model.pth\n```\n\nHowever, if you don't wish to modify the class mappings, the pre-trained Objects365 weights will still work without any changes. Modifying the class mappings is optional and can potentially accelerate convergence for specific tasks.\n\n\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Customizing Batch Size \u003C\u002Fsummary>\n\nFor example, if you want to double the total batch size when training D-FINE-L on COCO2017, here are the steps you should follow:\n\n1. **Modify your [dataloader.yml](.\u002Fconfigs\u002Fdfine\u002Finclude\u002Fdataloader.yml)** to increase the `total_batch_size`:\n\n    ```yaml\n    train_dataloader:\n        total_batch_size: 64  # Previously it was 32, now doubled\n    ```\n\n2. **Modify your [dfine_hgnetv2_l_coco.yml](.\u002Fconfigs\u002Fdfine\u002Fdfine_hgnetv2_l_coco.yml)**. Here’s how the key parameters should be adjusted:\n\n    ```yaml\n    optimizer:\n    type: AdamW\n    params:\n        -\n        params: '^(?=.*backbone)(?!.*norm|bn).*$'\n        lr: 0.000025  # doubled, linear scaling law\n        -\n        params: '^(?=.*(?:encoder|decoder))(?=.*(?:norm|bn)).*$'\n        weight_decay: 0.\n\n    lr: 0.0005  # doubled, linear scaling law\n    betas: [0.9, 0.999]\n    weight_decay: 0.0001  # need a grid search\n\n    ema:  # added EMA settings\n        decay: 0.9998  # adjusted by 1 - (1 - decay) * 2\n        warmups: 500  # halved\n\n    lr_warmup_scheduler:\n        warmup_duration: 250  # halved\n    ```\n\n\u003C\u002Fdetails>\n\n\n\u003Cdetails>\n\u003Csummary> Customizing Input Size \u003C\u002Fsummary>\n\nIf you'd like to train **D-FINE-L** on COCO2017 with an input size of 320x320, follow these steps:\n\n1. **Modify your [dataloader.yml](.\u002Fconfigs\u002Fdfine\u002Finclude\u002Fdataloader.yml)**:\n\n    ```yaml\n\n    train_dataloader:\n    dataset:\n        transforms:\n            ops:\n                - {type: Resize, size: [320, 320], }\n    collate_fn:\n        base_size: 320\n    dataset:\n        transforms:\n            ops:\n                - {type: Resize, size: [320, 320], }\n    ```\n\n2. **Modify your [dfine_hgnetv2.yml](.\u002Fconfigs\u002Fdfine\u002Finclude\u002Fdfine_hgnetv2.yml)**:\n\n    ```yaml\n    eval_spatial_size: [320, 320]\n    ```\n\n\u003C\u002Fdetails>\n\n## Tools\n\u003Cdetails>\n\u003Csummary> Deployment \u003C\u002Fsummary>\n\n\u003C!-- \u003Csummary>4. Export onnx \u003C\u002Fsummary> -->\n1. Setup\n```shell\npip install onnx onnxsim\nexport model=l  # n s m l x\n```\n\n2. Export onnx\n```shell\npython tools\u002Fdeployment\u002Fexport_onnx.py --check -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml -r model.pth\n```\n\n3. Export [tensorrt](https:\u002F\u002Fdocs.nvidia.com\u002Fdeeplearning\u002Ftensorrt\u002Finstall-guide\u002Findex.html)\n```shell\ntrtexec --onnx=\"model.onnx\" --saveEngine=\"model.engine\" --fp16\n```\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Inference (Visualization) \u003C\u002Fsummary>\n\n\n1. Setup\n```shell\npip install -r tools\u002Finference\u002Frequirements.txt\nexport model=l  # n s m l x\n```\n\n\n\u003C!-- \u003Csummary>5. Inference \u003C\u002Fsummary> -->\n2. Inference (onnxruntime \u002F tensorrt \u002F torch)\n\nInference on images and videos is now supported.\n```shell\npython tools\u002Finference\u002Fonnx_inf.py --onnx model.onnx --input image.jpg  # video.mp4\npython tools\u002Finference\u002Ftrt_inf.py --trt model.engine --input image.jpg\npython tools\u002Finference\u002Ftorch_inf.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml -r model.pth --input image.jpg --device cuda:0\n```\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Benchmark \u003C\u002Fsummary>\n\n1. Setup\n```shell\npip install -r tools\u002Fbenchmark\u002Frequirements.txt\nexport model=l  # n s m l x\n```\n\n\u003C!-- \u003Csummary>6. Benchmark \u003C\u002Fsummary> -->\n2. Model FLOPs, MACs, and Params\n```shell\npython tools\u002Fbenchmark\u002Fget_info.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml\n```\n\n2. TensorRT Latency\n```shell\npython tools\u002Fbenchmark\u002Ftrt_benchmark.py --COCO_dir path\u002Fto\u002FCOCO2017 --engine_dir model.engine\n```\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Fiftyone Visualization  \u003C\u002Fsummary>\n\n1. Setup\n```shell\npip install fiftyone\nexport model=l  # n s m l x\n```\n4. Voxel51 Fiftyone Visualization ([fiftyone](https:\u002F\u002Fgithub.com\u002Fvoxel51\u002Ffiftyone))\n```shell\npython tools\u002Fvisualization\u002Ffiftyone_vis.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml -r model.pth\n```\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Others \u003C\u002Fsummary>\n\n1. Auto Resume Training\n```shell\nbash reference\u002Fsafe_training.sh\n```\n\n2. Converting Model Weights\n```shell\npython reference\u002Fconvert_weight.py model.pth\n```\n\u003C\u002Fdetails>\n\n## Figures and Visualizations\n\n\u003Cdetails>\n\u003Csummary> FDR and GO-LSD \u003C\u002Fsummary>\n\n1. Overview of D-FINE with FDR. The probability distributions that act as a more fine-\ngrained intermediate representation are iteratively refined by the decoder layers in a residual manner.\nNon-uniform weighting functions are applied to allow for finer localization.\n\n\u003Cp align=\"center\">\n    \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FPeterande_D-FINE_readme_77e7127792f4.jpg\" alt=\"Fine-grained Distribution Refinement Process\" width=\"1000\">\n\u003C\u002Fp>\n\n2. Overview of GO-LSD process. Localization knowledge from the final layer’s refined\ndistributions is distilled into earlier layers through DDF loss with decoupled weighting strategies.\n\n\u003Cp align=\"center\">\n    \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FPeterande_D-FINE_readme_9f0f31a28f42.jpg\" alt=\"GO-LSD Process\" width=\"1000\">\n\u003C\u002Fp>\n\n\u003C\u002Fdetails>\n\n\u003Cdetails open>\n\u003Csummary> Distributions \u003C\u002Fsummary>\n\nVisualizations of FDR across detection scenarios with initial and refined bounding boxes, along with unweighted and weighted distributions.\n\n\u003Cp align=\"center\">\n    \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FPeterande_D-FINE_readme_fd6968d14724.jpg\" width=\"1000\">\n\u003C\u002Fp>\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Hard Cases \u003C\u002Fsummary>\n\nThe following visualization demonstrates D-FINE's predictions in various complex detection scenarios. These include cases with occlusion, low-light conditions, motion blur, depth of field effects, and densely populated scenes. Despite these challenges, D-FINE consistently produces accurate localization results.\n\n\u003Cp align=\"center\">\n    \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FPeterande_D-FINE_readme_2e5074d0b535.jpg\" alt=\"D-FINE Predictions in Challenging Scenarios\" width=\"1000\">\n\u003C\u002Fp>\n\n\u003C\u002Fdetails>\n\n\n\u003C!-- \u003Cdiv style=\"display: flex; flex-wrap: wrap; justify-content: center; margin: 0; padding: 0;\">\n    \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FPeterande_D-FINE_readme_fd6968d14724.jpg\" style=\"width:99.96%; margin: 0; padding: 0;\" \u002F>\n\u003C\u002Fdiv>\n\n\u003Ctable>\u003Ctr>\n\u003Ctd>\u003Cimg src=https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FPeterande_D-FINE_readme_fd6968d14724.jpg border=0 width=1000>\u003C\u002Ftd>\n\u003C\u002Ftr>\u003C\u002Ftable> -->\n\n\n\n\n## Citation\nIf you use `D-FINE` or its methods in your work, please cite the following BibTeX entries:\n\u003Cdetails open>\n\u003Csummary> bibtex \u003C\u002Fsummary>\n\n```latex\n@misc{peng2024dfine,\n      title={D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement},\n      author={Yansong Peng and Hebei Li and Peixi Wu and Yueyi Zhang and Xiaoyan Sun and Feng Wu},\n      year={2024},\n      eprint={2410.13842},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV}\n}\n```\n\u003C\u002Fdetails>\n\n## Acknowledgement\nOur work is built upon [RT-DETR](https:\u002F\u002Fgithub.com\u002Flyuwenyu\u002FRT-DETR).\nThanks to the inspirations from [RT-DETR](https:\u002F\u002Fgithub.com\u002Flyuwenyu\u002FRT-DETR), [GFocal](https:\u002F\u002Fgithub.com\u002Fimplus\u002FGFocal), [LD](https:\u002F\u002Fgithub.com\u002FHikariTJU\u002FLD), and [YOLOv9](https:\u002F\u002Fgithub.com\u002FWongKinYiu\u002Fyolov9).\n\n✨ Feel free to contribute and reach out if you have any questions! ✨\n","\u003C!--# [D-FINE: 将 DETR 的回归任务重新定义为细粒度分布精炼（Fine-grained Distribution Refinement）](https:\u002F\u002Farxiv.org\u002Fabs\u002Fxxxxxx) -->\n\n[English](README.md) | 简体中文 | [日本語](README_ja.md) | [English Blog](src\u002Fzoo\u002Fdfine\u002Fblog.md) | [中文博客](src\u002Fzoo\u002Fdfine\u002Fblog_cn.md)\n\n\u003Ch2 align=\"center\">\n  D-FINE: 将 DETR 的回归任务重新定义为细粒度分布精炼（Fine‑grained Distribution Refinement）\n\u003C\u002Fh2>\n\n\n\n\u003Cp align=\"center\">\n    \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fdeveloper0hye\u002FD-FINE\">\n        \u003Cimg alt=\"hf\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F%F0%9F%A4%97%20Hugging%20Face-Spaces-blue\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\u002Fblob\u002Fmaster\u002FLICENSE\">\n        \u003Cimg alt=\"license\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLICENSE-Apache%202.0-blue\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\u002Fpulls\">\n        \u003Cimg alt=\"prs\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fissues-pr\u002FPeterande\u002FD-FINE\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\u002Fissues\">\n        \u003Cimg alt=\"issues\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fissues\u002FPeterande\u002FD-FINE?color=olive\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.13842\">\n        \u003Cimg alt=\"arXiv\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FarXiv-2410.13842-red\">\n    \u003C\u002Fa>\n\u003C!--     \u003Ca href=\"mailto: pengyansong@mail.ustc.edu.cn\">\n        \u003Cimg alt=\"email\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fcontact_me-email-yellow\">\n    \u003C\u002Fa> -->\n      \u003Ca href=\"https:\u002F\u002Fresults.pre-commit.ci\u002Flatest\u002Fgithub\u002FPeterande\u002FD-FINE\u002Fmaster\">\n        \u003Cimg alt=\"pre-commit.ci status\" src=\"https:\u002F\u002Fresults.pre-commit.ci\u002Fbadge\u002Fgithub\u002FPeterande\u002FD-FINE\u002Fmaster.svg\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\">\n        \u003Cimg alt=\"stars\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fstars\u002FPeterande\u002FD-FINE\">\n    \u003C\u002Fa>\n\u003C\u002Fp>\n\n\n\n\u003Cp align=\"center\">\n    📄 这是论文的官方实现：\n    \u003Cbr>\n    \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.13842\">D-FINE: 将 DETR 的回归任务重新定义为细粒度分布精炼（Fine-grained Distribution Refinement）\u003C\u002Fa>\n\u003C\u002Fp>\n\n\n\n\u003Cp align=\"center\">\n彭彦松、李河北、吴培曦、张悦怡、孙晓燕、吴枫\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n中国科学技术大学\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n    \u003Ca href=\"https:\u002F\u002Fpaperswithcode.com\u002Fsota\u002Freal-time-object-detection-on-coco?p=d-fine-redefine-regression-task-in-detrs-as\">\n        \u003Cimg alt=\"sota\" src=\"https:\u002F\u002Fimg.shields.io\u002Fendpoint.svg?url=https:\u002F\u002Fpaperswithcode.com\u002Fbadge\u002Fd-fine-redefine-regression-task-in-detrs-as\u002Freal-time-object-detection-on-coco\">\n    \u003C\u002Fa>\n\u003C\u002Fp>\n\n\u003C!-- \u003Ctable>\u003Ctr>\n\u003Ctd>\u003Cimg src=https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Fblob\u002Fmaster\u002Flatency.png border=0 width=333>\u003C\u002Ftd>\n\u003Ctd>\u003Cimg src=https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Fblob\u002Fmaster\u002Fparams.png border=0 width=333>\u003C\u002Ftd>\n\u003Ctd>\u003Cimg src=https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Fblob\u002Fmaster\u002Fflops.png border=0 width=333>\u003C\u002Ftd>\n\u003C\u002Ftr>\u003C\u002Ftable> -->\n\n\u003Cp align=\"center\">\n\u003Cstrong>如果您喜欢 D-FINE，请给我们一个 ⭐！您的支持将激励我们持续改进！\u003C\u002Fstrong>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n    \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FPeterande_D-FINE_readme_367523e9164d.png\" width=\"1000\">\n\u003C\u002Fp>\n\nD-FINE 是一种强大的实时目标检测器，它将 DETR 中的边界框回归任务重新定义为**细粒度分布精炼**（Fine-grained Distribution Refinement, FDR），并引入了**全局最优定位自蒸馏**（Global Optimal Localization Self-Distillation, GO-LSD），在不增加额外推理和训练开销的前提下实现了卓越的性能。\n\n\u003Cdetails open>\n\u003Csummary> 视频演示 \u003C\u002Fsummary>\n\n我们在来自 [YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=CfhEWj9sd9A) 的复杂街景视频上使用 D-FINE 和 YOLO11 进行目标检测。尽管面临逆光、运动模糊和人群密集等挑战，D-FINE-X 成功检测到了几乎所有目标，包括背包、自行车和交通灯等细微的小物体。其置信度分数以及对模糊边缘的定位精度显著优于 YOLO11。\n\n\u003C!-- We use D-FINE and YOLO11 on a street scene video from [YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=CfhEWj9sd9A). Despite challenges like backlighting, motion blur, and dense crowds, D-FINE-X outperforms YOLO11x, detecting more objects with higher confidence and better precision. -->\n\nhttps:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002Fe5933d8e-3c8a-400e-870b-4e452f5321d9\n\n\u003C\u002Fdetails>\n\n## 🚀 更新日志\n- [x] **\\[2024.10.18\\]** 发布 D-FINE 系列模型。\n- [x] **\\[2024.10.25\\]** 添加自定义数据集微调配置 ([#7](https:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\u002Fissues\u002F7))。\n- [x] **\\[2024.10.30\\]** 更新 D-FINE-L (E25) 预训练模型，性能提升 2.0%。\n- [x] **\\[2024.11.07\\]** 发布 **D-FINE-N**，在 COCO 上达到 42.8% AP\u003Csup>val\u003C\u002Fsup> @ 472 FPS\u003Csup>T4\u003C\u002Fsup>！\n\n## 模型库（Model Zoo）\n\n### COCO\n| 模型 | 数据集 | AP\u003Csup>val\u003C\u002Fsup> | 参数量 | 延迟 | GFLOPs | 配置文件 | 检查点 | 日志 |\n| :---: | :---: | :---: |  :---: | :---: | :---: | :---: | :---: | :---: |\n**D‑FINE‑N** | COCO | **42.8** | 4M | 2.12ms | 7 | [yml](.\u002Fconfigs\u002Fdfine\u002Fdfine_hgnetv2_n_coco.yml) | [42.8](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_n_coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fcoco\u002Fdfine_n_coco_log.txt)\n**D‑FINE‑S** | COCO | **48.5** | 10M | 3.49ms | 25 | [yml](.\u002Fconfigs\u002Fdfine\u002Fdfine_hgnetv2_s_coco.yml) | [48.5](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_s_coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fcoco\u002Fdfine_s_coco_log.txt)\n**D‑FINE‑M** | COCO | **52.3** | 19M | 5.62ms | 57 | [yml](.\u002Fconfigs\u002Fdfine\u002Fdfine_hgnetv2_m_coco.yml) | [52.3](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_m_coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fcoco\u002Fdfine_m_coco_log.txt)\n**D‑FINE‑L** | COCO | **54.0** | 31M | 8.07ms | 91 | [yml](.\u002Fconfigs\u002Fdfine\u002Fdfine_hgnetv2_l_coco.yml) | [54.0](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_l_coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fcoco\u002Fdfine_l_coco_log.txt)\n**D‑FINE‑X** | COCO | **55.8** | 62M | 12.89ms | 202 | [yml](.\u002Fconfigs\u002Fdfine\u002Fdfine_hgnetv2_x_coco.yml) | [55.8](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_x_coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fcoco\u002Fdfine_x_coco_log.txt)\n\n### Objects365+COCO\n| Model | Dataset | AP\u003Csup>val\u003C\u002Fsup> | #Params | Latency | GFLOPs | config | checkpoint | logs |\n| :---: | :---: | :---: |  :---: | :---: | :---: | :---: | :---: | :---: |\n**D‑FINE‑S** | Objects365+COCO | **50.7** | 10M | 3.49ms | 25 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_s_obj2coco.yml) | [50.7](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_s_obj2coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj2coco\u002Fdfine_s_obj2coco_log.txt)\n**D‑FINE‑M** | Objects365+COCO | **55.1** | 19M | 5.62ms | 57 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_m_obj2coco.yml) | [55.1](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_m_obj2coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj2coco\u002Fdfine_m_obj2coco_log.txt)\n**D‑FINE‑L** | Objects365+COCO | **57.3** | 31M | 8.07ms | 91 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_l_obj2coco.yml) | [57.3](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_l_obj2coco_e25.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj2coco\u002Fdfine_l_obj2coco_log_e25.txt)\n**D‑FINE‑X** | Objects365+COCO | **59.3** | 62M | 12.89ms | 202 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_x_obj2coco.yml) | [59.3](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_x_obj2coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj2coco\u002Fdfine_x_obj2coco_log.txt)\n\n**We highly recommend that you use the Objects365 pre-trained model for fine-tuning:**\n\n⚠️ **Important**: Please note that this is generally beneficial for complex scene understanding. If your categories are very simple, it might lead to overfitting and suboptimal performance.\n\u003Cdetails>\n\u003Csummary>\u003Cstrong> 🔥 Pretrained Models on Objects365 (Best generalization) \u003C\u002Fstrong>\u003C\u002Fsummary>\n\n| Model | Dataset | AP\u003Csup>val\u003C\u002Fsup> | AP\u003Csup>5000\u003C\u002Fsup> | #Params | Latency | GFLOPs | config | checkpoint | logs |\n| :---: | :---: | :---: |  :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n**D‑FINE‑S** | Objects365 | **31.0** | **30.5** | 10M | 3.49ms | 25 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_s_obj365.yml) | [30.5](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_s_obj365.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj365\u002Fdfine_s_obj365_log.txt)\n**D‑FINE‑M** | Objects365 | **38.6** | **37.4** | 19M | 5.62ms | 57 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_m_obj365.yml) | [37.4](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_m_obj365.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj365\u002Fdfine_m_obj365_log.txt)\n**D‑FINE‑L** | Objects365 | - | **40.6** | 31M | 8.07ms | 91 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_l_obj365.yml) | [40.6](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_l_obj365.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj365\u002Fdfine_l_obj365_log.txt)\n**D‑FINE‑L (E25)** | Objects365 | **44.7** | **42.6** | 31M | 8.07ms | 91 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_l_obj365.yml) | [42.6](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_l_obj365_e25.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj365\u002Fdfine_l_obj365_log_e25.txt)\n**D‑FINE‑X** | Objects365 | **49.5** | **46.5** | 62M | 12.89ms | 202 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_x_obj365.yml) | [46.5](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_x_obj365.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj365\u002Fdfine_x_obj365_log.txt)\n- **E25**: Re-trained and extended the pretraining to 25 epochs.\n- **AP\u003Csup>val\u003C\u002Fsup>** is evaluated on *Objects365* full validation set.\n- **AP\u003Csup>5000\u003C\u002Fsup>** is evaluated on the first 5000 samples of the *Objects365* validation set.\n\u003C\u002Fdetails>\n\n**Notes:**\n- **AP\u003Csup>val\u003C\u002Fsup>** is evaluated on *MSCOCO val2017* dataset.\n- **Latency** is evaluated on a single T4 GPU with $batch\\\\_size = 1$, $fp16$, and $TensorRT==10.4.0$.\n- **Objects365+COCO** means finetuned model on *COCO* using pretrained weights trained on *Objects365*.\n\n\n\n## Quick start\n\n### Setup\n\n```shell\nconda create -n dfine python=3.11.9\nconda activate dfine\npip install -r requirements.txt\n```\n\n### Data Preparation\n\n\u003Cdetails>\n\u003Csummary> COCO2017 Dataset \u003C\u002Fsummary>\n\n1. Download COCO2017 from [OpenDataLab](https:\u002F\u002Fopendatalab.com\u002FOpenDataLab\u002FCOCO_2017) or [COCO](https:\u002F\u002Fcocodataset.org\u002F#download).\n1. Modify paths in [coco_detection.yml](.\u002Fconfigs\u002Fdataset\u002Fcoco_detection.yml)\n\n    ```yaml\n    train_dataloader:\n        img_folder: \u002Fdata\u002FCOCO2017\u002Ftrain2017\u002F\n        ann_file: \u002Fdata\u002FCOCO2017\u002Fannotations\u002Finstances_train2017.json\n    val_dataloader:\n        img_folder: \u002Fdata\u002FCOCO2017\u002Fval2017\u002F\n        ann_file: \u002Fdata\u002FCOCO2017\u002Fannotations\u002Finstances_val2017.json\n    ```\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Objects365 Dataset \u003C\u002Fsummary>\n\n1. Download Objects365 from [OpenDataLab](https:\u002F\u002Fopendatalab.com\u002FOpenDataLab\u002FObjects365).\n\n2. Set the Base Directory:\n```shell\nexport BASE_DIR=\u002Fdata\u002FObjects365\u002Fdata\n```\n\n3. Extract and organize the downloaded files, resulting directory structure:\n\n```shell\n${BASE_DIR}\u002Ftrain\n├── images\n│   ├── v1\n│   │   ├── patch0\n│   │   │   ├── 000000000.jpg\n│   │   │   ├── 000000001.jpg\n│   │   │   └── ... (more images)\n│   ├── v2\n│   │   ├── patchx\n│   │   │   ├── 000000000.jpg\n│   │   │   ├── 000000001.jpg\n│   │   │   └── ... (more images)\n├── zhiyuan_objv2_train.json\n```\n\n```shell\n${BASE_DIR}\u002Fval\n├── images\n│   ├── v1\n│   │   ├── patch0\n│   │   │   ├── 000000000.jpg\n│   │   │   └── ... (more images)\n│   ├── v2\n│   │   ├── patchx\n│   │   │   ├── 000000000.jpg\n│   │   │   └── ... (more images)\n├── zhiyuan_objv2_val.json\n```\n\n4. Create a New Directory to Store Images from the Validation Set:\n```shell\nmkdir -p ${BASE_DIR}\u002Ftrain\u002Fimages_from_val\n```\n\n5. Copy the v1 and v2 folders from the val directory into the train\u002Fimages_from_val directory\n```shell\ncp -r ${BASE_DIR}\u002Fval\u002Fimages\u002Fv1 ${BASE_DIR}\u002Ftrain\u002Fimages_from_val\u002F\ncp -r ${BASE_DIR}\u002Fval\u002Fimages\u002Fv2 ${BASE_DIR}\u002Ftrain\u002Fimages_from_val\u002F\n```\n\n6. Run remap_obj365.py to merge a subset of the validation set into the training set. Specifically, this script moves samples with indices between 5000 and 800000 from the validation set to the training set.\n```shell\npython tools\u002Fremap_obj365.py --base_dir ${BASE_DIR}\n```\n\n\n7. Run the resize_obj365.py script to resize any images in the dataset where the maximum edge length exceeds 640 pixels. Use the updated JSON file generated in Step 5 to process the sample data. Ensure that you resize images in both the train and val datasets to maintain consistency.\n```shell\npython tools\u002Fresize_obj365.py --base_dir ${BASE_DIR}\n```\n\n8. Modify paths in [obj365_detection.yml](.\u002Fconfigs\u002Fdataset\u002Fobj365_detection.yml)\n\n    ```yaml\n    train_dataloader:\n        img_folder: \u002Fdata\u002FObjects365\u002Fdata\u002Ftrain\n        ann_file: \u002Fdata\u002FObjects365\u002Fdata\u002Ftrain\u002Fnew_zhiyuan_objv2_train_resized.json\n    val_dataloader:\n        img_folder: \u002Fdata\u002FObjects365\u002Fdata\u002Fval\u002F\n        ann_file: \u002Fdata\u002FObjects365\u002Fdata\u002Fval\u002Fnew_zhiyuan_objv2_val_resized.json\n    ```\n\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>CrowdHuman\u003C\u002Fsummary>\n\nDownload COCO format dataset here: [url](https:\u002F\u002Faistudio.baidu.com\u002Fdatasetdetail\u002F231455)\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>Custom Dataset\u003C\u002Fsummary>\n\nTo train on your custom dataset, you need to organize it in the COCO format. Follow the steps below to prepare your dataset:\n\n1. **Set `remap_mscoco_category` to `False`:**\n\n    This prevents the automatic remapping of category IDs to match the MSCOCO categories.\n\n    ```yaml\n    remap_mscoco_category: False\n    ```\n\n2. **Organize Images:**\n\n    Structure your dataset directories as follows:\n\n    ```shell\n    dataset\u002F\n    ├── images\u002F\n    │   ├── train\u002F\n    │   │   ├── image1.jpg\n    │   │   ├── image2.jpg\n    │   │   └── ...\n    │   ├── val\u002F\n    │   │   ├── image1.jpg\n    │   │   ├── image2.jpg\n    │   │   └── ...\n    └── annotations\u002F\n        ├── instances_train.json\n        ├── instances_val.json\n        └── ...\n    ```\n\n    - **`images\u002Ftrain\u002F`**: Contains all training images.\n    - **`images\u002Fval\u002F`**: Contains all validation images.\n    - **`annotations\u002F`**: Contains COCO-formatted annotation files.\n\n3. **Convert Annotations to COCO Format:**\n\n    If your annotations are not already in COCO format, you'll need to convert them. You can use the following Python script as a reference or utilize existing tools:\n\n    ```python\n    import json\n\n    def convert_to_coco(input_annotations, output_annotations):\n        # Implement conversion logic here\n        pass\n\n    if __name__ == \"__main__\":\n        convert_to_coco('path\u002Fto\u002Fyour_annotations.json', 'dataset\u002Fannotations\u002Finstances_train.json')\n    ```\n\n4. **Update Configuration Files:**\n\n    Modify your [custom_detection.yml](.\u002Fconfigs\u002Fdataset\u002Fcustom_detection.yml).\n\n    ```yaml\n    task: detection\n\n    evaluator:\n      type: CocoEvaluator\n      iou_types: ['bbox', ]\n\n    num_classes: 777 # your dataset classes\n    remap_mscoco_category: False\n\n    train_dataloader:\n      type: DataLoader\n      dataset:\n        type: CocoDetection\n        img_folder: \u002Fdata\u002Fyourdataset\u002Ftrain\n        ann_file: \u002Fdata\u002Fyourdataset\u002Ftrain\u002Ftrain.json\n        return_masks: False\n        transforms:\n          type: Compose\n          ops: ~\n      shuffle: True\n      num_workers: 4\n      drop_last: True\n      collate_fn:\n        type: BatchImageCollateFunction\n\n    val_dataloader:\n      type: DataLoader\n      dataset:\n        type: CocoDetection\n        img_folder: \u002Fdata\u002Fyourdataset\u002Fval\n        ann_file: \u002Fdata\u002Fyourdataset\u002Fval\u002Fann.json\n        return_masks: False\n        transforms:\n          type: Compose\n          ops: ~\n      shuffle: False\n      num_workers: 4\n      drop_last: False\n      collate_fn:\n        type: BatchImageCollateFunction\n    ```\n\n\u003C\u002Fdetails>\n\n## Usage\n\u003Cdetails open>\n\u003Csummary> COCO2017 \u003C\u002Fsummary>\n\n\u003C!-- \u003Csummary>1. Training \u003C\u002Fsummary> -->\n1. Set Model\n```shell\nexport model=l  # n s m l x\n```\n\n2. Training\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml --use-amp --seed=0\n```\n\n\u003C!-- \u003Csummary>2. Testing \u003C\u002Fsummary> -->\n3. Testing\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml --test-only -r model.pth\n```\n\n\u003C!-- \u003Csummary>3. Tuning \u003C\u002Fsummary> -->\n4. Tuning\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml --use-amp --seed=0 -t model.pth\n```\n\u003C\u002Fdetails>\n\n\n\u003Cdetails>\n\u003Csummary> Objects365 to COCO2017 \u003C\u002Fsummary>\n\n1. Set Model\n```shell\nexport model=l  # n s m l x\n```\n\n2. Training on Objects365\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_${model}_obj365.yml --use-amp --seed=0\n```\n\n3. Tuning on COCO2017\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_${model}_obj2coco.yml --use-amp --seed=0 -t model.pth\n```\n\n\u003C!-- \u003Csummary>2. Testing \u003C\u002Fsummary> -->\n4. Testing\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml --test-only -r model.pth\n```\n\u003C\u002Fdetails>\n\n\n\u003Cdetails>\n\u003Csummary> Custom Dataset \u003C\u002Fsummary>\n\n1. Set Model\n```shell\nexport model=l  # n s m l x\n```\n\n2. Training on Custom Dataset\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fcustom\u002Fdfine_hgnetv2_${model}_custom.yml --use-amp --seed=0\n```\n\u003C!-- \u003Csummary>2. Testing \u003C\u002Fsummary> -->\n3. Testing\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fcustom\u002Fdfine_hgnetv2_${model}_custom.yml --test-only -r model.pth\n```\n\n4. Tuning on Custom Dataset\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fcustom\u002Fobjects365\u002Fdfine_hgnetv2_${model}_obj2custom.yml --use-amp --seed=0 -t model.pth\n```\n\n5. **[Optional]** Modify Class Mappings:\n\nWhen using the Objects365 pre-trained weights to train on your custom dataset, the example assumes that your dataset only contains the classes `'Person'` and `'Car'`. For faster convergence, you can modify `self.obj365_ids` in `src\u002Fsolver\u002F_solver.py` as follows:\n\n\n```python\nself.obj365_ids = [0, 5]  # Person, Cars\n```\nYou can replace these with any corresponding classes from your dataset. The list of Objects365 classes with their corresponding IDs:\nhttps:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\u002Fblob\u002F352a94ece291e26e1957df81277bef00fe88a8e3\u002Fsrc\u002Fsolver\u002F_solver.py#L330\n\nNew training command:\n\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fcustom\u002Fdfine_hgnetv2_${model}_custom.yml --use-amp --seed=0 -t model.pth\n```\n\nHowever, if you don't wish to modify the class mappings, the pre-trained Objects365 weights will still work without any changes. Modifying the class mappings is optional and can potentially accelerate convergence for specific tasks.\n\n\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Customizing Batch Size \u003C\u002Fsummary>\n\nFor example, if you want to double the total batch size when training D-FINE-L on COCO2017, here are the steps you should follow:\n\n1. **Modify your [dataloader.yml](.\u002Fconfigs\u002Fdfine\u002Finclude\u002Fdataloader.yml)** to increase the `total_batch_size`:\n\n    ```yaml\n    train_dataloader:\n        total_batch_size: 64  # Previously it was 32, now doubled\n    ```\n\n2. **Modify your [dfine_hgnetv2_l_coco.yml](.\u002Fconfigs\u002Fdfine\u002Fdfine_hgnetv2_l_coco.yml)**. Here’s how the key parameters should be adjusted:\n\n    ```yaml\n    optimizer:\n    type: AdamW\n    params:\n        -\n        params: '^(?=.*backbone)(?!.*norm|bn).*$'\n        lr: 0.000025  # doubled, linear scaling law\n        -\n        params: '^(?=.*(?:encoder|decoder))(?=.*(?:norm|bn)).*$'\n        weight_decay: 0.\n\n    lr: 0.0005  # doubled, linear scaling law\n    betas: [0.9, 0.999]\n    weight_decay: 0.0001  # need a grid search\n\n    ema:  # added EMA settings\n        decay: 0.9998  # adjusted by 1 - (1 - decay) * 2\n        warmups: 500  # halved\n\n    lr_warmup_scheduler:\n        warmup_duration: 250  # halved\n    ```\n\n\u003C\u002Fdetails>\n\n\n\u003Cdetails>\n\u003Csummary> Customizing Input Size \u003C\u002Fsummary>\n\nIf you'd like to train **D-FINE-L** on COCO2017 with an input size of 320x320, follow these steps:\n\n1. **Modify your [dataloader.yml](.\u002Fconfigs\u002Fdfine\u002Finclude\u002Fdataloader.yml)**:\n\n    ```yaml\n\n    train_dataloader:\n    dataset:\n        transforms:\n            ops:\n                - {type: Resize, size: [320, 320], }\n    collate_fn:\n        base_size: 320\n    dataset:\n        transforms:\n            ops:\n                - {type: Resize, size: [320, 320], }\n    ```\n\n2. **Modify your [dfine_hgnetv2.yml](.\u002Fconfigs\u002Fdfine\u002Finclude\u002Fdfine_hgnetv2.yml)**:\n\n    ```yaml\n    eval_spatial_size: [320, 320]\n    ```\n\n\u003C\u002Fdetails>\n\n## Tools\n\u003Cdetails>\n\u003Csummary> Deployment \u003C\u002Fsummary>\n\n\u003C!-- \u003Csummary>4. Export onnx \u003C\u002Fsummary> -->\n1. Setup\n```shell\npip install onnx onnxsim\nexport model=l  # n s m l x\n```\n\n2. Export onnx\n```shell\npython tools\u002Fdeployment\u002Fexport_onnx.py --check -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml -r model.pth\n```\n\n3. Export [tensorrt](https:\u002F\u002Fdocs.nvidia.com\u002Fdeeplearning\u002Ftensorrt\u002Finstall-guide\u002Findex.html)\n```shell\ntrtexec --onnx=\"model.onnx\" --saveEngine=\"model.engine\" --fp16\n```\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Inference (Visualization) \u003C\u002Fsummary>\n\n\n1. Setup\n```shell\npip install -r tools\u002Finference\u002Frequirements.txt\nexport model=l  # n s m l x\n```\n\n\n\u003C!-- \u003Csummary>5. Inference \u003C\u002Fsummary> -->\n2. Inference (onnxruntime \u002F tensorrt \u002F torch)\n\nInference on images and videos is now supported.\n```shell\npython tools\u002Finference\u002Fonnx_inf.py --onnx model.onnx --input image.jpg  # video.mp4\npython tools\u002Finference\u002Ftrt_inf.py --trt model.engine --input image.jpg\npython tools\u002Finference\u002Ftorch_inf.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml -r model.pth --input image.jpg --device cuda:0\n```\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Benchmark \u003C\u002Fsummary>\n\n1. Setup\n```shell\npip install -r tools\u002Fbenchmark\u002Frequirements.txt\nexport model=l  # n s m l x\n```\n\n\u003C!-- \u003Csummary>6. Benchmark \u003C\u002Fsummary> -->\n2. Model FLOPs, MACs, and Params\n```shell\npython tools\u002Fbenchmark\u002Fget_info.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml\n```\n\n2. TensorRT Latency\n```shell\npython tools\u002Fbenchmark\u002Ftrt_benchmark.py --COCO_dir path\u002Fto\u002FCOCO2017 --engine_dir model.engine\n```\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Fiftyone Visualization  \u003C\u002Fsummary>\n\n1. Setup\n```shell\npip install fiftyone\nexport model=l  # n s m l x\n```\n4. Voxel51 Fiftyone Visualization ([fiftyone](https:\u002F\u002Fgithub.com\u002Fvoxel51\u002Ffiftyone))\n```shell\npython tools\u002Fvisualization\u002Ffiftyone_vis.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml -r model.pth\n```\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Others \u003C\u002Fsummary>\n\n1. Auto Resume Training\n```shell\nbash reference\u002Fsafe_training.sh\n```\n\n2. Converting Model Weights\n```shell\npython reference\u002Fconvert_weight.py model.pth\n```\n\u003C\u002Fdetails>\n\n## Figures and Visualizations\n\n\u003Cdetails>\n\u003Csummary> FDR and GO-LSD \u003C\u002Fsummary>\n\n1. Overview of D-FINE with FDR. The probability distributions that act as a more fine-\ngrained intermediate representation are iteratively refined by the decoder layers in a residual manner.\nNon-uniform weighting functions are applied to allow for finer localization.\n\n\u003Cp align=\"center\">\n    \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FPeterande_D-FINE_readme_77e7127792f4.jpg\" alt=\"Fine-grained Distribution Refinement Process\" width=\"1000\">\n\u003C\u002Fp>\n\n2. Overview of GO-LSD process. Localization knowledge from the final layer’s refined\ndistributions is distilled into earlier layers through DDF loss with decoupled weighting strategies.\n\n\u003Cp align=\"center\">\n    \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FPeterande_D-FINE_readme_9f0f31a28f42.jpg\" alt=\"GO-LSD Process\" width=\"1000\">\n\u003C\u002Fp>\n\n\u003C\u002Fdetails>\n\n\u003Cdetails open>\n\u003Csummary> Distributions \u003C\u002Fsummary>\n\nVisualizations of FDR across detection scenarios with initial and refined bounding boxes, along with unweighted and weighted distributions.\n\n\u003Cp align=\"center\">\n    \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FPeterande_D-FINE_readme_fd6968d14724.jpg\" width=\"1000\">\n\u003C\u002Fp>\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Hard Cases \u003C\u002Fsummary>\n\nThe following visualization demonstrates D-FINE's predictions in various complex detection scenarios. These include cases with occlusion, low-light conditions, motion blur, depth of field effects, and densely populated scenes. Despite these challenges, D-FINE consistently produces accurate localization results.\n\n\u003Cp align=\"center\">\n    \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FPeterande_D-FINE_readme_2e5074d0b535.jpg\" alt=\"D-FINE Predictions in Challenging Scenarios\" width=\"1000\">\n\u003C\u002Fp>\n\n\u003C\u002Fdetails>\n\n\n\u003C!-- \u003Cdiv style=\"display: flex; flex-wrap: wrap; justify-content: center; margin: 0; padding: 0;\">\n    \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FPeterande_D-FINE_readme_fd6968d14724.jpg\" style=\"width:99.96%; margin: 0; padding: 0;\" \u002F>\n\u003C\u002Fdiv>\n\n\u003Ctable>\u003Ctr>\n\u003Ctd>\u003Cimg src=https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FPeterande_D-FINE_readme_fd6968d14724.jpg border=0 width=1000>\u003C\u002Ftd>\n\u003C\u002Ftr>\u003C\u002Ftable> -->\n\n\n\n\n## Citation\nIf you use `D-FINE` or its methods in your work, please cite the following BibTeX entries:\n\u003Cdetails open>\n\u003Csummary> bibtex \u003C\u002Fsummary>\n\n```latex\n@misc{peng2024dfine,\n      title={D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement},\n      author={Yansong Peng and Hebei Li and Peixi Wu and Yueyi Zhang and Xiaoyan Sun and Feng Wu},\n      year={2024},\n      eprint={2410.13842},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV}\n}\n```\n\u003C\u002Fdetails>\n\n## Acknowledgement\nOur work is built upon [RT-DETR](https:\u002F\u002Fgithub.com\u002Flyuwenyu\u002FRT-DETR).\nThanks to the inspirations from [RT-DETR](https:\u002F\u002Fgithub.com\u002Flyuwenyu\u002FRT-DETR), [GFocal](https:\u002F\u002Fgithub.com\u002Fimplus\u002FGFocal), [LD](https:\u002F\u002Fgithub.com\u002FHikariTJU\u002FLD), and [YOLOv9](https:\u002F\u002Fgithub.com\u002FWongKinYiu\u002Fyolov9).\n\n✨ Feel free to contribute and reach out if you have any questions! ✨","# D-FINE 快速上手指南\n\nD-FINE 是一个强大的实时目标检测器，它通过将 DETRs 中的回归任务重新定义为细粒度分布细化（FDR），并引入全局最优定位自蒸馏（GO-LSD），在未增加额外推理和训练成本的情况下实现了卓越的性能。\n\n## 1. 环境准备\n\n*   **操作系统**：Linux（推荐）\n*   **Python 版本**：3.11.9\n*   **依赖管理**：Conda\n*   **硬件要求**：建议使用 NVIDIA GPU（模型延迟测试基于 T4 GPU）\n\n## 2. 安装步骤\n\n通过 Conda 创建独立环境并安装依赖：\n\n```shell\nconda create -n dfine python=3.11.9\nconda activate dfine\npip install -r requirements.txt\n```\n\n## 3. 基本使用\n\n### 3.1 数据准备\n\n**COCO2017 数据集**\n\n1.  **下载数据**：推荐从国内镜像源 [OpenDataLab](https:\u002F\u002Fopendatalab.com\u002FOpenDataLab\u002FCOCO_2017) 或 [COCO 官网](https:\u002F\u002Fcocodataset.org\u002F#download) 下载数据集。\n2.  **配置路径**：修改 `configs\u002Fdataset\u002Fcoco_detection.yml` 文件中的数据路径：\n\n    ```yaml\n    train_dataloader:\n        img_folder: \u002Fdata\u002FCOCO2017\u002Ftrain2017\u002F\n        ann_file: \u002Fdata\u002FCOCO2017\u002Fannotations\u002Finstances_train2017.json\n    val_dataloader:\n        img_folder: \u002Fdata\u002FCOCO2017\u002Fval2017\u002F\n        ann_file: \u002Fdata\u002FCOCO2017\u002Fannotations\u002Finstances_val2017.json\n    ```\n\n**Objects365 数据集**\n\n1.  **下载数据**：从 [OpenDataLab](https:\u002F\u002Fopendatalab.com\u002FOpenDataLab\u002FObjects365) 下载。\n2.  **设置路径**：设置基础目录环境变量：\n\n    ```shell\n    export BASE_DIR=\u002Fdata\u002FObjects365\u002Fdata\n    ```\n\n### 3.2 模型选择与下载\n\n项目提供了在 COCO 和 Objects365 上预训练的多种规格模型（N\u002FS\u002FM\u002FL\u002FX）。\n\n*   **推荐模型**：如果需要在自定义数据集上微调，推荐使用 **Objects365 预训练模型**，它们具有更好的泛化能力。\n*   **下载地址**：请前往项目的 [Model Zoo](#model-zoo) 部分获取对应模型的权重链接（如 `dfine_l_coco.pth`）。\n\n### 3.3 配置文件\n\n训练和评估需要指定对应的配置文件（`.yml`），配置文件位于 `configs\u002Fdfine\u002F` 目录下。例如：\n\n*   COCO 配置：`.\u002Fconfigs\u002Fdfine\u002Fdfine_hgnetv2_l_coco.yml`\n*   Objects365 配置：`.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_l_obj365.yml`","某智能交通系统开发团队正在部署城市道路监控方案，需在复杂光照条件和高密度车流中实时检测车辆、行人及交通标志。团队原采用YOLO系列模型，但在实际测试中发现多项性能瓶颈。\n\n### 没有 D-FINE 时\n- **小目标漏检严重**：交通标志（如限速牌）因尺寸小、纹理简单，在逆光或雨雾天气下常被忽略\n- **复杂场景精度下降**：早高峰时段车流密集区域，模型对遮挡车辆的定位误差率高达18%\n- **推理延迟影响实时性**：4K分辨率视频流处理延迟达200ms，无法满足100ms内的实时响应要求\n- **边界定位模糊**：对运动模糊的电动车检测框存在15%的偏移量\n- **模型泛化能力弱**：从白天训练集迁移到夜间部署场景时，mAP值下降9.2个百分点\n\n### 使用 D-FINE 后\n- **小目标检测率提升**：通过细粒度分布优化，限速牌等微小目标的召回率提高至92.7%\n- **复杂场景鲁棒性增强**：GO-LSD机制使遮挡车辆的定位误差降低至5.3%，支持更密集的物体检测\n- **保持实时性的同时提升精度**：在相同硬件条件下，4K视频处理延迟稳定在85ms以内\n- **边缘定位更精准**：运动模糊场景下检测框偏移量控制在3%以内，显著提升轨迹预测准确性\n- **跨场景适应性优化**：夜间场景mAP仅下降2.1个百分点，通过分布细化机制实现更好的光照不变性\n\nD-FINE通过重构DETRs的回归范式，在不增加计算成本的前提下，实现了复杂交通场景下更精准、更鲁棒的实时目标检测，为智能交通系统提供了可靠的技术支撑。","https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002FPeterande_D-FINE_367523e9.png","Peterande","Yansong Peng","https:\u002F\u002Foss.gittoolsai.com\u002Favatars\u002FPeterande_386eafd3.png","Yes, a big brood like that. They need food. And shelter. And toys.","University of Science and Technology of China (USTC)",null,"https:\u002F\u002Fscholar.google.com\u002Fcitations?user=CTidez8AAAAJ","https:\u002F\u002Fgithub.com\u002FPeterande",[84,88,92,96,100],{"name":85,"color":86,"percentage":87},"Python","#3572A5",91.9,{"name":89,"color":90,"percentage":91},"C++","#f34b7d",6.5,{"name":93,"color":94,"percentage":95},"Shell","#89e051",0.6,{"name":97,"color":98,"percentage":99},"Dockerfile","#384d54",0.5,{"name":101,"color":102,"percentage":103},"CMake","#DA3434",0.4,3085,291,"2026-04-05T09:39:49","Apache-2.0","Linux, macOS","需要 NVIDIA GPU，显卡型号 T4 或更高，显存 8GB+，CUDA 11.7+","未说明",{"notes":112,"python":113,"dependencies":114},"需要安装 TensorRT 10.4.0 以实现最佳性能；训练和推理需下载预训练模型文件（约 5GB+）；数据集路径需手动配置；推荐使用 conda 环境管理","3.11.9",[115,116,117,118,119,120,121,122,123,124],"torch","torchvision","yacs","opencv-python","numpy","tqdm","tensorboard","pre-commit","omegaconf","hydra-core",[14],[127,128,129],"detr","object-detection","d-fine","2026-03-27T02:49:30.150509","2026-04-06T06:46:15.158371",[133,138,143,148,153,157],{"id":134,"question_zh":135,"answer_zh":136,"source_url":137},1688,"多卡训练时模型卡死或报 NCCL 通信超时错误怎么办？","这通常是因为某个 batch 中的标签全为空（例如包含负样本）导致的分布式训练死锁。可以在 `train_one_epoch` 函数中添加检查逻辑：检测所有 rank 是否存在空标签，如果存在则统一跳过该 batch。\n代码示例：\n```python\n# 当前 rank 是否标签全为空\nlocal_empty = torch.tensor(\n    int(all(len(t['boxes']) == 0 for t in targets)),\n    device=samples.device\n)\n# 将所有 rank 的 local_empty 相加\ndist.all_reduce(local_empty, op=dist.ReduceOp.SUM)\n# 只要有任何一个 rank 是空的，所有 rank 就统一 continue\nif local_empty.item() > 0:\n    if dist.get_rank() == 0:\n        print(f\"检测到至少一个 rank 的标签为空，全体跳过该 batch\")\n    continue\n```","https:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\u002Fissues\u002F39",{"id":139,"question_zh":140,"answer_zh":141,"source_url":142},1689,"TensorRT 转换 FP16 模型后精度严重下降（漏检严重）如何解决？","可以在 ONNX 转 Engine 时，将“归一化”层的精度强制保持为 FP32，其他层使用 FP16。这样可以在保证精度的同时提升速度。\nPython 代码示例：\n```python\nimport tensorrt as trt\nfor i in range(network.num_layers):\n    layer = network.get_layer(i)\n    if layer.type == trt.LayerType.NORMALIZATION:\n        layer.precision = trt.DataType.FLOAT\n```","https:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\u002Fissues\u002F44",{"id":144,"question_zh":145,"answer_zh":146,"source_url":147},1690,"训练时出现 'Image size exceeds limit' 警告或训练速度慢怎么解决？","这通常是因为数据集中存在超大像素图片（如 Objects365 数据集）。建议在训练前对图片进行预处理，将超大图片提前 resize 到合适大小（如 640x640）。虽然可以通过设置 `Image.MAX_IMAGE_PIXELS = None` 解除限制，但如果数据集包含损坏文件可能会报错，预处理是更稳妥的方案。","https:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\u002Fissues\u002F13",{"id":149,"question_zh":150,"answer_zh":151,"source_url":152},1691,"训练时 CPU 占用率高但 GPU 利用率低怎么办？","这种情况通常是由于数据加载瓶颈造成的。建议尝试调整 `num_workers` 和 `batch_size` 的组合，找到适合当前硬件配置的平衡点。在 Windows 系统上可能需要特别注意多进程配置。","https:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\u002Fissues\u002F23",{"id":154,"question_zh":155,"answer_zh":156,"source_url":152},1692,"Windows 下训练报错 'RuntimeError: Default process group has not been initialized' 怎么办？","这是 Windows 环境下分布式训练常见的配置问题。目前可以尝试调整 `num_workers` 和 `batch_size` 的组合来缓解，或者关注项目后续是否推出专门的 Windows 运行教程。",{"id":158,"question_zh":159,"answer_zh":160,"source_url":137},1693,"加入负样本（无 box 图片）后多卡训练卡死怎么解决？","加入负样本后，某些 batch 可能全为空标签，导致多卡训练同步卡死。可以通过修改数据加载逻辑跳过无 box 图片（但这会失去负样本作用），或者使用分布式训练中跳过空标签 batch 的代码方案来解决。",[]]