[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"similar-rafaelpadilla--Object-Detection-Metrics":3,"tool-rafaelpadilla--Object-Detection-Metrics":64},[4,17,27,35,43,56],{"id":5,"name":6,"github_repo":7,"description_zh":8,"stars":9,"difficulty_score":10,"last_commit_at":11,"category_tags":12,"status":16},3808,"stable-diffusion-webui","AUTOMATIC1111\u002Fstable-diffusion-webui","stable-diffusion-webui 是一个基于 Gradio 构建的网页版操作界面，旨在让用户能够轻松地在本地运行和使用强大的 Stable Diffusion 图像生成模型。它解决了原始模型依赖命令行、操作门槛高且功能分散的痛点，将复杂的 AI 绘图流程整合进一个直观易用的图形化平台。\n\n无论是希望快速上手的普通创作者、需要精细控制画面细节的设计师，还是想要深入探索模型潜力的开发者与研究人员，都能从中获益。其核心亮点在于极高的功能丰富度：不仅支持文生图、图生图、局部重绘（Inpainting）和外绘（Outpainting）等基础模式，还独创了注意力机制调整、提示词矩阵、负向提示词以及“高清修复”等高级功能。此外，它内置了 GFPGAN 和 CodeFormer 等人脸修复工具，支持多种神经网络放大算法，并允许用户通过插件系统无限扩展能力。即使是显存有限的设备，stable-diffusion-webui 也提供了相应的优化选项，让高质量的 AI 艺术创作变得触手可及。",162132,3,"2026-04-05T11:01:52",[13,14,15],"开发框架","图像","Agent","ready",{"id":18,"name":19,"github_repo":20,"description_zh":21,"stars":22,"difficulty_score":23,"last_commit_at":24,"category_tags":25,"status":16},1381,"everything-claude-code","affaan-m\u002Feverything-claude-code","everything-claude-code 是一套专为 AI 编程助手（如 Claude Code、Codex、Cursor 等）打造的高性能优化系统。它不仅仅是一组配置文件，而是一个经过长期实战打磨的完整框架，旨在解决 AI 代理在实际开发中面临的效率低下、记忆丢失、安全隐患及缺乏持续学习能力等核心痛点。\n\n通过引入技能模块化、直觉增强、记忆持久化机制以及内置的安全扫描功能，everything-claude-code 能显著提升 AI 在复杂任务中的表现，帮助开发者构建更稳定、更智能的生产级 AI 代理。其独特的“研究优先”开发理念和针对 Token 消耗的优化策略，使得模型响应更快、成本更低，同时有效防御潜在的攻击向量。\n\n这套工具特别适合软件开发者、AI 研究人员以及希望深度定制 AI 工作流的技术团队使用。无论您是在构建大型代码库，还是需要 AI 协助进行安全审计与自动化测试，everything-claude-code 都能提供强大的底层支持。作为一个曾荣获 Anthropic 黑客大奖的开源项目，它融合了多语言支持与丰富的实战钩子（hooks），让 AI 真正成长为懂上",140436,2,"2026-04-05T23:32:43",[13,15,26],"语言模型",{"id":28,"name":29,"github_repo":30,"description_zh":31,"stars":32,"difficulty_score":23,"last_commit_at":33,"category_tags":34,"status":16},2271,"ComfyUI","Comfy-Org\u002FComfyUI","ComfyUI 是一款功能强大且高度模块化的视觉 AI 引擎，专为设计和执行复杂的 Stable Diffusion 图像生成流程而打造。它摒弃了传统的代码编写模式，采用直观的节点式流程图界面，让用户通过连接不同的功能模块即可构建个性化的生成管线。\n\n这一设计巧妙解决了高级 AI 绘图工作流配置复杂、灵活性不足的痛点。用户无需具备编程背景，也能自由组合模型、调整参数并实时预览效果，轻松实现从基础文生图到多步骤高清修复等各类复杂任务。ComfyUI 拥有极佳的兼容性，不仅支持 Windows、macOS 和 Linux 全平台，还广泛适配 NVIDIA、AMD、Intel 及苹果 Silicon 等多种硬件架构，并率先支持 SDXL、Flux、SD3 等前沿模型。\n\n无论是希望深入探索算法潜力的研究人员和开发者，还是追求极致创作自由度的设计师与资深 AI 绘画爱好者，ComfyUI 都能提供强大的支持。其独特的模块化架构允许社区不断扩展新功能，使其成为当前最灵活、生态最丰富的开源扩散模型工具之一，帮助用户将创意高效转化为现实。",107662,"2026-04-03T11:11:01",[13,14,15],{"id":36,"name":37,"github_repo":38,"description_zh":39,"stars":40,"difficulty_score":23,"last_commit_at":41,"category_tags":42,"status":16},3704,"NextChat","ChatGPTNextWeb\u002FNextChat","NextChat 是一款轻量且极速的 AI 助手，旨在为用户提供流畅、跨平台的大模型交互体验。它完美解决了用户在多设备间切换时难以保持对话连续性，以及面对众多 AI 模型不知如何统一管理的痛点。无论是日常办公、学习辅助还是创意激发，NextChat 都能让用户随时随地通过网页、iOS、Android、Windows、MacOS 或 Linux 端无缝接入智能服务。\n\n这款工具非常适合普通用户、学生、职场人士以及需要私有化部署的企业团队使用。对于开发者而言，它也提供了便捷的自托管方案，支持一键部署到 Vercel 或 Zeabur 等平台。\n\nNextChat 的核心亮点在于其广泛的模型兼容性，原生支持 Claude、DeepSeek、GPT-4 及 Gemini Pro 等主流大模型，让用户在一个界面即可自由切换不同 AI 能力。此外，它还率先支持 MCP（Model Context Protocol）协议，增强了上下文处理能力。针对企业用户，NextChat 提供专业版解决方案，具备品牌定制、细粒度权限控制、内部知识库整合及安全审计等功能，满足公司对数据隐私和个性化管理的高标准要求。",87618,"2026-04-05T07:20:52",[13,26],{"id":44,"name":45,"github_repo":46,"description_zh":47,"stars":48,"difficulty_score":23,"last_commit_at":49,"category_tags":50,"status":16},2268,"ML-For-Beginners","microsoft\u002FML-For-Beginners","ML-For-Beginners 是由微软推出的一套系统化机器学习入门课程，旨在帮助零基础用户轻松掌握经典机器学习知识。这套课程将学习路径规划为 12 周，包含 26 节精炼课程和 52 道配套测验，内容涵盖从基础概念到实际应用的完整流程，有效解决了初学者面对庞大知识体系时无从下手、缺乏结构化指导的痛点。\n\n无论是希望转型的开发者、需要补充算法背景的研究人员，还是对人工智能充满好奇的普通爱好者，都能从中受益。课程不仅提供了清晰的理论讲解，还强调动手实践，让用户在循序渐进中建立扎实的技能基础。其独特的亮点在于强大的多语言支持，通过自动化机制提供了包括简体中文在内的 50 多种语言版本，极大地降低了全球不同背景用户的学习门槛。此外，项目采用开源协作模式，社区活跃且内容持续更新，确保学习者能获取前沿且准确的技术资讯。如果你正寻找一条清晰、友好且专业的机器学习入门之路，ML-For-Beginners 将是理想的起点。",84991,"2026-04-05T10:45:23",[14,51,52,53,15,54,26,13,55],"数据工具","视频","插件","其他","音频",{"id":57,"name":58,"github_repo":59,"description_zh":60,"stars":61,"difficulty_score":10,"last_commit_at":62,"category_tags":63,"status":16},3128,"ragflow","infiniflow\u002Fragflow","RAGFlow 是一款领先的开源检索增强生成（RAG）引擎，旨在为大语言模型构建更精准、可靠的上下文层。它巧妙地将前沿的 RAG 技术与智能体（Agent）能力相结合，不仅支持从各类文档中高效提取知识，还能让模型基于这些知识进行逻辑推理和任务执行。\n\n在大模型应用中，幻觉问题和知识滞后是常见痛点。RAGFlow 通过深度解析复杂文档结构（如表格、图表及混合排版），显著提升了信息检索的准确度，从而有效减少模型“胡编乱造”的现象，确保回答既有据可依又具备时效性。其内置的智能体机制更进一步，使系统不仅能回答问题，还能自主规划步骤解决复杂问题。\n\n这款工具特别适合开发者、企业技术团队以及 AI 研究人员使用。无论是希望快速搭建私有知识库问答系统，还是致力于探索大模型在垂直领域落地的创新者，都能从中受益。RAGFlow 提供了可视化的工作流编排界面和灵活的 API 接口，既降低了非算法背景用户的上手门槛，也满足了专业开发者对系统深度定制的需求。作为基于 Apache 2.0 协议开源的项目，它正成为连接通用大模型与行业专有知识之间的重要桥梁。",77062,"2026-04-04T04:44:48",[15,14,13,26,54],{"id":65,"github_repo":66,"name":67,"description_en":68,"description_zh":69,"ai_summary_zh":69,"readme_en":70,"readme_zh":71,"quickstart_zh":72,"use_case_zh":73,"hero_image_url":74,"owner_login":75,"owner_name":76,"owner_avatar_url":77,"owner_bio":78,"owner_company":79,"owner_location":79,"owner_email":79,"owner_twitter":79,"owner_website":79,"owner_url":80,"languages":81,"stars":86,"forks":87,"last_commit_at":88,"license":89,"difficulty_score":90,"env_os":91,"env_gpu":92,"env_ram":93,"env_deps":94,"category_tags":100,"github_topics":101,"view_count":23,"oss_zip_url":79,"oss_zip_packed_at":79,"status":16,"created_at":109,"updated_at":110,"faqs":111,"releases":142},3862,"rafaelpadilla\u002FObject-Detection-Metrics","Object-Detection-Metrics","Most popular metrics used to evaluate object detection algorithms.","Object-Detection-Metrics 是一个专为评估目标检测算法性能而设计的开源工具包，集成了学术界和各类竞赛中最主流的评价指标。在目标检测领域，不同研究和比赛往往采用各异的评估标准，且官方参考代码稀缺，导致研究人员在复现或对比模型时，常因自行实现指标的差异而产生偏差结果。Object-Detection-Metrics 正是为了解决这一痛点而生，它提供了一套灵活、统一且经过严格验证的计算函数，确保评估结果与官方实现完全一致，从而让不同数据集上的模型对比更加公平可信。\n\n该工具特别适合人工智能研究人员、算法工程师以及需要严谨评估模型表现的开发团队使用。其核心亮点在于极高的易用性与兼容性：用户无需将数据转换为复杂的 XML 或 JSON 格式，直接输入真实的边界框和检测到的边界框即可快速计算精度。此外，项目不仅涵盖了经典的精确率 - 召回率曲线等指标，还推出了支持 COCO 全套指标、具备图形化界面（UI）并能评估视频检测效果的全新版本，大大降低了技术门槛，帮助用户高效完成从实验验证到论文发表的全过程。","\n\n\u003Cp align=\"left\">\n    \u003Ca href=\"https:\u002F\u002Fzenodo.org\u002Fbadge\u002Flatestdoi\u002F134606465\">\n        \u003Cimg src=\"https:\u002F\u002Fzenodo.org\u002Fbadge\u002F134606465.svg\"\u002F>\u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fopensource.org\u002Flicenses\u002FMIT\" >\n        \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLicense-MIT-yellow.svg\" \u002F>\u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002FObject-Detection-Metrics\u002Fraw\u002Fmaster\u002Fpaper_survey_on_performance_metrics_for_object_detection_algorithms.pdf\">\n        \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fpaper-published-red\"\u002F>\u003C\u002Fa>\n\u003C\u002Fp>\n\n\n## Citation\n\nIf you use this code for your research, please consider citing:\n\n``` \n@Article{electronics10030279,\nAUTHOR = {Padilla, Rafael and Passos, Wesley L. and Dias, Thadeu L. B. and Netto, Sergio L. and da Silva, Eduardo A. B.},\nTITLE = {A Comparative Analysis of Object Detection Metrics with a Companion Open-Source Toolkit},\nJOURNAL = {Electronics},\nVOLUME = {10},\nYEAR = {2021},\nNUMBER = {3},\nARTICLE-NUMBER = {279},\nURL = {https:\u002F\u002Fwww.mdpi.com\u002F2079-9292\u002F10\u002F3\u002F279},\nISSN = {2079-9292},\nDOI = {10.3390\u002Felectronics10030279}\n}\n```\nDownload the paper [here](https:\u002F\u002Fwww.mdpi.com\u002F2079-9292\u002F10\u002F3\u002F279\u002Fpdf) or [here](https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002Freview_object_detection_metrics\u002Fblob\u002Fmain\u002Fpublished_paper.pdf).\n\n```\n@INPROCEEDINGS {padillaCITE2020,\n    author    = {R. {Padilla} and S. L. {Netto} and E. A. B. {da Silva}},\n    title     = {A Survey on Performance Metrics for Object-Detection Algorithms}, \n    booktitle = {2020 International Conference on Systems, Signals and Image Processing (IWSSIP)}, \n    year      = {2020},\n    pages     = {237-242},}\n```\nDownload the paper [here](https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002FObject-Detection-Metrics\u002Fraw\u002Fmaster\u002Fpaper_survey_on_performance_metrics_for_object_detection_algorithms.pdf)\n\n-----------------\n\nAttention! A new version of this tool is available [here](https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002Freview_object_detection_metrics)\n=======\n\nThe new version includes **all COCO metrics**, supports **other file formats**, provides a **User Interface (UI)** to guide the evaluation process, and presents the **STT-AP metric** to evaluate object detection in videos. \n\n-----------------\n\n# Metrics for object detection\n  \nThe motivation of this project is the lack of consensus used by different works and implementations concerning the **evaluation metrics of the object detection problem**. Although on-line competitions use their own metrics to evaluate the task of object detection, just some of them offer reference code snippets to calculate the accuracy of the detected objects.  \nResearchers who want to evaluate their work using different datasets than those offered by the competitions, need to implement their own version of the metrics. Sometimes a wrong or different implementation can create different and biased results. Ideally, in order to have trustworthy benchmarking among different approaches, it is necessary to have a flexible implementation that can be used by everyone regardless the dataset used.  \n\n**This project provides easy-to-use functions implementing the same metrics used by the the most popular competitions of object detection**. Our implementation does not require modifications of your detection model to complicated input formats, avoiding conversions to XML or JSON files. We simplified the input data (ground truth bounding boxes and detected bounding boxes) and gathered in a single project the main metrics used by the academia and challenges. Our implementation was carefully compared against the official implementations and our results are exactly the same.   \n\nIn the topics below you can find an overview of the most popular metrics used in different competitions and works, as well as samples showing how to use our code.\n\n## Table of contents\n\n- [Motivation](#metrics-for-object-detection)\n- [Different competitions, different metrics](#different-competitions-different-metrics)\n- [Important definitions](#important-definitions)\n- [Metrics](#metrics)\n  - [Precision x Recall curve](#precision-x-recall-curve)\n  - [Average Precision](#average-precision)\n    - [11-point interpolation](#11-point-interpolation)\n    - [Interpolating all  points](#interpolating-all-points)\n- [**How to use this project**](#how-to-use-this-project)\n- [References](#references)\n\n\u003Ca name=\"different-competitions-different-metrics\">\u003C\u002Fa> \n## Different competitions, different metrics \n\n* **[PASCAL VOC Challenge](http:\u002F\u002Fhost.robots.ox.ac.uk\u002Fpascal\u002FVOC\u002F)** offers a Matlab script in order to evaluate the quality of the detected objects. Participants of the competition can use the provided Matlab script to measure the accuracy of their detections before submitting their results. The official documentation explaining their criteria for object detection metrics can be accessed [here](http:\u002F\u002Fhost.robots.ox.ac.uk\u002Fpascal\u002FVOC\u002Fvoc2012\u002Fhtmldoc\u002Fdevkit_doc.html#SECTION00050000000000000000). The current metrics used by the current PASCAL VOC object detection challenge are the **Precision x Recall curve** and **Average Precision**.  \nThe PASCAL VOC Matlab evaluation code reads the ground truth bounding boxes from XML files, requiring changes in the code if you want to apply it to other datasets or to your specific cases. Even though projects such as [Faster-RCNN](https:\u002F\u002Fgithub.com\u002Frbgirshick\u002Fpy-faster-rcnn) implement PASCAL VOC evaluation metrics, it is also necessary to convert the detected bounding boxes into their specific format. [Tensorflow](https:\u002F\u002Fgithub.com\u002Ftensorflow\u002Fmodels\u002Fblob\u002Fmaster\u002Fresearch\u002Fobject_detection\u002Fg3doc\u002Fevaluation_protocols.md) framework also has their PASCAL VOC metrics implementation.  \n\n* **[COCO Detection Challenge](https:\u002F\u002Fcompetitions.codalab.org\u002Fcompetitions\u002F5181)** uses different metrics to evaluate the accuracy of object detection of different algorithms. [Here](http:\u002F\u002Fcocodataset.org\u002F#detection-eval) you can find a documentation explaining the 12 metrics used for characterizing the performance of an object detector on COCO. This competition offers Python and Matlab codes so users can verify their scores before submitting the results. It is also necessary to convert the results to a [format](http:\u002F\u002Fcocodataset.org\u002F#format-results) required by the competition.  \n\n* **[Google Open Images Dataset V4 Competition](https:\u002F\u002Fstorage.googleapis.com\u002Fopenimages\u002Fweb\u002Fchallenge.html)** also uses mean Average Precision (mAP) over the 500 classes to evaluate the object detection task. \n\n* **[ImageNet Object Localization Challenge](https:\u002F\u002Fwww.kaggle.com\u002Fc\u002Fimagenet-object-detection-challenge)** defines an error for each image considering the class and the overlapping region between ground truth and detected boxes. The total error is computed as the average of all min errors among all test dataset images. [Here](https:\u002F\u002Fwww.kaggle.com\u002Fc\u002Fimagenet-object-localization-challenge#evaluation) are more details about their evaluation method.  \n\n## Important definitions  \n\n### Intersection Over Union (IOU)\n\n\nIntersection Over Union (IOU) is a measure based on Jaccard Index that evaluates the overlap between two bounding boxes. It requires a ground truth bounding box ![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_e862be94b2d4.png) and a predicted bounding box ![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_0ee795a59a5a.png). By applying the IOU we can tell if a detection is valid (True Positive) or not (False Positive).  \n\nIOU is given by the overlapping area between the predicted bounding box and the ground truth bounding box divided by the area of union between them:  \n\n\u003Cp align=\"center\"> \n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_1efe91a42175.png\">\n\u003C\u002Fp>\n\n\u003C!---\n\\text{IOU}=\\frac{\\text{area}\\left(B_{p} \\cap B_{gt} \\right)}{\\text{area}\\left(B_{p} \\cup B_{gt} \\right)} \n--->\n\nThe image below illustrates the IOU between a ground truth bounding box (in green) and a detected bounding box (in red).\n\n\u003C!--- IOU --->\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_7d876231fbb3.png\" align=\"center\"\u002F>\u003C\u002Fp>\n\n### True Positive, False Positive, False Negative and True Negative  \n\nSome basic concepts used by the metrics:  \n\n* **True Positive (TP)**: A correct detection. Detection with IOU ≥ _threshold_  \n* **False Positive (FP)**: A wrong detection. Detection with IOU \u003C _threshold_  \n* **False Negative (FN)**: A ground truth not detected  \n* **True Negative (TN)**: Does not apply. It would represent a corrected misdetection. In the object detection task there are many possible bounding boxes that should not be detected within an image. Thus, TN would be all possible bounding boxes that were corrrectly not detected (so many possible boxes within an image). That's why it is not used by the metrics.\n\n_threshold_: depending on the metric, it is usually set to 50%, 75% or 95%.\n\n### Precision\n\nPrecision is the ability of a model to identify **only** the relevant objects. It is the percentage of correct positive predictions and is given by:\n\n\u003Cp align=\"center\"> \n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_88d4d72adaa7.png\">\n\u003C\u002Fp>\n\n\u003C!---\n\\text{Precision} = \\frac{\\text{TP}}{\\text{TP}+\\text{FP}}=\\frac{\\text{TP}}{\\text{all detections}}\n--->\n\n### Recall \n\nRecall is the ability of a model to find all the relevant cases (all ground truth bounding boxes). It is the percentage of true positive detected among all relevant ground truths and is given by:\n\n\u003Cp align=\"center\"> \n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_2a3790bb5af1.png\">\n\u003C\u002Fp>\n\u003C!--- \n\\text{Recall} = \\frac{\\text{TP}}{\\text{TP}+\\text{FN}}=\\frac{\\text{TP}}{\\text{all ground truths}}\n--->\n\n## Metrics\n\nIn the topics below there are some comments on the most popular metrics used for object detection.\n\n### Precision x Recall curve\n\nThe Precision x Recall curve is a good way to evaluate the performance of an object detector as the confidence is changed by plotting a curve for each object class. An object detector of a particular class is considered good if its precision stays high as recall increases, which means that if you vary the confidence threshold, the precision and recall will still be high. Another way to identify a good object detector is to look for a detector that can identify only relevant objects (0 False Positives = high precision), finding all ground truth objects (0 False Negatives = high recall).  \n\nA poor object detector needs to increase the number of detected objects (increasing False Positives = lower precision) in order to retrieve all ground truth objects (high recall). That's why the Precision x Recall curve usually starts with high precision values, decreasing as recall increases. You can see an example of the Prevision x Recall curve in the next topic (Average Precision). This kind of curve is used by the PASCAL VOC 2012 challenge and is available in our implementation.  \n\n### Average Precision\n\nAnother way to compare the performance of object detectors is to calculate the area under the curve (AUC) of the Precision x Recall curve. As AP curves are often zigzag curves going up and down, comparing different curves (different detectors) in the same plot usually is not an easy task - because the curves tend to cross each other much frequently. That's why Average Precision (AP), a numerical metric, can also help us compare different detectors. In practice AP is the precision averaged across all recall values between 0 and 1.  \n\nFrom 2010 on, the method of computing AP by the PASCAL VOC challenge has changed. Currently, **the interpolation performed by PASCAL VOC challenge uses all data points, rather than interpolating only 11 equally spaced points as stated in their [paper](http:\u002F\u002Fciteseerx.ist.psu.edu\u002Fviewdoc\u002Fdownload?doi=10.1.1.157.5766&rep=rep1&type=pdf)**. As we want to reproduce their default implementation, our default code (as seen further) follows their most recent application (interpolating all data points). However, we also offer the 11-point interpolation approach. \n\n#### 11-point interpolation\n\nThe 11-point interpolation tries to summarize the shape of the Precision x Recall curve by averaging the precision at a set of eleven equally spaced recall levels [0, 0.1, 0.2, ... , 1]:\n\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_dd0a7f53aef1.png\">\n\u003C\u002Fp>\n\u003C!---\n\\text{AP}=\\frac{1}{11} \\sum_{r\\in \\left \\{ 0, 0.1, ...,1 \\right \\}}\\rho_{\\text{interp}\\left ( r \\right )}\n--->\n\nwith\n\n\u003Cp align=\"center\"> \n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_ef827dfa2d33.png\">\n\u003C\u002Fp>\n\u003C!--- \n\\rho_{\\text{interp}} = \\max_{\\tilde{r}:\\tilde{r} \\geq r} \\rho\\left ( \\tilde{r} \\right )\n--->\n\nwhere ![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_a548c25dd22e.png) is the measured precision at recall ![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_6ae649c95987.png).\n\nInstead of using the precision observed at each point, the AP is obtained by interpolating the precision only at the 11 levels ![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_ffbbc5de35b8.png) taking the **maximum precision whose recall value is greater than ![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_ffbbc5de35b8.png)**.\n\n#### Interpolating all points\n\nInstead of interpolating only in the 11 equally spaced points, you could interpolate through all points \u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_697ffc5d5395.png\"> in such way that:\n\n\u003Cp align=\"center\"> \n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_b0dc40c5869a.png\">\n\u003C\u002Fp>\n\u003C!---\n\\sum_{n=0} \\left ( r_{n+1} - r_{n} \\right ) \\rho_{\\text{interp}}\\left ( r_{n+1} \\right )\n--->\n \nwith\n\n\u003Cp align=\"center\"> \n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_513dfbe61141.png\">\n\u003C\u002Fp>\n\n\u003C!---\n\\rho_{\\text{interp}}\\left ( r_{n+1} \\right ) = \\max_{\\tilde{r}:\\tilde{r} \\ge r_{n+1}} \\rho \\left ( \\tilde{r} \\right )\n--->\n\n\nwhere ![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_a548c25dd22e.png) is the measured precision at recall ![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_6ae649c95987.png).\n\nIn this case, instead of using the precision observed at only few points, the AP is now obtained by interpolating the precision at **each level**, ![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_ffbbc5de35b8.png) taking the **maximum precision whose recall value is greater or equal than ![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_ffbbc5de35b8.png+1)**. This way we calculate the estimated area under the curve.\n\nTo make things more clear, we provided an example comparing both interpolations.\n\n\n#### An ilustrated example \n\nAn example helps us understand better the concept of the interpolated average precision. Consider the detections below:\n  \n\u003C!--- Image samples 1 --->\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_793336302ec8.png\" align=\"center\"\u002F>\u003C\u002Fp>\n  \nThere are 7 images with 15 ground truth objects represented by the green bounding boxes and 24 detected objects represented by the red bounding boxes. Each detected object has a confidence level and is identified by a letter (A,B,...,Y).  \n\nThe following table shows the bounding boxes with their corresponding confidences. The last column identifies the detections as TP or FP. In this example a TP is considered if IOU ![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_7ccf27b22b6c.png) 30%, otherwise it is a FP. By looking at the images above we can roughly tell if the detections are TP or FP.\n\n\u003C!--- Table 1 --->\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_9b27a9cf247f.png\" align=\"center\"\u002F>\u003C\u002Fp>\n\n\u003C!---\n| Images | Detections | Confidences | TP or FP |\n|:------:|:----------:|:-----------:|:--------:|\n| Image 1 | A | 88% | FP |\n| Image 1 | B | 70% | TP |\n| Image 1 |\tC\t| 80% | FP |\n| Image 2 |\tD\t| 71% | FP |\n| Image 2 |\tE\t| 54% | TP |\n| Image 2 |\tF\t| 74% | FP |\n| Image 3 |\tG\t| 18% | TP |\n| Image 3 |\tH\t| 67% | FP |\n| Image 3 |\tI\t| 38% | FP |\n| Image 3 |\tJ\t| 91% | TP |\n| Image 3 |\tK\t| 44% | FP |\n| Image 4 |\tL\t| 35% | FP |\n| Image 4 |\tM\t| 78% | FP |\n| Image 4 |\tN\t| 45% | FP |\n| Image 4 |\tO\t| 14% | FP |\n| Image 5 |\tP\t| 62% | TP |\n| Image 5 |\tQ\t| 44% | FP |\n| Image 5 |\tR\t| 95% | TP |\n| Image 5 |\tS\t| 23% | FP |\n| Image 6 |\tT\t| 45% | FP |\n| Image 6 |\tU\t| 84% | FP |\n| Image 6 |\tV\t| 43% | FP |\n| Image 7 |\tX\t| 48% | TP |\n| Image 7 |\tY\t| 95% | FP |\n--->\n\nIn some images there are more than one detection overlapping a ground truth (Images 2, 3, 4, 5, 6 and 7). For those cases, the predicted box with the highest IOU is considered TP (e.g. in image 1 \"E\" is TP while \"D\" is FP because IOU between E and the groundtruth is greater than the IOU between D and the groundtruth). This rule is applied by the PASCAL VOC 2012 metric: \"e.g. 5 detections (TP) of a single object is counted as 1 correct detection and 4 false detections”.\n\nThe Precision x Recall curve is plotted by calculating the precision and recall values of the accumulated TP or FP detections.  For this, first we need to order the detections by their confidences, then we calculate the precision and recall for each accumulated detection as shown in the table below (Note that for recall computation, the denominator term (\"Acc TP + Acc FN\" or \"All ground truths\") is constant at 15 since GT boxes are constant irrespective of detections).: \n\n\u003C!--- Table 2 --->\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_c2173c9b088c.png\" align=\"center\"\u002F>\u003C\u002Fp>\n\n\u003C!---\n| Images | Detections | Confidences |  TP | FP | Acc TP | Acc FP | Precision | Recall |\n|:------:|:----------:|:-----------:|:---:|:--:|:------:|:------:|:---------:|:------:|\n| Image 5 |\tR\t| 95% | 1 | 0 | 1 | 0 | 1       | 0.0666 |\n| Image 7 |\tY\t| 95% | 0 | 1 | 1 | 1 | 0.5     | 0.6666 |\n| Image 3 |\tJ\t| 91% | 1 | 0 | 2 | 1 | 0.6666  | 0.1333 |\n| Image 1 | A | 88% | 0 | 1 | 2 | 2 | 0.5     | 0.1333 |\n| Image 6 |\tU\t| 84% | 0 | 1 | 2 | 3 | 0.4     | 0.1333 |\n| Image 1 |\tC\t| 80% | 0 | 1 | 2 | 4 | 0.3333  | 0.1333 |\n| Image 4 |\tM\t| 78% | 0 | 1 | 2 | 5 | 0.2857  | 0.1333 |\n| Image 2 |\tF\t| 74% | 0 | 1 | 2 | 6 | 0.25    | 0.1333 |\n| Image 2 |\tD\t| 71% | 0 | 1 | 2 | 7 | 0.2222  | 0.1333 |\n| Image 1 | B | 70% | 1 | 0 | 3 | 7 | 0.3     | 0.2    |\n| Image 3 |\tH\t| 67% | 0 | 1 | 3 | 8 | 0.2727  | 0.2    |\n| Image 5 |\tP\t| 62% | 1 | 0 | 4 | 8 | 0.3333  | 0.2666 |\n| Image 2 |\tE\t| 54% | 1 | 0 | 5 | 8 | 0.3846  | 0.3333 |\n| Image 7 |\tX\t| 48% | 1 | 0 | 6 | 8 | 0.4285  | 0.4    |\n| Image 4 |\tN\t| 45% | 0 | 1 | 6 | 9 | 0.7     | 0.4    |\n| Image 6 |\tT\t| 45% | 0 | 1 | 6 | 10 | 0.375  | 0.4    |\n| Image 3 |\tK\t| 44% | 0 | 1 | 6 | 11 | 0.3529 | 0.4    |\n| Image 5 |\tQ\t| 44% | 0 | 1 | 6 | 12 | 0.3333 | 0.4    |\n| Image 6 |\tV\t| 43% | 0 | 1 | 6 | 13 | 0.3157 | 0.4    |\n| Image 3 |\tI\t| 38% | 0 | 1 | 6 | 14 | 0.3    | 0.4    |\n| Image 4 |\tL\t| 35% | 0 | 1 | 6 | 15 | 0.2857 | 0.4    |\n| Image 5 |\tS\t| 23% | 0 | 1 | 6 | 16 | 0.2727 | 0.4    |\n| Image 3 |\tG\t| 18% | 1 | 0 | 7 | 16 | 0.3043 | 0.4666 |\n| Image 4 |\tO\t| 14% | 0 | 1 | 7 | 17 | 0.2916 | 0.4666 |\n--->\n \n Example computation for the 2nd row (Image 7):  Precision = TP\u002F(TP+FP) = 1\u002F2 = 0.5 and Recall = TP\u002F(TP+FN) = 1\u002F15 = 0.066\n \n Plotting the precision and recall values we have the following *Precision x Recall curve*:\n \n \u003C!--- Precision x Recall graph --->\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_fd0bd7bcfefd.png\" align=\"center\"\u002F>\n\u003C\u002Fp>\n \nAs mentioned before, there are two different ways to measure the interpolted average precision: **11-point interpolation** and **interpolating all points**. Below we make a comparisson between them:\n\n#### Calculating the 11-point interpolation\n\nThe idea of the 11-point interpolated average precision is to average the precisions at a set of 11 recall levels (0,0.1,...,1). The interpolated precision values are obtained by taking the maximum precision whose recall value is greater than its current recall value as follows: \n\n\u003C!--- interpolated precision curve --->\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_16275079aee8.png\" align=\"center\"\u002F>\n\u003C\u002Fp>\n\nBy applying the 11-point interpolation, we have:  \n\n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_aa250fbfeb36.png)  \n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_98bbd46df03f.png)  \n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_d15076925480.png)\n\n\n#### Calculating the interpolation performed in all points\n\nBy interpolating all points, the Average Precision (AP) can be interpreted as an approximated AUC of the Precision x Recall curve. The intention is to reduce the impact of the wiggles in the curve. By applying the equations presented before, we can obtain the areas as it will be demostrated here. We could also visually have the interpolated precision points by looking at the recalls starting from the highest (0.4666) to 0 (looking at the plot from right to left) and, as we decrease the recall, we collect the precision values that are the highest as shown in the image below:\n    \n\u003C!--- interpolated precision AUC --->\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_84aabdab84c9.png\" align=\"center\"\u002F>\n\u003C\u002Fp>\n  \nLooking at the plot above, we can divide the AUC into 4 areas (A1, A2, A3 and A4):\n  \n\u003C!--- interpolated precision AUC --->\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_b8d4689ea3a2.png\" align=\"center\"\u002F>\n\u003C\u002Fp>\n\nCalculating the total area, we have the AP:  \n\n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_a177615e7c31.png)  \n  \n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_943f86405365.png)  \n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_4fa717252508.png)  \n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_3b44c843dd0d.png)  \n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_4626426bf0b4.png)  \n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_f8790b340b4e.png)  \n   \n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_b3dce126cb74.png)  \n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_93a881607959.png)  \n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_149aebbe95e2.png)  \n\nThe results between the two different interpolation methods are a little different: 24.56% and 26.84% by the every point interpolation and the 11-point interpolation respectively.  \n\nOur default implementation is the same as VOC PASCAL: every point interpolation. If you want to use the 11-point interpolation, change the functions that use the argument ```method=MethodAveragePrecision.EveryPointInterpolation``` to ```method=MethodAveragePrecision.ElevenPointInterpolation```.   \n\nIf you want to reproduce these results, see the **[Sample 2](https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002FObject-Detection-Metrics\u002Ftree\u002Fmaster\u002Fsamples\u002Fsample_2\u002F)**.\n\u003C!--In order to evaluate your detections, you just need a simple list of `Detection` objects. A `Detection` object is a very simple class containing the class id, class probability and bounding boxes coordinates of the detected objects. This same structure is used for the groundtruth detections.-->\n\n## How to use this project\n\nThis project was created to evaluate your detections in a very easy way. If you want to evaluate your algorithm with the most used object detection metrics, you are in the right place.  \n\n[Sample_1](https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002FObject-Detection-Metrics\u002Ftree\u002Fmaster\u002Fsamples\u002Fsample_1) and [sample_2](https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002FObject-Detection-Metrics\u002Ftree\u002Fmaster\u002Fsamples\u002Fsample_2) are practical examples demonstrating how to access directly the core functions of this project, providing more flexibility on the usage of the metrics. But if you don't want to spend your time understanding our code, see the instructions below to easily evaluate your detections:  \n\nFollow the steps below to start evaluating your detections:\n\n1. [Create the ground truth files](#create-the-ground-truth-files)\n2. [Create your detection files](#create-your-detection-files)\n3. For **Pascal VOC metrics**, run the command: `python pascalvoc.py`  \n   If you want to reproduce the example above, run the command: `python pascalvoc.py -t 0.3`\n4. (Optional) [You can use arguments to control the IOU threshold, bounding boxes format, etc.](#optional-arguments)\n\n### Create the ground truth files\n\n- Create a separate ground truth text file for each image in the folder **groundtruths\u002F**.\n- In these files each line should be in the format: `\u003Cclass_name> \u003Cleft> \u003Ctop> \u003Cright> \u003Cbottom>`.    \n- E.g. The ground truth bounding boxes of the image \"2008_000034.jpg\" are represented in the file \"2008_000034.txt\":\n  ```\n  bottle 6 234 45 362\n  person 1 156 103 336\n  person 36 111 198 416\n  person 91 42 338 500\n  ```\n    \nIf you prefer, you can also have your bounding boxes in the format: `\u003Cclass_name> \u003Cleft> \u003Ctop> \u003Cwidth> \u003Cheight>` (see here [**\\***](#asterisk) how to use it). In this case, your \"2008_000034.txt\" would be represented as:\n  ```\n  bottle 6 234 39 128\n  person 1 156 102 180\n  person 36 111 162 305\n  person 91 42 247 458\n  ```\n\n### Create your detection files\n\n- Create a separate detection text file for each image in the folder **detections\u002F**.\n- The names of the detection files must match their correspond ground truth (e.g. \"detections\u002F2008_000182.txt\" represents the detections of the ground truth: \"groundtruths\u002F2008_000182.txt\").\n- In these files each line should be in the following format: `\u003Cclass_name> \u003Cconfidence> \u003Cleft> \u003Ctop> \u003Cright> \u003Cbottom>` (see here [**\\***](#asterisk) how to use it).\n- E.g. \"2008_000034.txt\":\n    ```\n    bottle 0.14981 80 1 295 500  \n    bus 0.12601 36 13 404 316  \n    horse 0.12526 430 117 500 307  \n    pottedplant 0.14585 212 78 292 118  \n    tvmonitor 0.070565 388 89 500 196  \n    ```\n\nAlso if you prefer, you could have your bounding boxes in the format: `\u003Cclass_name> \u003Cconfidence> \u003Cleft> \u003Ctop> \u003Cwidth> \u003Cheight>`.\n\n### Optional arguments\n\nOptional arguments:\n\n| Argument                          | Description | Example | Default |\n|:-------------:|:-----------:|:-----------:|:-----------:|\n| `-h`,\u003Cbr>`--help ` |\tshow help message | `python pascalvoc.py -h` | |  \n|  `-v`,\u003Cbr>`--version` | check version | `python pascalvoc.py -v` | |  \n| `-gt`,\u003Cbr>`--gtfolder` | folder that contains the ground truth bounding boxes files | `python pascalvoc.py -gt \u002Fhome\u002Fwhatever\u002Fmy_groundtruths\u002F` | `\u002FObject-Detection-Metrics\u002Fgroundtruths`|  \n| `-det`,\u003Cbr>`--detfolder` | folder that contains your detected bounding boxes files | `python pascalvoc.py -det \u002Fhome\u002Fwhatever\u002Fmy_detections\u002F` | `\u002FObject-Detection-Metrics\u002Fdetections\u002F`|  \n| `-t`,\u003Cbr>`--threshold` | IOU thershold that tells if a detection is TP or FP | `python pascalvoc.py -t 0.75` | `0.50` |  \n| `-gtformat` | format of the coordinates of the ground truth bounding boxes [**\\***](#asterisk) | `python pascalvoc.py -gtformat xyrb` | `xywh` |\n| `-detformat` | format of the coordinates of the detected bounding boxes [**\\***](#asterisk) | `python pascalvoc.py -detformat xyrb` | `xywh` | |  \n| `-gtcoords` | reference of the ground truth bounding bounding box coordinates.\u003Cbr>If the annotated coordinates are relative to the image size (as used in YOLO), set it to `rel`.\u003Cbr>If the coordinates are absolute values, not depending to the image size, set it to `abs` |  `python pascalvoc.py -gtcoords rel` | `abs` |  \n| `-detcoords` | reference of the detected bounding bounding box coordinates.\u003Cbr>If the coordinates are relative to the image size (as used in YOLO), set it to `rel`.\u003Cbr>If the coordinates are absolute values, not depending to the image size, set it to `abs` | `python pascalvoc.py -detcoords rel` | `abs` |  \n| `-imgsize ` | image size in the format `width,height` \u003Cint,int>.\u003Cbr>Required if `-gtcoords` or `-detcoords` is set to `rel` | `python pascalvoc.py -imgsize 600,400` |  \n| `-sp`,\u003Cbr>`--savepath` | folder where the plots are saved | `python pascalvoc.py -sp \u002Fhome\u002Fwhatever\u002Fmy_results\u002F` | `Object-Detection-Metrics\u002Fresults\u002F` |  \n| `-np`,\u003Cbr>`--noplot` | if present no plot is shown during execution | `python pascalvoc.py -np` | not presented.\u003Cbr>Therefore, plots are shown |  \n\n\u003Ca name=\"asterisk\"> \u003C\u002Fa>\n(**\\***) set `-gtformat xywh` and\u002For `-detformat xywh` if format is `\u003Cleft> \u003Ctop> \u003Cwidth> \u003Cheight>`. Set to `-gtformat xyrb` and\u002For `-detformat xyrb`  if format is `\u003Cleft> \u003Ctop> \u003Cright> \u003Cbottom>`.\n  \n## References\n\n* The Relationship Between Precision-Recall and ROC Curves (Jesse Davis and Mark Goadrich)\nDepartment of Computer Sciences and Department of Biostatistics and Medical Informatics, University of\nWisconsin  \nhttp:\u002F\u002Fpages.cs.wisc.edu\u002F~jdavis\u002Fdavisgoadrichcamera2.pdf\n\n* The PASCAL Visual Object Classes (VOC) Challenge  \nhttp:\u002F\u002Fciteseerx.ist.psu.edu\u002Fviewdoc\u002Fdownload?doi=10.1.1.157.5766&rep=rep1&type=pdf\n\n* Evaluation of ranked retrieval results (Salton and Mcgill 1986)  \nhttps:\u002F\u002Fwww.amazon.com\u002FIntroduction-Information-Retrieval-COMPUTER-SCIENCE\u002Fdp\u002F0070544840  \nhttps:\u002F\u002Fnlp.stanford.edu\u002FIR-book\u002Fhtml\u002Fhtmledition\u002Fevaluation-of-ranked-retrieval-results-1.html\n","\u003Cp align=\"left\">\n    \u003Ca href=\"https:\u002F\u002Fzenodo.org\u002Fbadge\u002Flatestdoi\u002F134606465\">\n        \u003Cimg src=\"https:\u002F\u002Fzenodo.org\u002Fbadge\u002F134606465.svg\"\u002F>\u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fopensource.org\u002Flicenses\u002FMIT\" >\n        \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLicense-MIT-yellow.svg\" \u002F>\u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002FObject-Detection-Metrics\u002Fraw\u002Fmaster\u002Fpaper_survey_on_performance_metrics_for_object_detection_algorithms.pdf\">\n        \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fpaper-published-red\"\u002F>\u003C\u002Fa>\n\u003C\u002Fp>\n\n\n## 引用\n\n如果您在研究中使用此代码，请考虑引用以下内容：\n\n``` \n@Article{electronics10030279,\nAUTHOR = {Padilla, Rafael and Passos, Wesley L. and Dias, Thadeu L. B. and Netto, Sergio L. and da Silva, Eduardo A. B.},\nTITLE = {一种基于开源工具包的对象检测指标比较分析},\nJOURNAL = {Electronics},\nVOLUME = {10},\nYEAR = {2021},\nNUMBER = {3},\nARTICLE-NUMBER = {279},\nURL = {https:\u002F\u002Fwww.mdpi.com\u002F2079-9292\u002F10\u002F3\u002F279},\nISSN = {2079-9292},\nDOI = {10.3390\u002Felectronics10030279}\n}\n```\n您可以通过[此处](https:\u002F\u002Fwww.mdpi.com\u002F2079-9292\u002F10\u002F3\u002F279\u002Fpdf)或[此处](https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002Freview_object_detection_metrics\u002Fblob\u002Fmain\u002Fpublished_paper.pdf)下载该论文。\n\n```\n@INPROCEEDINGS {padillaCITE2020,\n    author    = {R. {Padilla} and S. L. {Netto} and E. A. B. {da Silva}},\n    title     = {目标检测算法性能指标综述}, \n    booktitle = {2020年国际系统、信号与图像处理会议（IWSSIP）}, \n    year      = {2020},\n    pages     = {237-242},}\n```\n您可以通过[此处](https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002FObject-Detection-Metrics\u002Fraw\u002Fmaster\u002Fpaper_survey_on_performance_metrics_for_object_detection_algorithms.pdf)下载该论文。\n\n-----------------\n\n注意！本工具的新版本已在[这里](https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002Freview_object_detection_metrics)发布。\n=======\n\n新版本包含**所有COCO指标**，支持**其他文件格式**，提供一个**用户界面（UI）**来指导评估过程，并引入了用于评估视频中目标检测的**STT-AP指标**。 \n\n-----------------\n\n# 目标检测指标\n  \n本项目的出发点在于，不同研究和实现之间对于**目标检测问题的评价指标**缺乏共识。尽管在线竞赛会采用各自的指标来评估目标检测任务，但只有少数竞赛提供了用于计算检测精度的参考代码片段。  \n希望使用不同于竞赛所提供数据集的研究人员，往往需要自行实现相应的指标计算方法。然而，错误或不一致的实现方式可能会导致结果偏差。理想情况下，为了确保不同方法之间的可比性，有必要开发一种灵活且通用的实现方案，以便无论使用何种数据集都能进行准确的基准测试。  \n\n**本项目提供易于使用的函数，实现了当前最流行的目标检测竞赛所采用的相同指标**。我们的实现无需对您的检测模型进行复杂输入格式的修改，也避免了将数据转换为XML或JSON文件的麻烦。我们简化了输入数据（真实框和预测框），并将学术界及各类挑战赛中常用的主流指标整合到一个统一的项目中。经过与官方实现的仔细对比，我们的结果完全一致。   \n\n在下文中，您将找到关于不同竞赛和研究中常用指标的概述，以及如何使用我们代码的示例。\n\n## 目录\n\n- [项目背景](#metrics-for-object-detection)\n- [不同竞赛，不同指标](#different-competitions-different-metrics)\n- [重要定义](#important-definitions)\n- [指标](#metrics)\n  - [精确率-召回率曲线](#precision-x-recall-curve)\n  - [平均精度](#average-precision)\n    - [11点插值法](#11-point-interpolation)\n    - [全点插值法](#interpolating-all-points)\n- [**如何使用本项目**](#how-to-use-this-project)\n- [参考文献](#references)\n\n\u003Ca name=\"different-competitions-different-metrics\">\u003C\u002Fa> \n## 不同竞赛，不同指标 \n\n* **[PASCAL VOC挑战赛](http:\u002F\u002Fhost.robots.ox.ac.uk\u002Fpascal\u002FVOC\u002F)** 提供了一个Matlab脚本，用于评估检测结果的质量。参赛者可以在提交结果之前，利用该脚本测量其检测精度。有关其目标检测指标标准的官方文档可在[此处](http:\u002F\u002Fhost.robots.ox.ac.uk\u002Fpascal\u002FVOC\u002Fvoc2012\u002Fhtmldoc\u002Fdevkit_doc.html#SECTION00050000000000000000)查阅。目前PASCAL VOC目标检测挑战赛所采用的指标包括**精确率-召回率曲线**和**平均精度**。  \nPASCAL VOC的Matlab评估代码从XML文件中读取真实框，若要将其应用于其他数据集或特定场景，则需对代码进行相应修改。尽管像[Faster-RCNN](https:\u002F\u002Fgithub.com\u002Frbgirshick\u002Fpy-faster-rcnn)这样的项目已经实现了PASCAL VOC的评估指标，但仍需将检测到的边界框转换为特定格式。[TensorFlow](https:\u002F\u002Fgithub.com\u002Ftensorflow\u002Fmodels\u002Fblob\u002Fmaster\u002Fresearch\u002Fobject_detection\u002Fg3doc\u002Fevaluation_protocols.md)框架也提供了PASCAL VOC指标的实现。  \n\n* **[COCO目标检测挑战赛](https:\u002F\u002Fcompetitions.codalab.org\u002Fcompetitions\u002F5181)** 使用不同的指标来评估各种算法的目标检测精度。[在这里](http:\u002F\u002Fcocodataset.org\u002F#detection-eval)可以找到一份文档，详细介绍了用于表征COCO数据集上目标检测器性能的12种指标。该竞赛提供了Python和Matlab代码，方便用户在提交结果前验证自己的分数。不过，仍需将结果转换为竞赛要求的[特定格式](http:\u002F\u002Fcocodataset.org\u002F#format-results)。  \n\n* **[Google Open Images Dataset V4竞赛](https:\u002F\u002Fstorage.googleapis.com\u002Fopenimages\u002Fweb\u002Fchallenge.html)** 同样使用500个类别的平均精度（mAP）来评估目标检测任务。 \n\n* **[ImageNet物体定位挑战赛](https:\u002F\u002Fwww.kaggle.com\u002Fc\u002Fimagenet-object-detection-challenge)** 根据每个图像的真实框与预测框之间的类别匹配情况及重叠区域大小来计算误差。最终的总误差是所有测试图像中最小误差的平均值。[此处](https:\u002F\u002Fwww.kaggle.com\u002Fc\u002Fimagenet-object-localization-challenge#evaluation)提供了更多关于其评估方法的详细信息。  \n\n## 重要定义\n\n### 交并比（IOU）\n\n\n交并比（IOU）是一种基于Jaccard指数的度量方法，用于评估两个边界框之间的重叠程度。它需要一个真实边界框 ![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_e862be94b2d4.png) 和一个预测边界框 ![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_0ee795a59a5a.png)。通过计算IOU，我们可以判断检测结果是否有效（真正例）或无效（假正例）。  \n\nIOU的计算公式是：预测边界框与真实边界框的交集面积除以它们的并集面积：  \n\n\u003Cp align=\"center\"> \n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_1efe91a42175.png\">\n\u003C\u002Fp>\n\n\u003C!---\n\\text{IOU}=\\frac{\\text{area}\\left(B_{p} \\cap B_{gt} \\right)}{\\text{area}\\left(B_{p} \\cup B_{gt} \\right)} \n--->\n\n下图展示了真实边界框（绿色）与检测到的边界框（红色）之间的IOU。\n\n\u003C!--- IOU --->\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_7d876231fbb3.png\" align=\"center\"\u002F>\u003C\u002Fp>\n\n### 真正例、假正例、假负例和真负例  \n\n指标中使用的一些基本概念：  \n\n* **真正例（TP）**：正确的检测。IOU ≥ _阈值_ 的检测  \n* **假正例（FP）**：错误的检测。IOU \u003C _阈值_ 的检测  \n* **假负例（FN）**：未被检测到的真实目标  \n* **真负例（TN）**：不适用。它表示对误检的纠正。在目标检测任务中，图像中有很多不应被检测到的候选边界框。因此，TN将代表所有正确未被检测到的候选边界框（即图像中的大量可能边界框）。这就是为什么该指标不使用真负例的原因。\n\n_阈值_：根据不同的指标，通常设置为50%、75%或95%。\n\n### 精确率\n\n精确率是指模型仅识别相关目标的能力。它是正确正例预测所占的百分比，计算公式如下：\n\n\u003Cp align=\"center\"> \n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_88d4d72adaa7.png\">\n\u003C\u002Fp>\n\n\u003C!---\n\\text{Precision} = \\frac{\\text{TP}}{\\text{TP}+\\text{FP}}=\\frac{\\text{TP}}{\\text{all detections}}\n--->\n\n### 召回率\n\n召回率是指模型找到所有相关样本（所有真实边界框）的能力。它是真正例占所有相关真实目标的百分比，计算公式如下：\n\n\u003Cp align=\"center\"> \n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_2a3790bb5af1.png\">\n\u003C\u002Fp>\n\u003C!--- \n\\text{Recall} = \\frac{\\text{TP}}{\\text{TP}+\\text{FN}}=\\frac{\\text{TP}}{\\text{all ground truths}}\n--->\n\n## 指标\n\n在下面的主题中，我们将介绍一些用于目标检测的最常用指标。\n\n### 精确率-召回率曲线\n\n精确率-召回率曲线是一种很好的方式，可以通过绘制每个类别对象的曲线来评估目标检测器在不同置信度下的性能。如果随着召回率的增加，精确率仍然保持较高，则认为该类别的目标检测器表现良好。这意味着当调整置信度阈值时，精确率和召回率仍能维持在较高水平。另一种判断优秀目标检测器的方法是寻找能够仅识别相关目标（假正例为0，即高精确率），同时又能找到所有真实目标（假负例为0，即高召回率）的检测器。  \n\n而性能较差的目标检测器则需要通过增加检测到的对象数量（从而导致假正例增多，精确率降低）来获取所有真实目标（高召回率）。因此，精确率-召回率曲线通常从较高的精确率开始，随着召回率的增加而逐渐下降。您可以在下一节（平均精度）中看到精确率-召回率曲线的示例。这种类型的曲线被PASCAL VOC 2012挑战赛所采用，并且在我们的实现中也有提供。\n\n### 平均精度\n\n比较目标检测器性能的另一种方法是计算精确率-召回率曲线下的面积（AUC）。由于AP曲线通常是上下波动的锯齿状曲线，在同一张图上比较不同曲线（即不同检测器）往往并不容易——因为这些曲线经常会相互交叉。因此，平均精度（AP）这一数值型指标也可以帮助我们比较不同的检测器。实际上，AP是在召回率从0到1的所有取值范围内对精确率求平均得到的值。  \n\n自2010年起，PASCAL VOC挑战赛计算AP的方法发生了变化。目前，**PASCAL VOC挑战赛采用的插值方法会使用所有数据点，而不是像其[论文](http:\u002F\u002Fciteseerx.ist.psu.edu\u002Fviewdoc\u002Fdownload?doi=10.1.1.157.5766&rep=rep1&type=pdf)中所述那样仅插值11个等间距的点**。为了重现他们的默认实现，我们的默认代码（见后文）遵循他们最新的应用方式（插值所有数据点）。不过，我们也提供了11点插值的方法。\n\n#### 11点插值\n\n11点插值试图通过在一组11个等间距的召回率水平 [0, 0.1, 0.2, ... , 1] 上对精确率求平均来概括精确率-召回率曲线的形状：\n\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_dd0a7f53aef1.png\">\n\u003C\u002Fp>\n\u003C!---\n\\text{AP}=\\frac{1}{11} \\sum_{r\\in \\left \\{ 0, 0.1, ...,1 \\right \\}}\\rho_{\\text{interp}\\left ( r \\right )}\n--->\n\n其中\n\n\u003Cp align=\"center\"> \n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_ef827dfa2d33.png\">\n\u003C\u002Fp>\n\u003C!--- \n\\rho_{\\text{interp}} = \\max_{\\tilde{r}:\\tilde{r} \\geq r} \\rho\\left ( \\tilde{r} \\right )\n--->\n\n这里 ![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_a548c25dd22e.png) 是在召回率 ![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_6ae649c95987.png) 处测得的精确率。\n\n与直接使用每个点的精确率不同，AP是通过对11个召回率 ![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_ffbbc5de35b8.png) 进行插值来获得的，取的是**召回值大于 ![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_ffbbc5de35b8.png) 的最大精确率**。\n\n#### 插值所有点\n\n与其仅在11个等间距的点上进行插值，你也可以通过所有点\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_697ffc5d5395.png\">来进行插值，使得：\n\n\u003Cp align=\"center\"> \n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_b0dc40c5869a.png\">\n\u003C\u002Fp>\n\u003C!---\n\\sum_{n=0} \\left ( r_{n+1} - r_{n} \\right ) \\rho_{\\text{interp}}\\left ( r_{n+1} \\right )\n--->\n \n其中\n\n\u003Cp align=\"center\"> \n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_513dfbe61141.png\">\n\u003C\u002Fp>\n\n\u003C!---\n\\rho_{\\text{interp}}\\left ( r_{n+1} \\right ) = \\max_{\\tilde{r}:\\tilde{r} \\ge r_{n+1}} \\rho \\left ( \\tilde{r} \\right )\n--->\n\n\n这里![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_a548c25dd22e.png)是在召回率![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_6ae649c95987.png)下的实测精度。\n\n在这种情况下，不再只使用少数几个点上的精度，而是通过对**每个级别**的精度进行插值来计算AP，即取那些召回值大于或等于![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_ffbbc5de35b8.png+1)的最高精度。这样我们就能计算出曲线下方的估计面积。\n\n为了更清楚地说明，我们提供了一个比较两种插值方法的示例。\n\n\n#### 一个图解示例 \n\n通过一个例子可以帮助我们更好地理解插值平均精度的概念。考虑以下检测结果：\n  \n\u003C!--- 图片样本1 --->\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_793336302ec8.png\" align=\"center\"\u002F>\u003C\u002Fp>\n  \n这里有7张图片，共包含15个真实标注目标（用绿色框表示）和24个检测到的目标（用红色框表示）。每个检测到的目标都有一个置信度，并用字母（A, B, ..., Y）来标识。  \n\n下表展示了各个边界框及其对应的置信度。最后一列标明了这些检测是TP还是FP。在这个例子中，如果IOU ![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_7ccf27b22b6c.png) 30%，则视为TP；否则为FP。通过观察上面的图片，我们可以大致判断哪些检测是TP，哪些是FP。\n\n\u003C!--- 表格1 --->\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_9b27a9cf247f.png\" align=\"center\"\u002F>\u003C\u002Fp>\n\n\u003C!---\n| 图片 | 检测 | 置信度 | TP 或 FP |\n|:------:|:----------:|:-----------:|:--------:|\n| 图片1 | A | 88% | FP |\n| 图片1 | B | 70% | TP |\n| 图片1 | C | 80% | FP |\n| 图片2 | D | 71% | FP |\n| 图片2 | E | 54% | TP |\n| 图片2 | F | 74% | FP |\n| 图片3 | G | 18% | TP |\n| 图片3 | H | 67% | FP |\n| 图片3 | I | 38% | FP |\n| 图片3 | J | 91% | TP |\n| 图片3 | K | 44% | FP |\n| 图片4 | L | 35% | FP |\n| 图片4 | M | 78% | FP |\n| 图片4 | N | 45% | FP |\n| 图片4 | O | 14% | FP |\n| 图片5 | P | 62% | TP |\n| 图片5 | Q | 44% | FP |\n| 图片5 | R | 95% | TP |\n| 图片5 | S | 23% | FP |\n| 图片6 | T | 45% | FP |\n| 图片6 | U | 84% | FP |\n| 图片6 | V | 43% | FP |\n| 图片7 | X | 48% | TP |\n| 图片7 | Y | 95% | FP |\n--->\n\n在某些图片中，多个检测框会与同一个真实标注框重叠（图片2、3、4、5、6和7）。对于这种情况，我们以具有最高IOU的预测框作为TP（例如，在图片1中，“E”是TP而“D”是FP，因为E与真实标注框的IOU大于D与真实标注框的IOU）。这一规则也适用于PASCAL VOC 2012指标：“例如，对单个物体的5次正确检测，只计为1次正确检测和4次错误检测”。\n\n精确率与召回率曲线是通过计算累计TP或FP检测的精确率和召回率来绘制的。为此，我们首先需要按照置信度对检测结果进行排序，然后为每次累积检测计算精确率和召回率，如下表所示（请注意，对于召回率的计算，分母项（“累计TP + 累计FN”或“所有真实标注”）始终为15，因为无论检测结果如何，真实标注框的数量都是固定的）：\n\n\u003C!--- 表格2 --->\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_c2173c9b088c.png\" align=\"center\"\u002F>\u003C\u002Fp>\n\n\u003C!---\n| 图片 | 检测 | 置信度 | TP | FP | 累计TP | 累计FP | 精确率 | 召回率 |\n|:------:|:----------:|:-----------:|:---:|:--:|:------:|:------:|:---------:|:------:|\n| 图片5 | R | 95% | 1 | 0 | 1 | 0 | 1.0000 | 0.0667 |\n| 图片7 | Y | 95% | 0 | 1 | 1 | 1 | 0.5000 | 0.6667 |\n| 图片3 | J | 91% | 1 | 0 | 2 | 1 | 0.6667 | 0.1333 |\n| 图片1 | A | 88% | 0 | 1 | 2 | 2 | 0.5000 | 0.1333 |\n| 图片6 | U | 84% | 0 | 1 | 2 | 3 | 0.4000 | 0.1333 |\n| 图片1 | C | 80% | 0 | 1 | 2 | 4 | 0.3333 | 0.1333 |\n| 图片4 | M | 78% | 0 | 1 | 2 | 5 | 0.2857 | 0.1333 |\n| 图片2 | F | 74% | 0 | 1 | 2 | 6 | 0.2500 | 0.1333 |\n| 图片2 | D | 71% | 0 | 1 | 2 | 7 | 0.2222 | 0.1333 |\n| 图片1 | B | 70% | 1 | 0 | 3 | 7 | 0.3000 | 0.2000 |\n| 图片3 | H | 67% | 0 | 1 | 3 | 8 | 0.2727 | 0.2000 |\n| 图片5 | P | 62% | 1 | 0 | 4 | 8 | 0.3333 | 0.2666 |\n| 图片2 | E | 54% | 1 | 0 | 5 | 8 | 0.3846 | 0.3333 |\n| 图片7 | X | 48% | 1 | 0 | 6 | 8 | 0.4285 | 0.4000 |\n| 图片4 | N | 45% | 0 | 1 | 6 | 9 | 0.7000 | 0.4000 |\n| 图片6 | T | 45% | 0 | 1 | 6 | 10 | 0.3750 | 0.4000 |\n| 图片3 | K | 44% | 0 | 1 | 6 | 11 | 0.3529 | 0.4000 |\n| 图片5 | Q | 44% | 0 | 1 | 6 | 12 | 0.3333 | 0.4000 |\n| 图片6 | V | 43% | 0 | 1 | 6 | 13 | 0.3157 | 0.4000 |\n| 图片3 | I | 38% | 0 | 1 | 6 | 14 | 0.3000 | 0.4000 |\n| 图片4 | L | 35% | 0 | 1 | 6 | 15 | 0.2857 | 0.4000 |\n| 图片5 | S | 23% | 0 | 1 | 6 | 16 | 0.2727 | 0.4000 |\n| 图片3 | G | 18% | 1 | 0 | 7 | 16 | 0.3043 | 0.4666 |\n| 图片4 | O | 14% | 0 | 1 | 7 | 17 | 0.2916 | 0.4666 |\n--->\n \n以第二行（图片7）为例进行计算：精确率 = TP\u002F(TP+FP) = 1\u002F2 = 0.5，召回率 = TP\u002F(TP+FN) = 1\u002F15 = 0.066。\n\n绘制精确率和召回率的数值后，我们得到如下的*精确率与召回率曲线*：\n\n\u003C!--- 精确率与召回率图表 --->\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_fd0bd7bcfefd.png\" align=\"center\"\u002F>\n\u003C\u002Fp>\n \n正如前面提到的，测量插值平均精度有两种不同的方法：**11点插值**和**对所有点进行插值**。下面我们对这两种方法进行比较：\n\n#### 计算11点插值\n\n11点插值平均精度的思想是在一组11个召回率水平（0, 0.1, ..., 1）上对精度进行平均。插值后的精度值通过取当前召回率以上所有召回率对应的最高精度来获得，具体如下：\n\n\u003C!--- 插值精度曲线 --->\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_16275079aee8.png\" align=\"center\"\u002F>\n\u003C\u002Fp>\n\n通过应用11点插值法，我们得到：\n\n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_aa250fbfeb36.png)  \n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_98bbd46df03f.png)  \n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_d15076925480.png)\n\n\n#### 计算所有点上的插值\n通过对所有点进行插值，平均精度（AP）可以被解释为精度-召回率曲线的近似AUC。其目的是减少曲线中波动的影响。通过应用前面介绍的公式，我们可以计算出各个区域的面积，如下所示。此外，我们也可以通过观察从最高召回率（0.4666）到0的召回率变化（即从右向左看图），收集每个召回率对应的最高精度值，从而直观地看到插值后的精度点，如图下所示：\n    \n\u003C!--- 插值精度 AUC --->\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_84aabdab84c9.png\" align=\"center\"\u002F>\n\u003C\u002Fp>\n  \n观察上图，我们可以将AUC分为4个区域（A1、A2、A3和A4）：\n  \n\u003C!--- 插值精度 AUC --->\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_b8d4689ea3a2.png\" align=\"center\"\u002F>\n\u003C\u002Fp>\n\n计算总面积后，即可得到AP：\n\n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_a177615e7c31.png)  \n  \n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_943f86405365.png)  \n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_4fa717252508.png)  \n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_3b44c843dd0d.png)  \n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_4626426bf0b4.png)  \n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_f8790b340b4e.png)  \n   \n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_b3dce126cb74.png)  \n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_93a881607959.png)  \n![](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_readme_149aebbe95e2.png)  \n\n两种不同插值方法的结果略有差异：每点插值法得到24.56%，而11点插值法得到26.84%。\n\n我们的默认实现与PASCAL VOC一致，采用每点插值法。如果希望使用11点插值法，只需将使用参数`method=MethodAveragePrecision.EveryPointInterpolation`的函数改为`method=MethodAveragePrecision.ElevenPointInterpolation`即可。\n\n若想复现这些结果，请参阅**[示例2](https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002FObject-Detection-Metrics\u002Ftree\u002Fmaster\u002Fsamples\u002Fsample_2\u002F)**。\n\u003C!--为了评估你的检测结果，你只需要一个简单的`Detection`对象列表。`Detection`对象是一个非常简单的类，包含检测到的目标的类别ID、类别置信度以及边界框坐标。同样的结构也用于真实标注的检测结果。-->\n\n\n## 如何使用本项目\n\n本项目旨在以非常简便的方式评估你的检测结果。如果你希望使用最常用的目标检测指标来评估你的算法，那么这里正是你需要的地方。\n\n[示例1](https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002FObject-Detection-Metrics\u002Ftree\u002Fmaster\u002Fsamples\u002Fsample_1) 和 [示例2](https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002FObject-Detection-Metrics\u002Ftree\u002Fmaster\u002Fsamples\u002Fsample_2) 是实际案例，展示了如何直接调用本项目的核心函数，从而在使用指标时提供更大的灵活性。但如果你不想花时间理解我们的代码，可以按照以下说明轻松评估你的检测结果：\n\n请按照以下步骤开始评估你的检测结果：\n\n1. [创建真实标注文件](#create-the-ground-truth-files)\n2. [创建你的检测文件](#create-your-detection-files)\n3. 对于**Pascal VOC指标**，运行命令：`python pascalvoc.py`  \n   如果你想复现上面的例子，运行命令：`python pascalvoc.py -t 0.3`\n4. （可选）[你可以使用参数来控制IOU阈值、边界框格式等](#optional-arguments)\n\n### 创建真实标注文件\n\n- 在文件夹**groundtruths\u002F**中，为每张图片创建一个单独的真实标注文本文件。\n- 每个文件中的每一行应采用以下格式：\u003Cclass_name> \u003Cleft> \u003Ctop> \u003Cright> \u003Cbottom>。    \n- 例如，图像“2008_000034.jpg”的真实标注边界框在文件“2008_000034.txt”中表示如下：\n  ```\n  bottle 6 234 45 362\n  person 1 156 103 336\n  person 36 111 198 416\n  person 91 42 338 500\n  ```\n    \n如果你愿意，也可以使用\u003Cclass_name> \u003Cleft> \u003Ctop> \u003Cwidth> \u003Cheight>的格式（详见此处[**\\***](#asterisk)的使用方法）。在这种情况下，“2008_000034.txt”将表示为：\n  ```\n  bottle 6 234 39 128\n  person 1 156 102 180\n  person 36 111 162 305\n  person 91 42 247 458\n  ```\n\n### 创建你的检测文件\n\n- 在文件夹**detections\u002F**中，为每张图片创建一个单独的检测文本文件。\n- 检测文件的名称必须与其对应的真实标注文件匹配（例如，“detections\u002F2008_000182.txt”表示与“groundtruths\u002F2008_000182.txt”相对应的检测结果）。\n- 每个检测文件中的每一行应采用以下格式：\u003Cclass_name> \u003Cconfidence> \u003Cleft> \u003Ctop> \u003Cright> \u003Cbottom>（详见此处[**\\***](#asterisk)的使用方法）。\n- 例如，“2008_000034.txt”：\n    ```\n    bottle 0.14981 80 1 295 500  \n    bus 0.12601 36 13 404 316  \n    horse 0.12526 430 117 500 307  \n    pottedplant 0.14585 212 78 292 118  \n    tvmonitor 0.070565 388 89 500 196  \n    ```\n\n同样，如果你更喜欢，也可以使用\u003Cclass_name> \u003Cconfidence> \u003Cleft> \u003Ctop> \u003Cwidth> \u003Cheight>的格式。\n\n### 可选参数\n\n可选参数：\n\n| 参数                          | 描述 | 示例 | 默认值 |\n|:-------------:|:-----------:|:-----------:|:-----------:|\n| `-h`,\u003Cbr>`--help ` | 显示帮助信息 | `python pascalvoc.py -h` | |  \n|  `-v`,\u003Cbr>`--version` | 检查版本 | `python pascalvoc.py -v` | |  \n| `-gt`,\u003Cbr>`--gtfolder` | 包含真实框文件的文件夹 | `python pascalvoc.py -gt \u002Fhome\u002Fwhatever\u002Fmy_groundtruths\u002F` | `\u002FObject-Detection-Metrics\u002Fgroundtruths`|  \n| `-det`,\u003Cbr>`--detfolder` | 包含检测框文件的文件夹 | `python pascalvoc.py -det \u002Fhome\u002Fwhatever\u002Fmy_detections\u002F` | `\u002FObject-Detection-Metrics\u002Fdetections\u002F`|  \n| `-t`,\u003Cbr>`--threshold` | IOU 阈值，用于判断检测结果是真正例还是假正例 | `python pascalvoc.py -t 0.75` | `0.50` |  \n| `-gtformat` | 真实框坐标格式 [**\\***](#asterisk) | `python pascalvoc.py -gtformat xyrb` | `xywh` |\n| `-detformat` | 检测框坐标格式 [**\\***](#asterisk) | `python pascalvoc.py -detformat xyrb` | `xywh` | |  \n| `-gtcoords` | 真实框坐标参考。\u003Cbr>如果标注的坐标是相对于图像尺寸的（如 YOLO 中使用），则设置为 `rel`。\u003Cbr>如果坐标是绝对值，不依赖于图像尺寸，则设置为 `abs` |  `python pascalvoc.py -gtcoords rel` | `abs` |  \n| `-detcoords` | 检测框坐标参考。\u003Cbr>如果坐标是相对于图像尺寸的（如 YOLO 中使用），则设置为 `rel`。\u003Cbr>如果坐标是绝对值，不依赖于图像尺寸，则设置为 `abs` | `python pascalvoc.py -detcoords rel` | `abs` |  \n| `-imgsize ` | 图像尺寸，格式为 `width,height` \u003Cint,int>。\u003Cbr>如果 `-gtcoords` 或 `-detcoords` 设置为 `rel`，则此参数为必填项 | `python pascalvoc.py -imgsize 600,400` |  \n| `-sp`,\u003Cbr>`--savepath` | 保存图表的文件夹 | `python pascalvoc.py -sp \u002Fhome\u002Fwhatever\u002Fmy_results\u002F` | `Object-Detection-Metrics\u002Fresults\u002F` |  \n| `-np`,\u003Cbr>`--noplot` | 如果存在，则执行过程中不显示任何图表 | `python pascalvoc.py -np` | 不显示。\u003Cbr>因此，默认会显示图表 |  \n\n\u003Ca name=\"asterisk\"> \u003C\u002Fa>\n(**\\***) 如果格式为 `\u003Cleft> \u003Ctop> \u003Cwidth> \u003Cheight>`，则设置 `-gtformat xywh` 和\u002F或 `-detformat xywh`。\u003Cbr>如果格式为 `\u003Cleft> \u003Ctop> \u003Cright> \u003Cbottom>`，则设置 `-gtformat xyrb` 和\u002F或 `-detformat xyrb`。\n  \n## 参考文献\n\n* 精确率-召回率曲线与 ROC 曲线之间的关系（Jesse Davis 和 Mark Goadrich）  \n威斯康星大学计算机科学系及生物统计学与医学信息学系  \nhttp:\u002F\u002Fpages.cs.wisc.edu\u002F~jdavis\u002Fdavisgoadrichcamera2.pdf\n\n* PASCAL 视觉目标类别（VOC）挑战赛  \nhttp:\u002F\u002Fciteseerx.ist.psu.edu\u002Fviewdoc\u002Fdownload?doi=10.1.1.157.5766&rep=rep1&type=pdf\n\n* 排序检索结果的评估（Salton 和 McGill，1986 年）  \nhttps:\u002F\u002Fwww.amazon.com\u002FIntroduction-Information-Retrieval-COMPUTER-SCIENCE\u002Fdp\u002F0070544840  \nhttps:\u002F\u002Fnlp.stanford.edu\u002FIR-book\u002Fhtml\u002Fhtmledition\u002Fevaluation-of-ranked-retrieval-results-1.html","# Object-Detection-Metrics 快速上手指南\n\nObject-Detection-Metrics 是一个开源工具包，旨在统一并简化目标检测算法的性能评估。它实现了 PASCAL VOC、COCO 等主流竞赛使用的评估指标（如 mAP），无需将数据转换为复杂的 XML 或 JSON 格式，直接支持简单的文本输入。\n\n## 环境准备\n\n在开始之前，请确保您的开发环境满足以下要求：\n\n*   **操作系统**：Linux, macOS 或 Windows\n*   **Python 版本**：Python 3.6 或更高版本\n*   **前置依赖**：\n    *   `numpy`\n    *   `matplotlib` (用于绘制 Precision-Recall 曲线)\n    *   `scipy` (可选，用于部分插值计算)\n\n建议创建一个虚拟环境以避免依赖冲突：\n\n```bash\npython -m venv od_metrics_env\nsource od_metrics_env\u002Fbin\u002Factivate  # Windows 用户使用: od_metrics_env\\Scripts\\activate\n```\n\n安装依赖库：\n\n```bash\npip install numpy matplotlib scipy\n# 国内用户推荐使用清华源加速安装\n# pip install -i https:\u002F\u002Fpypi.tuna.tsinghua.edu.cn\u002Fsimple numpy matplotlib scipy\n```\n\n## 安装步骤\n\n该项目主要通过直接克隆源码使用，无需通过 pip 安装包。\n\n1.  **克隆仓库**：\n    ```bash\n    git clone https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002FObject-Detection-Metrics.git\n    cd Object-Detection-Metrics\n    ```\n\n2.  **验证目录结构**：\n    进入目录后，您应看到 `metrics.py`（核心逻辑）、`samples`（示例数据）等文件。\n\n## 基本使用\n\n该工具的核心使用流程分为三步：**准备数据** -> **调用评估函数** -> **获取结果**。\n\n### 1. 数据格式准备\n\n工具接受简单的文本文件或内存列表作为输入，无需转换格式。每行代表一个检测框或真实框。\n\n**格式说明**：\n`\u003Cimage_name> \u003Cclass_id> \u003Cconfidence> \u003Cx_left> \u003Cy_top> \u003Cx_right> \u003Cy_bottom>`\n\n*   **Ground Truth (真实值)**：`confidence` 字段通常设为 0 或忽略（代码中处理为无置信度）。\n*   **Detections (预测值)**：包含模型输出的置信度。\n\n**示例文件 (`groundtruths.txt`)**:\n```text\nimg_001 0 0 10 10 50 50\nimg_001 1 0 20 20 60 60\nimg_002 0 0 5 5 45 45\n```\n\n**示例文件 (`detections.txt`)**:\n```text\nimg_001 0 0.95 12 12 52 52\nimg_001 0 0.80 100 100 150 150\nimg_002 0 0.85 6 6 46 46\n```\n\n### 2. 运行评估脚本\n\n项目根目录下提供了示例脚本 `main.py`（或在 `samples` 文件夹中），您可以直接运行它来测试功能，或者参考其逻辑编写自己的评估代码。\n\n**直接运行示例：**\n\n```bash\npython main.py\n```\n\n*注：如果根目录没有直接的 `main.py`，请运行 `python samples\u002Fsample_1.py` (具体文件名视版本而定，通常 samples 目录下有演示如何加载 txt 文件并计算 mAP 的脚本)。*\n\n### 3. 代码集成示例\n\n如果您想在 Python 代码中直接调用，可以参考以下简化逻辑：\n\n```python\nfrom metrics import *\n\n# 创建评估器对象\n# 参数：IOU 阈值 (例如 0.5), 是否使用 COCO 风格评估等\nevaluator = Evaluator()\n\n# 添加真实框 (Ground Truth)\n# 格式：(imageName, classId, x_left, y_top, x_right, y_bottom)\ngt = GroundTruths()\ngt.addGroundTruth(\"img_001\", \"class_0\", 10, 10, 50, 50)\n\n# 添加检测结果 (Detections)\n# 格式：(imageName, classId, confidence, x_left, y_top, x_right, y_bottom)\ndet = Detections()\ndet.addDetection(\"img_001\", \"class_0\", 0.95, 12, 12, 52, 52)\n\n# 执行评估\nresults = evaluator.GetPascalVOCMetrics(gt, det, IOUThreshold=0.5)\n\n# 打印平均精度 (AP)\nfor r in results:\n    print(f\"Class: {r['class']}, AP: {r['AP']}\")\n    \n# 绘制 Precision-Recall 曲线 (可选)\nplot_precision_recall_curve(results)\n```\n\n### 注意事项\n*   **坐标系统**：默认支持 `(x_left, y_top, x_right, y_bottom)` 格式。\n*   **新版提示**：如果您需要评估视频数据 (STT-AP) 或使用更完善的图形界面 (UI)，请参考 README 中提到的新版本仓库：[review_object_detection_metrics](https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002Freview_object_detection_metrics)。","某自动驾驶初创团队的算法工程师正在迭代其车辆检测模型，急需在不同数据集上公平对比新旧版本的性能差异。\n\n### 没有 Object-Detection-Metrics 时\n- **标准混乱导致结果不可比**：团队参考不同论文实现了多种指标（如 VOC mAP 与 COCO AP），因计算逻辑细微差别，导致同一模型在不同脚本下得分不一，无法判断优化是否有效。\n- **数据格式转换耗时费力**：官方评估代码通常强制要求将标注文件转换为特定的 XML 或复杂 JSON 格式，工程师需编写大量胶水代码进行预处理，极易引入人为错误。\n- **复现权威基准困难**：缺乏经过验证的参考实现，自行编写的评估逻辑难以确保与顶级竞赛（如 COCO Challenge）的官方结果完全一致，削弱了技术报告的可信度。\n- **调试与可视化缺失**：手动计算指标时难以直观生成精度 - 召回率曲线（Precision-Recall Curve），定位模型在特定置信度阈值下的失效原因变得如同“盲人摸象”。\n\n### 使用 Object-Detection-Metrics 后\n- **统一度量衡确保公平性**：直接调用封装好的函数即可计算主流竞赛指标，消除了实现歧义，确保新旧模型在同一把“尺子”下进行精准对比。\n- **输入灵活零负担**：支持直接传入简单的边界框坐标列表，无需修改现有数据管道去适配复杂的文件格式，大幅缩短了从训练到评估的周期。\n- **结果权威可信赖**：该工具的核心算法已与官方实现严格对齐并经过学术验证，输出的每一项数据都能经得起同行评审和业界基准的检验。\n- **分析洞察一目了然**：一键生成标准的精度 - 召回率曲线及详细统计报表，帮助工程师快速定位模型在漏检或误报上的具体短板，指导针对性调优。\n\nObject-Detection-Metrics 通过提供标准化、免配置且经学术验证的评估方案，让研发团队从繁琐的指标实现中解脱，专注于核心算法的真正突破。","https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Frafaelpadilla_Object-Detection-Metrics_c33892c3.png","rafaelpadilla","Rafael Padilla","https:\u002F\u002Foss.gittoolsai.com\u002Favatars\u002Frafaelpadilla_2719fbec.jpg","Computer Vision, Machine Learning and Anomaly Detection",null,"https:\u002F\u002Fgithub.com\u002Frafaelpadilla",[82],{"name":83,"color":84,"percentage":85},"Python","#3572A5",100,5099,1032,"2026-04-02T14:11:09","MIT",1,"未说明 (跨平台，依赖 Python 环境)","未说明 (该工具主要用于计算评估指标，通常仅需 CPU 即可运行)","未说明",{"notes":95,"python":96,"dependencies":97},"该工具是一个用于计算目标检测指标（如 mAP、Precision-Recall 曲线）的评估套件，而非深度学习训练框架。它不强制要求特定的深度学习框架（如 PyTorch 或 TensorFlow），而是接受通用的边界框坐标输入。官方实现主要基于 Python 和 Matlab。新版工具提供了用户界面 (UI) 并支持 COCO 所有指标及视频评估指标 (STT-AP)。","未说明 (需安装 Python 以运行脚本)",[98,99],"matplotlib (用于绘制曲线)","numpy (用于数值计算)",[14,13],[102,103,104,105,106,107,108],"metrics","object-detection","average-precision","mean-average-precision","bounding-boxes","precision-recall","pascal-voc","2026-03-27T02:49:30.150509","2026-04-06T08:41:34.637384",[112,117,122,127,132,137],{"id":113,"question_zh":114,"answer_zh":115,"source_url":116},17677,"当多个检测框重叠同一个真实框（Ground Truth）时，如何选择真阳性（TP）？","当有多个检测框与同一个真实框重叠且 IoU 满足阈值时，应选择 IoU 最高的那个检测框作为真阳性（TP）。虽然有些观点认为应选择置信度最高的，但该库的标准实现是依据最高 IoU 来判定。","https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002FObject-Detection-Metrics\u002Fissues\u002F46",{"id":118,"question_zh":119,"answer_zh":120,"source_url":121},17678,"遇到 'AssertionError: assert iou >= 0' 错误该如何解决？","如果您使用的是 'xyrb' 格式（左、上、右、下），请确保文件中的第二列（Top）和第四列（Bottom）数值顺序正确。Top 值必须小于 Bottom 值（即坐标从上到下递增）。如果顺序颠倒，计算交集面积的函数会返回负数，从而触发断言错误。请交换这两列的数据。","https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002FObject-Detection-Metrics\u002Fissues\u002F53",{"id":123,"question_zh":124,"answer_zh":125,"source_url":126},17679,"计算精度和召回率时，置信度阈值（Confidence Threshold）有什么规则吗？","标准的 mAP 计算方法（如 PASCAL VOC 和 COCO 官方代码）不会手动设定单一的置信度阈值来过滤结果，而是利用所有检测结果的置信度分数进行排序，并遍历所有可能的阈值来计算曲线。某些实现（如部分 sklearn 用法或非官方复现）可能会手动设置阈值或忽略低置信度检测，这不符合标准评估规范，可能导致结果偏差。建议遵循官方标准，不要随意截断置信度范围。","https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002FObject-Detection-Metrics\u002Fissues\u002F160",{"id":128,"question_zh":129,"answer_zh":130,"source_url":131},17680,"如何使用该工具获取 COCO 定义的平均召回率（AR, Average Recall）？","该库当前版本尚未直接支持 COCO 定义的 AR 指标。AR 是指在固定每张图像的检测数量（如 1、10 或 100 个）下的最大召回率，并在不同类别和 11 个 IoU 阈值（0.50 到 0.95，步长 0.05）上进行平均。您可以参考 COCO 的评估说明自行基于此库代码实现，或使用 COCO 官方评估工具进行验证。","https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002FObject-Detection-Metrics\u002Fissues\u002F22",{"id":133,"question_zh":134,"answer_zh":135,"source_url":136},17681,"如果没有预测框的置信度分数（confidence scores），能使用此代码计算精度和召回率吗？","不能。对于检测类型（bbType='Detection'），必须提供类别置信度值（classConfidence）。如果您的模型输出（如某些 YOLO 或 DPM 变体）不包含置信度，您需要先修改数据生成脚本以添加该字段，或者联系维护者提供特定输出格式的转换帮助，否则代码会报错。","https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002FObject-Detection-Metrics\u002Fissues\u002F51",{"id":138,"question_zh":139,"answer_zh":140,"source_url":141},17682,"如何生成混淆矩阵（Confusion Matrix）？","该仓库目前尚未实现生成混淆矩阵的功能。维护者表示短期内没有计划添加此功能，除非有社区贡献者提交相关代码实现。目前无法直接通过运行脚本来生成此类图表。","https:\u002F\u002Fgithub.com\u002Frafaelpadilla\u002FObject-Detection-Metrics\u002Fissues\u002F120",[143,148],{"id":144,"version":145,"summary_zh":146,"released_at":147},107984,"v0.2","使用此版本，可通过以下指标评估您的检测结果：\n\n* VOC Pascal **精确率-召回率曲线**\n* VOC Pascal **平均精度**\n\n与 v0.1 版本相比，本版本的改进包括：\n* 新增可选的**11点插值**方法，同时仍将所有点插值作为默认设置。","2019-01-31T15:57:55",{"id":149,"version":150,"summary_zh":151,"released_at":152},107985,"v0.1","使用此版本，可通过以下指标评估您的检测结果：\n\n* VOC Pascal **精确率-召回率曲线**\n* VOC Pascal **平均精度**","2018-06-22T00:42:26"]