[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"similar-understandable-machine-intelligence-lab--Quantus":3,"tool-understandable-machine-intelligence-lab--Quantus":61},[4,18,26,36,44,53],{"id":5,"name":6,"github_repo":7,"description_zh":8,"stars":9,"difficulty_score":10,"last_commit_at":11,"category_tags":12,"status":17},4358,"openclaw","openclaw\u002Fopenclaw","OpenClaw 是一款专为个人打造的本地化 AI 助手，旨在让你在自己的设备上拥有完全可控的智能伙伴。它打破了传统 AI 助手局限于特定网页或应用的束缚，能够直接接入你日常使用的各类通讯渠道，包括微信、WhatsApp、Telegram、Discord、iMessage 等数十种平台。无论你在哪个聊天软件中发送消息，OpenClaw 都能即时响应，甚至支持在 macOS、iOS 和 Android 设备上进行语音交互，并提供实时的画布渲染功能供你操控。\n\n这款工具主要解决了用户对数据隐私、响应速度以及“始终在线”体验的需求。通过将 AI 部署在本地，用户无需依赖云端服务即可享受快速、私密的智能辅助，真正实现了“你的数据，你做主”。其独特的技术亮点在于强大的网关架构，将控制平面与核心助手分离，确保跨平台通信的流畅性与扩展性。\n\nOpenClaw 非常适合希望构建个性化工作流的技术爱好者、开发者，以及注重隐私保护且不愿被单一生态绑定的普通用户。只要具备基础的终端操作能力（支持 macOS、Linux 及 Windows WSL2），即可通过简单的命令行引导完成部署。如果你渴望拥有一个懂你",349277,3,"2026-04-06T06:32:30",[13,14,15,16],"Agent","开发框架","图像","数据工具","ready",{"id":19,"name":20,"github_repo":21,"description_zh":22,"stars":23,"difficulty_score":10,"last_commit_at":24,"category_tags":25,"status":17},3808,"stable-diffusion-webui","AUTOMATIC1111\u002Fstable-diffusion-webui","stable-diffusion-webui 是一个基于 Gradio 构建的网页版操作界面，旨在让用户能够轻松地在本地运行和使用强大的 Stable Diffusion 图像生成模型。它解决了原始模型依赖命令行、操作门槛高且功能分散的痛点，将复杂的 AI 绘图流程整合进一个直观易用的图形化平台。\n\n无论是希望快速上手的普通创作者、需要精细控制画面细节的设计师，还是想要深入探索模型潜力的开发者与研究人员，都能从中获益。其核心亮点在于极高的功能丰富度：不仅支持文生图、图生图、局部重绘（Inpainting）和外绘（Outpainting）等基础模式，还独创了注意力机制调整、提示词矩阵、负向提示词以及“高清修复”等高级功能。此外，它内置了 GFPGAN 和 CodeFormer 等人脸修复工具，支持多种神经网络放大算法，并允许用户通过插件系统无限扩展能力。即使是显存有限的设备，stable-diffusion-webui 也提供了相应的优化选项，让高质量的 AI 艺术创作变得触手可及。",162132,"2026-04-05T11:01:52",[14,15,13],{"id":27,"name":28,"github_repo":29,"description_zh":30,"stars":31,"difficulty_score":32,"last_commit_at":33,"category_tags":34,"status":17},1381,"everything-claude-code","affaan-m\u002Feverything-claude-code","everything-claude-code 是一套专为 AI 编程助手（如 Claude Code、Codex、Cursor 等）打造的高性能优化系统。它不仅仅是一组配置文件，而是一个经过长期实战打磨的完整框架，旨在解决 AI 代理在实际开发中面临的效率低下、记忆丢失、安全隐患及缺乏持续学习能力等核心痛点。\n\n通过引入技能模块化、直觉增强、记忆持久化机制以及内置的安全扫描功能，everything-claude-code 能显著提升 AI 在复杂任务中的表现，帮助开发者构建更稳定、更智能的生产级 AI 代理。其独特的“研究优先”开发理念和针对 Token 消耗的优化策略，使得模型响应更快、成本更低，同时有效防御潜在的攻击向量。\n\n这套工具特别适合软件开发者、AI 研究人员以及希望深度定制 AI 工作流的技术团队使用。无论您是在构建大型代码库，还是需要 AI 协助进行安全审计与自动化测试，everything-claude-code 都能提供强大的底层支持。作为一个曾荣获 Anthropic 黑客大奖的开源项目，它融合了多语言支持与丰富的实战钩子（hooks），让 AI 真正成长为懂上",158594,2,"2026-04-16T23:34:05",[14,13,35],"语言模型",{"id":37,"name":38,"github_repo":39,"description_zh":40,"stars":41,"difficulty_score":32,"last_commit_at":42,"category_tags":43,"status":17},2271,"ComfyUI","Comfy-Org\u002FComfyUI","ComfyUI 是一款功能强大且高度模块化的视觉 AI 引擎，专为设计和执行复杂的 Stable Diffusion 图像生成流程而打造。它摒弃了传统的代码编写模式，采用直观的节点式流程图界面，让用户通过连接不同的功能模块即可构建个性化的生成管线。\n\n这一设计巧妙解决了高级 AI 绘图工作流配置复杂、灵活性不足的痛点。用户无需具备编程背景，也能自由组合模型、调整参数并实时预览效果，轻松实现从基础文生图到多步骤高清修复等各类复杂任务。ComfyUI 拥有极佳的兼容性，不仅支持 Windows、macOS 和 Linux 全平台，还广泛适配 NVIDIA、AMD、Intel 及苹果 Silicon 等多种硬件架构，并率先支持 SDXL、Flux、SD3 等前沿模型。\n\n无论是希望深入探索算法潜力的研究人员和开发者，还是追求极致创作自由度的设计师与资深 AI 绘画爱好者，ComfyUI 都能提供强大的支持。其独特的模块化架构允许社区不断扩展新功能，使其成为当前最灵活、生态最丰富的开源扩散模型工具之一，帮助用户将创意高效转化为现实。",108322,"2026-04-10T11:39:34",[14,15,13],{"id":45,"name":46,"github_repo":47,"description_zh":48,"stars":49,"difficulty_score":32,"last_commit_at":50,"category_tags":51,"status":17},6121,"gemini-cli","google-gemini\u002Fgemini-cli","gemini-cli 是一款由谷歌推出的开源 AI 命令行工具，它将强大的 Gemini 大模型能力直接集成到用户的终端环境中。对于习惯在命令行工作的开发者而言，它提供了一条从输入提示词到获取模型响应的最短路径，无需切换窗口即可享受智能辅助。\n\n这款工具主要解决了开发过程中频繁上下文切换的痛点，让用户能在熟悉的终端界面内直接完成代码理解、生成、调试以及自动化运维任务。无论是查询大型代码库、根据草图生成应用，还是执行复杂的 Git 操作，gemini-cli 都能通过自然语言指令高效处理。\n\n它特别适合广大软件工程师、DevOps 人员及技术研究人员使用。其核心亮点包括支持高达 100 万 token 的超长上下文窗口，具备出色的逻辑推理能力；内置 Google 搜索、文件操作及 Shell 命令执行等实用工具；更独特的是，它支持 MCP（模型上下文协议），允许用户灵活扩展自定义集成，连接如图像生成等外部能力。此外，个人谷歌账号即可享受免费的额度支持，且项目基于 Apache 2.0 协议完全开源，是提升终端工作效率的理想助手。",100752,"2026-04-10T01:20:03",[52,13,15,14],"插件",{"id":54,"name":55,"github_repo":56,"description_zh":57,"stars":58,"difficulty_score":32,"last_commit_at":59,"category_tags":60,"status":17},4721,"markitdown","microsoft\u002Fmarkitdown","MarkItDown 是一款由微软 AutoGen 团队打造的轻量级 Python 工具，专为将各类文件高效转换为 Markdown 格式而设计。它支持 PDF、Word、Excel、PPT、图片（含 OCR）、音频（含语音转录）、HTML 乃至 YouTube 链接等多种格式的解析，能够精准提取文档中的标题、列表、表格和链接等关键结构信息。\n\n在人工智能应用日益普及的今天，大语言模型（LLM）虽擅长处理文本，却难以直接读取复杂的二进制办公文档。MarkItDown 恰好解决了这一痛点，它将非结构化或半结构化的文件转化为模型“原生理解”且 Token 效率极高的 Markdown 格式，成为连接本地文件与 AI 分析 pipeline 的理想桥梁。此外，它还提供了 MCP（模型上下文协议）服务器，可无缝集成到 Claude Desktop 等 LLM 应用中。\n\n这款工具特别适合开发者、数据科学家及 AI 研究人员使用，尤其是那些需要构建文档检索增强生成（RAG）系统、进行批量文本分析或希望让 AI 助手直接“阅读”本地文件的用户。虽然生成的内容也具备一定可读性，但其核心优势在于为机器",93400,"2026-04-06T19:52:38",[52,14],{"id":62,"github_repo":63,"name":64,"description_en":65,"description_zh":66,"ai_summary_zh":66,"readme_en":67,"readme_zh":68,"quickstart_zh":69,"use_case_zh":70,"hero_image_url":71,"owner_login":72,"owner_name":73,"owner_avatar_url":74,"owner_bio":75,"owner_company":76,"owner_location":76,"owner_email":76,"owner_twitter":77,"owner_website":76,"owner_url":78,"languages":79,"stars":88,"forks":89,"last_commit_at":90,"license":91,"difficulty_score":92,"env_os":93,"env_gpu":94,"env_ram":93,"env_deps":95,"category_tags":107,"github_topics":109,"view_count":32,"oss_zip_url":76,"oss_zip_packed_at":76,"status":17,"created_at":118,"updated_at":119,"faqs":120,"releases":141},8353,"understandable-machine-intelligence-lab\u002FQuantus","Quantus","Quantus is an eXplainable AI toolkit for responsible evaluation of neural network explanations","Quantus 是一款专为可解释人工智能（XAI）设计的开源评估工具包，旨在帮助开发者和研究人员科学、负责地衡量神经网络解释方法的有效性。随着深度学习模型日益复杂，理解模型“为何做出某种预测”变得至关重要，但市面上众多的解释算法往往缺乏统一的评估标准。Quantus 正是为了解决这一痛点而生，它提供了一套系统化的指标体系，让用户能够量化评估不同解释方法的可靠性与准确性。\n\n该工具特别适合 AI 研究人员、算法工程师以及需要验证模型透明度的数据科学家使用。Quantus 内置了超过 35 种评估指标，涵盖六大类别，支持图像、时间序列及表格等多种数据类型，并兼容 PyTorch 和 TensorFlow 主流框架。其独特亮点在于高效的批量计算实现，可将现有忠诚度指标的评估速度提升 12 倍，大幅缩短实验周期。此外，Quantus 还无缝集成 Captum、tf-explain 等流行库，并持续更新如 EfficientMPRT 等前沿度量方法。无论是学术研究还是工业级模型审计，Quantus 都能为用户提供严谨的评估依据，推动可信 AI 的发展。","\u003Cp align=\"center\">\n  \u003Cimg width=\"350\" src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Funderstandable-machine-intelligence-lab_Quantus_readme_cd4ad09d9a34.png\">\n\u003C\u002Fp>\n\u003C!--\u003Ch1 align=\"center\">\u003Cb>Quantus\u003C\u002Fb>\u003C\u002Fh1>-->\n\u003Ch3 align=\"center\">\u003Cb>A toolkit to evaluate neural network explanations\u003C\u002Fb>\u003C\u002Fh3>\n\u003Cp align=\"center\">\n  PyTorch and TensorFlow\n\n[![Getting started!](https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg)](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002Ftutorials\u002FTutorial_ImageNet_Example_All_Metrics.ipynb)\n[![Launch Tutorials](https:\u002F\u002Fmybinder.org\u002Fbadge_logo.svg)](https:\u002F\u002Fmybinder.org\u002Fv2\u002Fgh\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002FHEAD?labpath=tutorials)\n![Python version](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fpython-3.8%20%7C%203.9%20%7C%203.10%20%7C%203.11-blue.svg)\n[![PyPI version](https:\u002F\u002Fbadge.fury.io\u002Fpy\u002Fquantus.svg)](https:\u002F\u002Fbadge.fury.io\u002Fpy\u002Fquantus)\n[![Code style: black](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fcode%20style-black-000000.svg)](https:\u002F\u002Fgithub.com\u002Fpsf\u002Fblack)\n[![Documentation Status](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Funderstandable-machine-intelligence-lab_Quantus_readme_13d664e1afd7.png)](https:\u002F\u002Fquantus.readthedocs.io\u002Fen\u002Flatest\u002F?badge=latest)\n[![codecov.io](https:\u002F\u002Fcodecov.io\u002Fgithub\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcoverage.svg?branch=master)](https:\u002F\u002Fcodecov.io\u002Fgithub\u002Funderstandable-machine-intelligence-lab\u002FQuantus?branch=master)\n[![Downloads](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Funderstandable-machine-intelligence-lab_Quantus_readme_029878a584ab.png)](https:\u002F\u002Fpepy.tech\u002Fproject\u002Fquantus)\n\u003C!--[![Python package](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Factions\u002Fworkflows\u002Fpython-package.yml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Factions\u002Fworkflows\u002Fpython-package.yml)\n[![Code coverage](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Factions\u002Fworkflows\u002Fcodecov.yml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Factions\u002Fworkflows\u002Fcodecov.yml)\n-->\n\n_Quantus is currently under active development so carefully note the Quantus release version to ensure reproducibility of your work._\n\n[📑 Shortcut to paper!](https:\u002F\u002Fjmlr.org\u002Fpapers\u002Fvolume24\u002F22-0142\u002F22-0142.pdf)\n\nIf you want to contribute\u002F improve\u002F extend Quantus, join our [Discord](https:\u002F\u002Fdiscord.gg\u002FHB77krUE)!\n\n## News and Highlights! :rocket:\n\n- 🐼 For **training data attribution** evaluation, check out [quanda](https:\u002F\u002Fgithub.com\u002Fdilyabareeva\u002Fquanda)!\n- New [batch implementation](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F351) for 12X speedup of existing faithfulness metrics (!)\n- New metrics added: [EfficientMPRT](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002Fquantus\u002Fmetrics\u002Frandomisation\u002Fefficient_mprt.py) and [SmoothMPRT](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002Fquantus\u002Fmetrics\u002Frandomisation\u002Fsmooth_mprt.py) by [Hedström et al., (2023)](https:\u002F\u002Fopenreview.net\u002Fpdf?id=vVpefYmnsG)\n- Accepted to Journal of Machine Learning Research (MLOSS), read the [paper](https:\u002F\u002Fjmlr.org\u002Fpapers\u002Fv24\u002F22-0142.html)\n- Offers more than **35+ metrics in 6 categories** for XAI evaluation\n- Supports different data types (image, time-series, tabular, NLP next up!) and models (PyTorch, TensorFlow)\n- Extended built-in support for explanation methods ([captum](https:\u002F\u002Fcaptum.ai\u002F), [tf-explain](https:\u002F\u002Ftf-explain.readthedocs.io\u002Fen\u002Flatest\u002F) and [zennit](https:\u002F\u002Fgithub.com\u002Fchr5tphr\u002Fzennit))\n\u003C!--- Released a new version [here](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Freleases) with [Python 3.7 discontinued](https:\u002F\u002Fdevguide.python.org\u002Fversions\u002F)-->\n\n## Citation\n\nIf you find this toolkit or its companion paper\n[**Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond**](https:\u002F\u002Fjmlr.org\u002Fpapers\u002Fv24\u002F22-0142.html)\ninteresting or useful in your research, use the following Bibtex annotation to cite us:\n\n```bibtex\n@article{hedstrom2023quantus,\n  author  = {Anna Hedstr{\\\"{o}}m and Leander Weber and Daniel Krakowczyk and Dilyara Bareeva and Franz Motzkus and Wojciech Samek and Sebastian Lapuschkin and Marina Marina M.{-}C. H{\\\"{o}}hne},\n  title   = {Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond},\n  journal = {Journal of Machine Learning Research},\n  year    = {2023},\n  volume  = {24},\n  number  = {34},\n  pages   = {1--11},\n  url     = {http:\u002F\u002Fjmlr.org\u002Fpapers\u002Fv24\u002F22-0142.html}\n}\n```\n\nWhen applying the individual metrics of Quantus, please make sure to also properly cite the work of the original authors (as linked below).\n\n## Table of contents\n\n* [Library overview](#library-overview)\n* [Installation](#installation)\n* [Getting started](#getting-started)\n* [Tutorials](#tutorials)\n* [Contributing](#contributing)\n\u003C!--* [Citation](#citation)-->\n\n## Library overview \n\nA simple visual comparison of eXplainable Artificial Intelligence (XAI) methods is often not sufficient to decide which explanation method works best as shown exemplarily in Figure a) for four gradient-based methods — Saliency ([Mørch et al., 1995](https:\u002F\u002Fieeexplore.ieee.org\u002Fdocument\u002F488997); [Baehrens et al., 2010](https:\u002F\u002Fwww.jmlr.org\u002Fpapers\u002Fvolume11\u002Fbaehrens10a\u002Fbaehrens10a.pdf)), Integrated Gradients ([Sundararajan et al., 2017](http:\u002F\u002Fproceedings.mlr.press\u002Fv70\u002Fsundararajan17a\u002Fsundararajan17a.pdf)), GradientShap ([Lundberg and Lee, 2017](https:\u002F\u002Farxiv.org\u002Fabs\u002F1705.07874)) or FusionGrad ([Bykov et al., 2021](https:\u002F\u002Farxiv.org\u002Fabs\u002F2106.10185)), yet it is a common practice for evaluation XAI methods in absence of ground truth data. Therefore, we developed Quantus, an easy-to-use yet comprehensive toolbox for quantitative evaluation of explanations — including 30+ different metrics. \n\n\u003C\u002Fp>\n\u003Cp align=\"center\">\n  \u003Cimg width=\"800\" src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Funderstandable-machine-intelligence-lab_Quantus_readme_ad4d4c0efd26.png\">\n\u003C\u002Fp>\n\nWith Quantus, we can obtain richer insights on how the methods compare e.g., b) by holistic quantification on several evaluation criteria and c) by providing sensitivity analysis of how a single parameter e.g. the pixel replacement strategy of a faithfulness test influences the ranking of the XAI methods.\n \n### Metrics\n\nThis project started with the goal of collecting existing evaluation metrics that have been introduced in the context of XAI research — to help automate the task of _XAI quantification_. Along the way of implementation, it became clear that XAI metrics most often belong to one out of six categories i.e., 1) faithfulness, 2) robustness, 3) localisation 4) complexity 5) randomisation (sensitivity) or 6) axiomatic metrics. The library contains implementations of the following evaluation metrics:\n\n\u003Cdetails>\n  \u003Csummary>\u003Cb>Faithfulness\u003C\u002Fb>\u003C\u002Fsummary>\nquantifies to what extent explanations follow the predictive behaviour of the model (asserting that more important features play a larger role in model outcomes)\n \u003Cbr>\u003Cbr>\n  \u003Cul>\n    \u003Cli>\u003Cb>Faithfulness Correlation \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Fwww.ijcai.org\u002FProceedings\u002F2020\u002F0417.pdf\">(Bhatt et al., 2020)\u003C\u002Fa>: iteratively replaces a random subset of given attributions with a baseline value and then measuring the correlation between the sum of this attribution subset and the difference in function output \n    \u003Cli>\u003Cb>Faithfulness Estimate \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F1806.07538.pdf\">(Alvarez-Melis et al., 2018)\u003C\u002Fa>: computes the correlation between probability drops and attribution scores on various points\n    \u003Cli>\u003Cb>Monotonicity Metric \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F1909.03012\">(Arya et al. 2019)\u003C\u002Fa>: starts from a reference baseline to then incrementally replace each feature in a sorted attribution vector, measuring the effect on model performance\n    \u003Cli>\u003Cb>Monotonicity Metric \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F2007.07584.pdf\"> (Nguyen et al, 2020)\u003C\u002Fa>: measures the spearman rank correlation between the absolute values of the attribution and the uncertainty in the probability estimation\n    \u003Cli>\u003Cb>Pixel Flipping \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Fjournals.plos.org\u002Fplosone\u002Farticle?id=10.1371\u002Fjournal.pone.0130140\">(Bach et al., 2015)\u003C\u002Fa>: captures the impact of perturbing pixels in descending order according to the attributed value on the classification score\n    \u003Cli>\u003Cb>Region Perturbation \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F1509.06321.pdf\">(Samek et al., 2015)\u003C\u002Fa>: is an extension of Pixel-Flipping to flip an area rather than a single pixel\n    \u003Cli>\u003Cb>Selectivity \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F1706.07979.pdf\">(Montavon et al., 2018)\u003C\u002Fa>: measures how quickly an evaluated prediction function starts to drop when removing features with the highest attributed values\n    \u003Cli>\u003Cb>SensitivityN \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F1711.06104.pdf\">(Ancona et al., 2019)\u003C\u002Fa>: computes the correlation between the sum of the attributions and the variation in the target output while varying the fraction of the total number of features, averaged over several test samples\n    \u003Cli>\u003Cb>IROF \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F2003.08747.pdf\">(Rieger at el., 2020)\u003C\u002Fa>: computes the area over the curve per class for sorted mean importances of feature segments (superpixels) as they are iteratively removed (and prediction scores are collected), averaged over several test samples\n    \u003Cli>\u003Cb>Infidelity \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F1901.09392.pdf\">(Chih-Kuan, Yeh, et al., 2019)\u003C\u002Fa>: represents the expected mean square error between 1) a dot product of an attribution and input perturbation and 2) difference in model output after significant perturbation \n    \u003Cli>\u003Cb>ROAD \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F2202.00449.pdf\">(Rong, Leemann, et al., 2022)\u003C\u002Fa>: measures the accuracy of the model on the test set in an iterative process of removing k most important pixels, at each step k most relevant pixels (MoRF order) are replaced with noisy linear imputations\n    \u003Cli>\u003Cb>Sufficiency \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2202.00734\">(Dasgupta et al., 2022)\u003C\u002Fa>: measures the extent to which similar explanations have the same prediction label\n\u003C\u002Ful>\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>Robustness\u003C\u002Fb>\u003C\u002Fsummary>\nmeasures to what extent explanations are stable when subject to slight perturbations of the input, assuming that model output approximately stayed the same\n     \u003Cbr>\u003Cbr>\n\u003Cul>\n    \u003Cli>\u003Cb>Local Lipschitz Estimate \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F1806.08049.pdf\">(Alvarez-Melis et al., 2018)\u003C\u002Fa>: tests the consistency in the explanation between adjacent examples\n    \u003Cli>\u003Cb>Max-Sensitivity \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F1901.09392.pdf\">(Yeh et al., 2019)\u003C\u002Fa>: measures the maximum sensitivity of an explanation using a Monte Carlo sampling-based approximation\n    \u003Cli>\u003Cb>Avg-Sensitivity \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F1901.09392.pdf\">(Yeh et al., 2019)\u003C\u002Fa>: measures the average sensitivity of an explanation using a Monte Carlo sampling-based approximation\n    \u003Cli>\u003Cb>Continuity \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F1706.07979.pdf\">(Montavon et al., 2018)\u003C\u002Fa>: captures the strongest variation in explanation of an input and its perturbed version\n    \u003Cli>\u003Cb>Consistency \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2202.00734\">(Dasgupta et al., 2022)\u003C\u002Fa>: measures the probability that the inputs with the same explanation have the same prediction label\n    \u003Cli>\u003Cb>Relative Input Stability (RIS)\u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F2203.06877.pdf\"> (Agarwal, et. al., 2022)\u003C\u002Fa>: measures the relative distance between explanations e_x and e_x' with respect to the distance between the two inputs x and x'\n    \u003Cli>\u003Cb>Relative Representation Stability (RRS)\u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F2203.06877.pdf\"> (Agarwal, et. al., 2022)\u003C\u002Fa>: measures the relative distance between explanations e_x and e_x' with respect to the distance between internal models representations L_x and L_x' for x and x' respectively\n    \u003Cli>\u003Cb>Relative Output Stability (ROS)\u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F2203.06877.pdf\"> (Agarwal, et. al., 2022)\u003C\u002Fa>: measures the relative distance between explanations e_x and e_x' with respect to the distance between output logits h(x) and h(x') for x and x' respectively\n\u003C\u002Ful>\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>Localisation\u003C\u002Fb>\u003C\u002Fsummary>\ntests if the explainable evidence is centred around a region of interest (RoI) which may be defined around an object by a bounding box, a segmentation mask or, a cell within a grid\n     \u003Cbr>\u003Cbr>\n\u003Cul>\n    \u003Cli>\u003Cb>Pointing Game \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F1608.00507\">(Zhang et al., 2018)\u003C\u002Fa>: checks whether attribution with the highest score is located within the targeted object\n    \u003Cli>\u003Cb>Attribution Localization \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F1910.09840\">(Kohlbrenner et al., 2020)\u003C\u002Fa>: measures the ratio of positive attributions within the targeted object towards the total positive attributions\n    \u003Cli>\u003Cb>Top-K Intersection \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2104.14995\">(Theiner et al., 2021)\u003C\u002Fa>: computes the intersection between a ground truth mask and the binarized explanation at the top k feature locations\n    \u003Cli>\u003Cb>Relevance Rank Accuracy \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2003.07258\">(Arras et al., 2021)\u003C\u002Fa>: measures the ratio of highly attributed pixels within a ground-truth mask towards the size of the ground truth mask\n    \u003Cli>\u003Cb>Relevance Mass Accuracy \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2003.07258\">(Arras et al., 2021)\u003C\u002Fa>: measures the ratio of positively attributed attributions inside the ground-truth mask towards the overall positive attributions\n    \u003Cli>\u003Cb>AUC \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Fdoi.org\u002F10.1016\u002Fj.patrec.2005.10.010\">(Fawcett et al., 2006)\u003C\u002Fa>: compares the ranking between attributions and a given ground-truth mask\n    \u003Cli>\u003Cb>Focus \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2109.15035\">(Arias et al., 2022)\u003C\u002Fa>: quantifies the precision of the explanation by creating mosaics of data instances from different classes\n\u003C\u002Ful>\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>Complexity\u003C\u002Fb>\u003C\u002Fsummary>\ncaptures to what extent explanations are concise i.e., that few features are used to explain a model prediction\n     \u003Cbr>\u003Cbr>\n\u003Cul>\n    \u003Cli>\u003Cb>Sparseness \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F1810.06583\">(Chalasani et al., 2020)\u003C\u002Fa>: uses the Gini Index for measuring, if only highly attributed features are truly predictive of the model output\n    \u003Cli>\u003Cb>Complexity \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2005.00631\">(Bhatt et al., 2020)\u003C\u002Fa>: computes the entropy of the fractional contribution of all features to the total magnitude of the attribution individually\n    \u003Cli>\u003Cb>Effective Complexity \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2007.07584\">(Nguyen at el., 2020)\u003C\u002Fa>: measures how many attributions in absolute values are exceeding a certain threshold\n\u003C\u002Ful>\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>Randomisation (Sensitivity)\u003C\u002Fb>\u003C\u002Fsummary>\ntests to what extent explanations deteriorate as inputs to the evaluation problem e.g., model parameters are increasingly randomised\n     \u003Cbr>\u003Cbr>\n\u003Cul>\n    \u003Cli>\u003Cb>MPRT (Model Parameter Randomisation Test) \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F1810.03292\">(Adebayo et. al., 2018)\u003C\u002Fa>: randomises the parameters of single model layers in a cascading or independent way and measures the distance of the respective explanation to the original explanation\n    \u003Cli>\u003Cb>Smooth MPRT \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Fopenreview.net\u002Fpdf?id=vVpefYmnsG\">(Hedström et. al., 2023)\u003C\u002Fa>: adds a \"denoising\" preprocessing step to the original MPRT, where the explanations are averaged over N noisy samples before the similarity between the original- and fully random model's explanations is measured\n    \u003Cli>\u003Cb>Efficient MPRT \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Fopenreview.net\u002Fpdf?id=vVpefYmnsG\">(Hedström et. al., 2023)\u003C\u002Fa>: reinterprets MPRT by evaluating the rise in explanation complexity (discrete entropy) before and after full model randomisation, asking for increased explanation complexity post-randomisation\n    \u003Cli>\u003Cb>Random Logit Test \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F1912.09818\">(Sixt et al., 2020)\u003C\u002Fa>: computes for the distance between the original explanation and the explanation for a random other class\n\u003C\u002Ful>\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>Axiomatic\u003C\u002Fb>\u003C\u002Fsummary>\n  assesses if explanations fulfil certain axiomatic properties\n     \u003Cbr>\u003Cbr>\n\u003Cul>\n    \u003Cli>\u003Cb>Completeness \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F1703.01365\">(Sundararajan et al., 2017)\u003C\u002Fa>: evaluates whether the sum of attributions is equal to the difference between the function values at the input x and baseline x' (and referred to as Summation to Delta (Shrikumar et al., 2017), Sensitivity-n (slight variation, Ancona et al., 2018) and Conservation (Montavon et al., 2018))\n    \u003Cli>\u003Cb>Non-Sensitivity \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2007.07584\">(Nguyen at el., 2020)\u003C\u002Fa>: measures whether the total attribution is proportional to the explainable evidence at the model output\n    \u003Cli>\u003Cb>Input Invariance \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F1711.00867\">(Kindermans et al., 2017)\u003C\u002Fa>: adds a shift to input, asking that attributions should not change in response (assuming the model does not)\n\u003C\u002Ful>\n\u003C\u002Fdetails>\n\nAdditional metrics will be included in future releases. Please [open an issue](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fissues\u002Fnew\u002Fchoose) if you have a metric you believe should be apart of Quantus.\n\n**Disclaimers.** It is worth noting that the implementations of the metrics in this library have not been verified by the original authors. Thus any metric implementation in this library may differ from the original authors. Further, bear in mind that evaluation metrics for XAI methods are often empirical interpretations (or translations) of qualities that some researcher(s) claimed were important for explanations to fulfil, so it may be a discrepancy between what the author claims to measure by the proposed metric and what is actually measured e.g., using entropy as an operationalisation of explanation complexity. Please read the [user guidelines](https:\u002F\u002Fquantus.readthedocs.io\u002Fen\u002Flatest\u002Fguidelines\u002Fguidelines_and_disclaimers.html) for further guidance on how to best use the library. \n\n## Installation\n\nIf you already have [PyTorch](https:\u002F\u002Fpytorch.org\u002F) or [TensorFlow](https:\u002F\u002Fwww.TensorFlow.org) installed on your machine, \nthe most light-weight version of Quantus can be obtained from [PyPI](https:\u002F\u002Fpypi.org\u002Fproject\u002Fquantus\u002F) as follows (no additional explainability functionality or deep learning framework will be included):\n\n```setup\npip install quantus\n```\nAlternatively, you can simply add the desired deep learning framework (in brackets) to have the package installed together with Quantus.\nTo install Quantus with PyTorch, please run:\n```setup\npip install \"quantus[torch]\"\n```\n\nFor TensorFlow, please run:\n\n```setup\npip install \"quantus[tensorflow]\"\n```\n\n### Package requirements\n\nThe package requirements are as follows:\n```\npython>=3.8.0\ntorch>=1.11.0\ntensorflow>=2.5.0\n```\n\nPlease note that the exact [PyTorch](https:\u002F\u002Fpytorch.org\u002F) and\u002F or [TensorFlow](https:\u002F\u002Fwww.TensorFlow.org) versions\nto be installed depends on your Python version (3.8-3.11) and platform (`darwin`, `linux`, …).\nSee `[project.optional-dependencies]` section in the `pyproject.toml` file.\n\n## Getting started\n\nThe following will give a short introduction to how to get started with Quantus. Note that this example is based on the [PyTorch](https:\u002F\u002Fpytorch.org\u002F) framework, but we also support \n[TensorFlow](https:\u002F\u002Fwww.tensorflow.org), which would differ only in the loading of the model, data and explanations. To get started with Quantus, you need:\n* A model (`model`), inputs (`x_batch`) and labels (`y_batch`)\n* Some explanations you want to evaluate (`a_batch`)\n\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>\u003Cbig>Step 1. Load data and model\u003C\u002Fbig>\u003C\u002Fb>\u003C\u002Fsummary>\n\nLet's first load the data and model. In this example, a pre-trained LeNet available from Quantus \nfor the purpose of this tutorial is loaded, but generally, you might use any Pytorch (or TensorFlow) model instead. To follow this example, one needs to have quantus and torch installed, by e.g., `pip install 'quantus[torch]'`.\n\n```python\nimport quantus\nfrom quantus.helpers.model.models import LeNet\nimport torch\nimport torchvision\nfrom torchvision import transforms\n  \n# Enable GPU.\ndevice = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n\n# Load a pre-trained LeNet classification model (architecture at quantus\u002Fhelpers\u002Fmodels).\nmodel = LeNet()\nif device.type == \"cpu\":\n    model.load_state_dict(torch.load(\"tests\u002Fassets\u002Fmnist\", map_location=torch.device('cpu')))\nelse: \n    model.load_state_dict(torch.load(\"tests\u002Fassets\u002Fmnist\"))\n\n# Load datasets and make loaders.\ntest_set = torchvision.datasets.MNIST(root='.\u002Fsample_data', download=True, transform=transforms.Compose([transforms.ToTensor()]))\ntest_loader = torch.utils.data.DataLoader(test_set, batch_size=24)\n\n# Load a batch of inputs and outputs to use for XAI evaluation.\nx_batch, y_batch = next(iter(test_loader))\nx_batch, y_batch = x_batch.cpu().numpy(), y_batch.cpu().numpy()\n```\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>\u003Cbig>Step 2. Load explanations\u003C\u002Fbig>\u003C\u002Fb>\u003C\u002Fsummary>\n\nWe still need some explanations to evaluate. \nFor this, there are two possibilities in Quantus. You can provide either:\n1. a set of re-computed attributions (`np.ndarray`)\n2. any arbitrary explanation function (`callable`), e.g., the built-in method `quantus.explain` or your own customised function\n\nWe show the different options below.\n\n#### Using pre-computed explanations\n\nQuantus allows you to evaluate explanations that you have pre-computed, \nassuming that they match the data you provide in `x_batch`. Let's say you have explanations \nfor [Saliency](https:\u002F\u002Farxiv.org\u002Fabs\u002F1312.6034) and [Integrated Gradients](https:\u002F\u002Farxiv.org\u002Fabs\u002F1703.01365)\nalready pre-computed.\n\nIn that case, you can simply load these into corresponding variables `a_batch_saliency` \nand `a_batch_intgrad`:\n\n```python\na_batch_saliency = load(\"path\u002Fto\u002Fprecomputed\u002Fsaliency\u002Fexplanations\")\na_batch_intgrad = load(\"path\u002Fto\u002Fprecomputed\u002Fintgrad\u002Fexplanations\")\n```\n\nAnother option is to simply obtain the attributions using one of many XAI frameworks out there, \nsuch as [Captum](https:\u002F\u002Fcaptum.ai\u002F), \n[Zennit](https:\u002F\u002Fgithub.com\u002Fchr5tphr\u002Fzennit), \n[tf.explain](https:\u002F\u002Fgithub.com\u002Fsicara\u002Ftf-explain),\nor [iNNvestigate](https:\u002F\u002Fgithub.com\u002Falbermax\u002Finnvestigate). The following code example shows how to obtain explanations ([Saliency](https:\u002F\u002Farxiv.org\u002Fabs\u002F1312.6034) \nand [Integrated Gradients](https:\u002F\u002Farxiv.org\u002Fabs\u002F1703.01365), to be specific) \nusing [Captum](https:\u002F\u002Fcaptum.ai\u002F):\n\n```python\nimport captum\nfrom captum.attr import Saliency, IntegratedGradients\n\n# Generate Integrated Gradients attributions of the first batch of the test set.\na_batch_saliency = Saliency(model).attribute(inputs=torch.tensor(x_batch, dtype=torch.float32), target=torch.tensor(y_batch, dtype=torch.int64), abs=True).sum(axis=1).cpu().cpu().numpy()\na_batch_intgrad = IntegratedGradients(model).attribute(inputs=torch.tensor(x_batch, dtype=torch.float32), target=torch.tensor(y_batch, dtype=torch.int64)).sum(axis=1).cpu().numpy()\n\n# Quick assert.\nassert [isinstance(obj, np.ndarray) for obj in [x_batch, y_batch, a_batch_saliency, a_batch_intgrad]]\n```\n\n#### Passing an explanation function\n\nIf you don't have a pre-computed set of explanations but rather want to pass an arbitrary explanation function \nthat you wish to evaluate with Quantus, this option exists. \n\nFor this, you can for example rely on the built-in `quantus.explain` function to get started, which includes some popular explanation methods \n(please run `quantus.available_methods()` to see which ones).  Examples of how to use `quantus.explain` \nor your own customised explanation function are included in the next section.\n\n\u003Cimg class=\"center\" width=\"500\" alt=\"drawing\"  src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Funderstandable-machine-intelligence-lab_Quantus_readme_2cb44192d8d8.png\"\u002F>\n\nAs seen in the above image, the qualitative aspects of explanations \nmay look fairly uninterpretable --- since we lack ground truth of what the explanations\nshould be looking like, it is hard to draw conclusions about the explainable evidence. To gather quantitative evidence for the quality of the different explanation methods, we can apply Quantus.\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>\u003Cbig>Step 3. Evaluate with Quantus\u003C\u002Fbig>\u003C\u002Fb>\u003C\u002Fsummary> \n\nQuantus implements XAI evaluation metrics from different categories, \ne.g., Faithfulness, Localisation and Robustness etc which all inherit from the base `quantus.Metric` class. \nTo apply a metric to your setting (e.g., [Max-Sensitivity](https:\u002F\u002Farxiv.org\u002Fabs\u002F1901.09392)) \nit first needs to be instantiated:\n\n```python\nmetric = quantus.MaxSensitivity(nr_samples=10,\n                                lower_bound=0.2,\n                                norm_numerator=quantus.norm_func.fro_norm,\n                                norm_denominator=quantus.norm_func.fro_norm,\n                                similarity_func=quantus.difference,\n                                abs=True,\n                                normalise=True)\n```\n\nand then applied to your model, data, and (pre-computed) explanations:\n\n```python\nscores = metric(\n    model=model,\n    x_batch=x_batch,\n    y_batch=y_batch,\n    a_batch=a_batch_saliency,\n    device=device,\n    explain_func=quantus.explain,\n    explain_func_kwargs={\"method\": \"Saliency\"},\n)\n```\n\n#### Use quantus.explain\n\nSince a re-computation of the explanations is necessary for robustness evaluation, in this example, we also pass an explanation function (`explain_func`) to the metric call. Here, we rely on the built-in `quantus.explain` function to recompute the explanations. The hyperparameters are set with the `explain_func_kwargs` dictionary. Please find more details on how to use  `quantus.explain` at [API documentation](https:\u002F\u002Fquantus.readthedocs.io\u002Fen\u002Flatest\u002Fdocs_api\u002Fquantus.functions.explanation_func.html).\n\n#### Employ customised functions\n\nYou can alternatively use your own customised explanation function\n(assuming it returns an `np.ndarray` in a shape that matches the input `x_batch`). This is done as follows:\n\n```python\ndef your_own_callable(model, models, targets, **kwargs) -> np.ndarray\n  \"\"\"Logic goes here to compute the attributions and return an \n  explanation  in the same shape as x_batch (np.array), \n  (flatten channels if necessary).\"\"\"\n  return explanation(model, x_batch, y_batch)\n\nscores = metric(\n    model=model,\n    x_batch=x_batch,\n    y_batch=y_batch,\n    device=device,\n    explain_func=your_own_callable\n)\n```\n#### Run large-scale evaluation\n\nQuantus also provides high-level functionality to support large-scale evaluations,\ne.g., multiple XAI methods, multifaceted evaluation through several metrics, or a combination thereof. To utilise `quantus.evaluate()`, you simply need to define two things:\n\n1. The **Metrics** you would like to use for evaluation (each `__init__` parameter configuration counts as its own metric):\n    ```python\n    metrics = {\n        \"max-sensitivity-10\": quantus.MaxSensitivity(nr_samples=10),\n        \"max-sensitivity-20\": quantus.MaxSensitivity(nr_samples=20),\n        \"region-perturbation\": quantus.RegionPerturbation(),\n    }\n    ```\n   \n2. The **XAI methods** you would like to evaluate, e.g., a `dict` with pre-computed attributions:\n    ```python\n    xai_methods = {\n        \"Saliency\": a_batch_saliency,\n        \"IntegratedGradients\": a_batch_intgrad\n    }\n    ```\n\nYou can then simply run a large-scale evaluation as follows (this aggregates the result by `np.mean` averaging):\n\n```python\nimport numpy as np\nresults = quantus.evaluate(\n      metrics=metrics,\n      xai_methods=xai_methods,\n      agg_func=np.mean,\n      model=model,\n      x_batch=x_batch,\n      y_batch=y_batch,\n      **{\"softmax\": False,}\n)\n```\n\u003C\u002Fdetails>\n\nPlease see [\nGetting started tutorial](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002Fquantus\u002Fblob\u002Fmain\u002Ftutorials\u002FTutorial_Getting_Started.ipynb) to run code similar to this example. For more information on how to customise metrics and extend Quantus' functionality, please see [Getting started guide](https:\u002F\u002Fquantus.readthedocs.io\u002Fen\u002Flatest\u002Fgetting_started\u002Fgetting_started_example.html).\n\n\n## Tutorials\n\nFurther tutorials are available that showcase the many types of analysis that can be done using Quantus.\nFor this purpose, please see notebooks in the [tutorials](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002Ftutorials\u002F) folder which includes examples such as:\n* [All Metrics ImageNet Example](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002Ftutorials\u002FTutorial_ImageNet_Example_All_Metrics.ipynb): shows how to instantiate the different metrics for ImageNet dataset\n* [Metric Parameterisation Analysis](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002Ftutorials\u002FTutorial_Metric_Parameterisation_Analysis.ipynb): explores how sensitive a metric could be to its hyperparameters\n* [Robustness Analysis Model Training](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002Ftutorials\u002FTutorial_XAI_Sensitivity_Model_Training.ipynb): measures robustness of explanations as model accuracy increases \n* [Full Quantification with Quantus](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002Ftutorials\u002FTutorial_ImageNet_Quantification_with_Quantus.ipynb): example of benchmarking explanation methods\n* [Tabular Data Example](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002Ftutorials\u002FTutorial_Getting_Started_with_Tabular_Data.ipynb): example of how to use Quantus with tabular data\n* [Quantus and TensorFlow Data Example](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002Ftutorials\u002FTutorial_Getting_Started_with_Tensorflow.ipynb): showcases how to use Quantus with TensorFlow\n\n... and more.\n\n## Contributing\n\nWe welcome any sort of contribution to Quantus! For a detailed contribution guide, please refer to [Contributing](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002FCONTRIBUTING.md) documentation first. \n\nIf you have any developer-related questions, please [open an issue](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fissues\u002Fnew\u002Fchoose)\nor write us at [hedstroem.anna@gmail.com](mailto:hedstroem.anna@gmail.com).\n","\u003Cp align=\"center\">\n  \u003Cimg width=\"350\" src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Funderstandable-machine-intelligence-lab_Quantus_readme_cd4ad09d9a34.png\">\n\u003C\u002Fp>\n\u003C!--\u003Ch1 align=\"center\">\u003Cb>Quantus\u003C\u002Fb>\u003C\u002Fh1>-->\n\u003Ch3 align=\"center\">\u003Cb>用于评估神经网络解释的工具包\u003C\u002Fb>\u003C\u002Fh3>\n\u003Cp align=\"center\">\n  PyTorch 和 TensorFlow\n\n[![开始使用！](https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg)](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002Ftutorials\u002FTutorial_ImageNet_Example_All_Metrics.ipynb)\n[![启动教程](https:\u002F\u002Fmybinder.org\u002Fbadge_logo.svg)](https:\u002F\u002Fmybinder.org\u002Fv2\u002Fgh\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002FHEAD?labpath=tutorials)\n![Python 版本](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fpython-3.8%20%7C%203.9%20%7C%203.10%20%7C%203.11-blue.svg)\n[![PyPI 版本](https:\u002F\u002Fbadge.fury.io\u002Fpy\u002Fquantus.svg)](https:\u002F\u002Fbadge.fury.io\u002Fpy\u002Fquantus)\n[![代码风格：black](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fcode%20style-black-000000.svg)](https:\u002F\u002Fgithub.com\u002Fpsf\u002Fblack)\n[![文档状态](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Funderstandable-machine-intelligence-lab_Quantus_readme_13d664e1afd7.png)](https:\u002F\u002Fquantus.readthedocs.io\u002Fen\u002Flatest\u002F?badge=latest)\n[![codecov.io](https:\u002F\u002Fcodecov.io\u002Fgithub\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcoverage.svg?branch=master)](https:\u002F\u002Fcodecov.io\u002Fgithub\u002Funderstandable-machine-intelligence-lab\u002FQuantus?branch=master)\n[![下载量](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Funderstandable-machine-intelligence-lab_Quantus_readme_029878a584ab.png)](https:\u002F\u002Fpepy.tech\u002Fproject\u002Fquantus)\n\u003C!--[![Python 包](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Factions\u002Fworkflows\u002Fpython-package.yml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Factions\u002Fworkflows\u002Fpython-package.yml)\n[![代码覆盖率](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Factions\u002Fworkflows\u002Fcodecov.yml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Factions\u002Fworkflows\u002Fcodecov.yml)\n-->\n\n_Quantus 目前仍在积极开发中，请务必注意 Quantus 的版本，以确保工作的可重复性。_\n\n[📑 论文快捷链接！](https:\u002F\u002Fjmlr.org\u002Fpapers\u002Fvolume24\u002F22-0142\u002F22-0142.pdf)\n\n如果您想为 Quantus 做出贡献、改进或扩展，请加入我们的 [Discord](https:\u002F\u002Fdiscord.gg\u002FHB77krUE)！\n\n## 新闻与亮点！ :rocket:\n\n- 🐼 对于**训练数据归因**评估，可以查看 [quanda](https:\u002F\u002Fgithub.com\u002Fdilyabareeva\u002Fquanda)!\n- 新增[批量实现](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F351)，使现有忠实度指标的速度提升 12 倍 (!)\n- 新增指标：由[Hedström 等人（2023）](https:\u002F\u002Fopenreview.net\u002Fpdf?id=vVpefYmnsG)提出的[EfficientMPRT](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002Fquantus\u002Fmetrics\u002Frandomisation\u002Fefficient_mprt.py)和[SmoothMPRT](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002Fquantus\u002Fmetrics\u002Frandomisation\u002Fsmooth_mprt.py)\n- 被《机器学习研究期刊》（MLOSS）接收，阅读[论文](https:\u002F\u002Fjmlr.org\u002Fpapers\u002Fv24\u002F22-0142.html)\n- 提供超过**35 种六大类别的指标**，用于 XAI 评估\n- 支持多种数据类型（图像、时间序列、表格数据、NLP 即将支持！）和模型（PyTorch、TensorFlow）\n- 扩展了对解释方法的内置支持（[captum](https:\u002F\u002Fcaptum.ai\u002F)、[tf-explain](https:\u002F\u002Ftf-explain.readthedocs.io\u002Fen\u002Flatest\u002F) 和 [zennit](https:\u002F\u002Fgithub.com\u002Fchr5tphr\u002Fzennit))\n\u003C!--- 发布了新版本[这里](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Freleases)，并停止支持 Python 3.7 (https:\u002F\u002Fdevguide.python.org\u002Fversions\u002F) -->\n\n## 引用\n\n如果您在研究中发现本工具包或其配套论文\n[**Quantus：用于负责任地评估神经网络解释及其他领域的可解释 AI 工具包**](https:\u002F\u002Fjmlr.org\u002Fpapers\u002Fv24\u002F22-0142.html)\n有趣或有用，请使用以下 Bibtex 格式引用我们：\n\n```bibtex\n@article{hedstrom2023quantus,\n  author  = {Anna Hedstr{\\\"{o}}m and Leander Weber and Daniel Krakowczyk and Dilyara Bareeva and Franz Motzkus and Wojciech Samek and Sebastian Lapuschkin and Marina Marina M.{-}C. H{\\\"{o}}hne},\n  title   = {Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond},\n  journal = {Journal of Machine Learning Research},\n  year    = {2023},\n  volume  = {24},\n  number  = {34},\n  pages   = {1--11},\n  url     = {http:\u002F\u002Fjmlr.org\u002Fpapers\u002Fv24\u002F22-0142.html}\n}\n```\n\n在应用 Quantus 的各项指标时，请务必同时正确引用原始作者的工作（如下所示）。\n\n## 目录\n\n* [库概览](#library-overview)\n* [安装](#installation)\n* [快速入门](#getting-started)\n* [教程](#tutorials)\n* [贡献](#contributing)\n\u003C!--* [引用](#citation)-->\n\n## 库概览\n\n仅通过简单直观地比较可解释人工智能（XAI）方法，往往不足以判断哪种解释方法效果最好，如图 a) 中所示的四种基于梯度的方法——显著图([Mørch 等，1995](https:\u002F\u002Fieeexplore.ieee.org\u002Fdocument\u002F488997); [Baehrens 等，2010](https:\u002F\u002Fwww.jmlr.org\u002Fpapers\u002Fvolume11\u002Fbaehrens10a\u002Fbaehrens10a.pdf))、积分梯度([Sundararajan 等，2017](http:\u002F\u002Fproceedings.mlr.press\u002Fv70\u002Fsundararajan17a\u002Fsundararajan17a.pdf))、GradientShap([Lundberg 和 Lee，2017](https:\u002F\u002Farxiv.org\u002Fabs\u002F1705.07874)) 或 FusionGrad([Bykov 等，2021](https:\u002F\u002Farxiv.org\u002Fabs\u002F2106.10185))——尽管如此，在缺乏真实标签数据的情况下，这种做法仍是评估 XAI 方法的常见方式。因此，我们开发了 Quantus，一个易于使用但功能全面的工具箱，用于对解释进行定量评估——包含 30 多种不同的指标。\n\n\u003C\u002Fp>\n\u003Cp align=\"center\">\n  \u003Cimg width=\"800\" src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Funderstandable-machine-intelligence-lab_Quantus_readme_ad4d4c0efd26.png\">\n\u003C\u002Fp>\n\n借助 Quantus，我们可以获得更丰富的见解，例如 b) 通过在多个评估标准上进行全面量化来比较这些方法，以及 c) 通过对单个参数（如忠实度测试中的像素替换策略）进行敏感性分析，从而了解其如何影响 XAI 方法的排名。\n \n### 指标\n\n该项目最初的目标是收集已在 XAI 研究背景下提出的现有评估指标，以帮助自动化“XAI 定量评估”的任务。在实施过程中，我们逐渐认识到，XAI 指标通常属于六类之一，即：1) 忠实度、2) 鲁棒性、3) 局部化、4) 复杂性、5) 随机化（敏感性）或 6) 公理化指标。该库包含了以下评估指标的实现：\n\n\u003Cdetails>\n  \u003Csummary>\u003Cb>忠实性\u003C\u002Fb>\u003C\u002Fsummary>\n用于量化解释在多大程度上遵循模型的预测行为（即强调更重要的特征在模型输出中起更大作用）。\n \u003Cbr>\u003Cbr>\n  \u003Cul>\n    \u003Cli>\u003Cb>忠实性相关性 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Fwww.ijcai.org\u002FProceedings\u002F2020\u002F0417.pdf\">(Bhatt et al., 2020)\u003C\u002Fa>：迭代地将给定归因的随机子集替换为基准值，然后测量该归因子集之和与函数输出差异之间的相关性。\n    \u003Cli>\u003Cb>忠实性估计 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F1806.07538.pdf\">(Alvarez-Melis et al., 2018)\u003C\u002Fa>：计算不同点上概率下降与归因分数之间的相关性。\n    \u003Cli>\u003Cb>单调性度量 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F1909.03012\">(Arya et al. 2019)\u003C\u002Fa>：从参考基准开始，逐步替换排序后的归因向量中的每个特征，以衡量其对模型性能的影响。\n    \u003Cli>\u003Cb>单调性度量 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F2007.07584.pdf\"> (Nguyen et al, 2020)\u003C\u002Fa>：测量归因绝对值与概率估计不确定性之间的斯皮尔曼等级相关系数。\n    \u003Cli>\u003Cb>像素翻转 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Fjournals.plos.org\u002Fplosone\u002Farticle?id=10.1371\u002Fjournal.pone.0130140\">(Bach et al., 2015)\u003C\u002Fa>：捕捉按照归因值降序扰动像素对分类得分的影响。\n    \u003Cli>\u003Cb>区域扰动 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F1509.06321.pdf\">(Samek et al., 2015)\u003C\u002Fa>：是像素翻转的扩展，将单个像素替换为一个区域。\n    \u003Cli>\u003Cb>选择性 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F1706.07979.pdf\">(Montavon et al., 2018)\u003C\u002Fa>：衡量当移除具有最高归因值的特征时，被评估的预测函数开始下降的速度。\n    \u003Cli>\u003Cb>SensitivityN \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F1711.06104.pdf\">(Ancona et al., 2019)\u003C\u002Fa>：在改变总特征数量的比例时，计算归因之和与目标输出变化之间的相关性，并对多个测试样本取平均。\n    \u003Cli>\u003Cb>IROF \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F2003.08747.pdf\">(Rieger at el., 2020)\u003C\u002Fa>：对于多个测试样本，计算按顺序移除特征片段（超像素）时，其平均重要性排序曲线下的面积，并取平均。\n    \u003Cli>\u003Cb>不忠实性 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F1901.09392.pdf\">(Chih-Kuan, Yeh, et al., 2019)\u003C\u002Fa>：表示归因与输入扰动的点积与显著扰动后模型输出差异之间的期望均方误差。\n    \u003Cli>\u003Cb>ROAD \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F2202.00449.pdf\">(Rong, Leemann, et al., 2022)\u003C\u002Fa>：通过迭代移除k个最重要的像素来衡量模型在测试集上的准确率；每一步都用噪声线性插值替换最相关的k个像素（MoRF顺序）。\n    \u003Cli>\u003Cb>充分性 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2202.00734\">(Dasgupta et al., 2022)\u003C\u002Fa>：衡量相似解释是否具有相同预测标签的程度。\n\u003C\u002Ful>\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>鲁棒性\u003C\u002Fb>\u003C\u002Fsummary>\n用于衡量在输入发生轻微扰动时，解释在多大程度上保持稳定，假设模型输出大致不变。\n     \u003Cbr>\u003Cbr>\n\u003Cul>\n    \u003Cli>\u003Cb>局部利普希茨估计 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F1806.08049.pdf\">(Alvarez-Melis et al., 2018)\u003C\u002Fa>：检验相邻示例之间解释的一致性。\n    \u003Cli>\u003Cb>最大敏感度 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F1901.09392.pdf\">(Yeh et al., 2019)\u003C\u002Fa>：使用蒙特卡洛采样近似法测量解释的最大敏感度。\n    \u003Cli>\u003Cb>平均敏感度 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F1901.09392.pdf\">(Yeh et al., 2019)\u003C\u002Fa>：使用蒙特卡洛采样近似法测量解释的平均敏感度。\n    \u003Cli>\u003Cb>连续性 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F1706.07979.pdf\">(Montavon et al., 2018)\u003C\u002Fa>：捕捉输入及其扰动版本之间解释的最大变化。\n    \u003Cli>\u003Cb>一致性 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2202.00734\">(Dasgupta et al., 2022)\u003C\u002Fa>：衡量具有相同解释的输入拥有相同预测标签的概率。\n    \u003Cli>\u003Cb>相对输入稳定性（RIS）\u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F2203.06877.pdf\"> (Agarwal, et. al., 2022)\u003C\u002Fa>：衡量解释e_x与e_x'之间的相对距离，相对于两个输入x与x'之间的距离。\n    \u003Cli>\u003Cb>相对表征稳定性（RRS）\u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F2203.06877.pdf\"> (Agarwal, et. al., 2022)\u003C\u002Fa>：衡量解释e_x与e_x'之间的相对距离，相对于分别对应于x和x'的内部模型表征L_x与L_x'之间的距离。\n    \u003Cli>\u003Cb>相对输出稳定性（ROS）\u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F2203.06877.pdf\"> (Agarwal, et. al., 2022)\u003C\u002Fa>：衡量解释e_x与e_x'之间的相对距离，相对于分别对应于x和x'的输出logit h(x)与h(x')之间的距离。\n\u003C\u002Ful>\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>定位性\u003C\u002Fb>\u003C\u002Fsummary>\n测试可解释性证据是否集中在一个感兴趣区域（RoI）周围，该区域可以由边界框、分割掩码或网格中的某个单元格来定义。\n     \u003Cbr>\u003Cbr>\n\u003Cul>\n    \u003Cli>\u003Cb>指向游戏 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F1608.00507\">(Zhang et al., 2018)\u003C\u002Fa>: 检查得分最高的归因是否位于目标对象内\n    \u003Cli>\u003Cb>归因定位 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F1910.09840\">(Kohlbrenner et al., 2020)\u003C\u002Fa>: 衡量目标对象内正向归因占总正向归因的比例\n    \u003Cli>\u003Cb>Top-K 交集 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2104.14995\">(Theiner et al., 2021)\u003C\u002Fa>: 计算真实标签掩码与在前 k 个特征位置上二值化的解释之间的交集\n    \u003Cli>\u003Cb>相关性排名准确率 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2003.07258\">(Arras et al., 2021)\u003C\u002Fa>: 衡量真实标签掩码中高归因像素所占比例与真实标签掩码大小之比\n    \u003Cli>\u003Cb>相关性质量准确率 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2003.07258\">(Arras et al., 2021)\u003C\u002Fa>: 衡量真实标签掩码内正向归因占总正向归因的比例\n    \u003Cli>\u003Cb>AUC \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Fdoi.org\u002F10.1016\u002Fj.patrec.2005.10.010\">(Fawcett et al., 2006)\u003C\u002Fa>: 比较归因与给定真实标签掩码之间的排序\n    \u003Cli>\u003Cb>聚焦 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2109.15035\">(Arias et al., 2022)\u003C\u002Fa>: 通过创建来自不同类别的数据实例的马赛克来量化解释的精确度\n\u003C\u002Ful>\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>复杂性\u003C\u002Fb>\u003C\u002Fsummary>\n衡量解释在多大程度上简洁，即使用较少的特征来解释模型预测。\n     \u003Cbr>\u003Cbr>\n\u003Cul>\n    \u003Cli>\u003Cb>稀疏性 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F1810.06583\">(Chalasani et al., 2020)\u003C\u002Fa>: 使用基尼指数来衡量是否只有高归因的特征真正能够预测模型输出\n    \u003Cli>\u003Cb>复杂性 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2005.00631\">(Bhatt et al., 2020)\u003C\u002Fa>: 计算所有特征对归因总幅度的分数贡献的熵\n    \u003Cli>\u003Cb>有效复杂性 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2007.07584\">(Nguyen at el., 2020)\u003C\u002Fa>: 衡量绝对值上超过某一阈值的归因数量\n\u003C\u002Ful>\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>随机化（敏感性）\u003C\u002Fb>\u003C\u002Fsummary>\n测试当评估问题的输入（例如模型参数）逐渐被随机化时，解释会受到多大程度的影响。\n     \u003Cbr>\u003Cbr>\n\u003Cul>\n    \u003Cli>\u003Cb>MPRT（模型参数随机化测试） \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F1810.03292\">(Adebayo et. al., 2018)\u003C\u002Fa>: 以级联或独立的方式随机化单个模型层的参数，并测量相应解释与原始解释之间的距离\n    \u003Cli>\u003Cb>平滑 MPRT \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Fopenreview.net\u002Fpdf?id=vVpefYmnsG\">(Hedström et. al., 2023)\u003C\u002Fa>: 在原始 MPRT 中增加一个“去噪”预处理步骤，在测量原始模型和完全随机模型的解释相似性之前，先对 N 个噪声样本的解释取平均值\n    \u003Cli>\u003Cb>高效 MPRT \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Fopenreview.net\u002Fpdf?id=vVpefYmnsG\">(Hedström et. al., 2023)\u003C\u002Fa>: 重新解释 MPRT，通过评估模型完全随机化前后解释复杂度（离散熵）的变化，要求随机化后解释复杂度有所提高\n    \u003Cli>\u003Cb>随机逻辑回归测试 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F1912.09818\">(Sixt et al., 2020)\u003C\u002Fa>: 计算原始解释与针对随机其他类别的解释之间的距离\n\u003C\u002Ful>\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>公理性\u003C\u002Fb>\u003C\u002Fsummary>\n评估解释是否满足某些公理性质。\n     \u003Cbr>\u003Cbr>\n\u003Cul>\n    \u003Cli>\u003Cb>完备性 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F1703.01365\">(Sundararajan et al., 2017)\u003C\u002Fa>: 评估归因之和是否等于输入 x 和基准 x' 处函数值之差（也称为求和到差值法（Shrikumar et al., 2017）、灵敏度-n（轻微变化，Ancona et al., 2018）以及守恒性（Montavon et al., 2018））\n    \u003Cli>\u003Cb>非灵敏性 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2007.07584\">(Nguyen at el., 2020)\u003C\u002Fa>: 衡量总归因是否与模型输出处的可解释证据成比例\n    \u003Cli>\u003Cb>输入不变性 \u003C\u002Fb>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F1711.00867\">(Kindermans et al., 2017)\u003C\u002Fa>: 对输入进行偏移，要求归因不应随之改变（假设模型本身不改变）\n\u003C\u002Ful>\n\u003C\u002Fdetails>\n\n未来版本将包含更多指标。如果您认为有值得加入 Quantus 的指标，请[提交一个问题](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fissues\u002Fnew\u002Fchoose)。\n\n**免责声明。** 值得注意的是，本库中各项指标的实现尚未经过原作者验证。因此，本库中的任何指标实现都可能与原作者有所不同。此外，请记住，XAI 方法的评估指标通常是某些研究者认为解释应具备的重要属性的实证诠释（或转化），因此，作者声称通过所提指标衡量的内容与实际衡量的内容之间可能存在差异，例如使用熵作为解释复杂性的操作化定义。请阅读[用户指南](https:\u002F\u002Fquantus.readthedocs.io\u002Fen\u002Flatest\u002Fguidelines\u002Fguidelines_and_disclaimers.html)，以获取有关如何最佳使用该库的进一步指导。\n\n\n\n## 安装\n\n如果您机器上已经安装了 [PyTorch](https:\u002F\u002Fpytorch.org\u002F) 或 [TensorFlow](https:\u002F\u002Fwww.TensorFlow.org)，则可以从 [PyPI](https:\u002F\u002Fpypi.org\u002Fproject\u002Fquantus\u002F) 获取 Quantus 最轻量的版本，如下所示（不会包含额外的可解释性功能或深度学习框架）：\n\n```setup\npip install quantus\n```\n或者，您也可以简单地添加所需的深度学习框架（用方括号括起来），以便将该包与 Quantus 一起安装。\n要安装带有 PyTorch 的 Quantus，请运行：\n```setup\npip install \"quantus[torch]\"\n```\n\n对于 TensorFlow，请运行：\n\n```setup\npip install \"quantus[tensorflow]\"\n```\n\n### 软件包要求\n\n软件包要求如下：\n```\npython>=3.8.0\ntorch>=1.11.0\ntensorflow>=2.5.0\n```\n\n请注意，要安装的确切 [PyTorch](https:\u002F\u002Fpytorch.org\u002F) 和\u002F或 [TensorFlow](https:\u002F\u002Fwww.TensorFlow.org) 版本取决于您的 Python 版本（3.8-3.11）和平台（`darwin`、`linux` 等）。详情请参阅 `pyproject.toml` 文件中的 `[project.optional-dependencies]` 部分。\n\n## 开始使用\n\n以下将简要介绍如何开始使用 Quantus。请注意，本示例基于 [PyTorch](https:\u002F\u002Fpytorch.org\u002F) 框架，但我们同时也支持 [TensorFlow](https:\u002F\u002Fwww.tensorflow.org\u002F)，两者之间的差异仅在于模型、数据和解释的加载方式。要开始使用 Quantus，您需要：\n* 一个模型 (`model`)、输入数据 (`x_batch`) 和标签 (`y_batch`)\n* 一些您想要评估的解释 (`a_batch`)\n\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>\u003Cbig>步骤 1. 加载数据和模型\u003C\u002Fbig>\u003C\u002Fb>\u003C\u002Fsummary>\n\n首先，我们来加载数据和模型。在本示例中，我们从 Quantus 中加载了一个用于本教程的预训练 LeNet 模型；不过通常情况下，您也可以使用任何 PyTorch（或 TensorFlow）模型。要按照此示例操作，您需要安装 quantus 和 torch，例如通过运行 `pip install 'quantus[torch]'`。\n\n```python\nimport quantus\nfrom quantus.helpers.model.models import LeNet\nimport torch\nimport torchvision\nfrom torchvision import transforms\n  \n# 启用 GPU。\ndevice = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n\n# 加载一个预训练的 LeNet 分类模型（架构位于 quantus\u002Fhelpers\u002Fmodels）。\nmodel = LeNet()\nif device.type == \"cpu\":\n    model.load_state_dict(torch.load(\"tests\u002Fassets\u002Fmnist\", map_location=torch.device('cpu')))\nelse: \n    model.load_state_dict(torch.load(\"tests\u002Fassets\u002Fmnist\"))\n\n# 加载数据集并创建数据加载器。\ntest_set = torchvision.datasets.MNIST(root='.\u002Fsample_data', download=True, transform=transforms.Compose([transforms.ToTensor()]))\ntest_loader = torch.utils.data.DataLoader(test_set, batch_size=24)\n\n# 加载一批用于 XAI 评估的输入和输出数据。\nx_batch, y_batch = next(iter(test_loader))\nx_batch, y_batch = x_batch.cpu().numpy(), y_batch.cpu().numpy()\n```\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>\u003Cbig>步骤 2. 加载解释\u003C\u002Fbig>\u003C\u002Fb>\u003C\u002Fsummary>\n\n我们还需要一些解释来进行评估。在 Quantus 中，有两种方法可以实现这一点。您可以提供：\n1. 一组重新计算的归因值 (`np.ndarray`)\n2. 任意解释函数 (`callable`)，例如内置的 `quantus.explain` 方法，或者您自定义的函数\n\n下面我们将展示这两种不同的选项。\n\n#### 使用预先计算的解释\n\nQuantus 允许您评估已经预先计算好的解释，前提是这些解释与您在 `x_batch` 中提供的数据相匹配。假设您已经预先计算好了 [显著性图](https:\u002F\u002Farxiv.org\u002Fabs\u002F1312.6034) 和 [积分梯度](https:\u002F\u002Farxiv.org\u002Fabs\u002F1703.01365) 的解释。\n\n在这种情况下，您可以直接将这些解释加载到对应的变量 `a_batch_saliency` 和 `a_batch_intgrad` 中：\n\n```python\na_batch_saliency = load(\"path\u002Fto\u002Fprecomputed\u002Fsaliency\u002Fexplanations\")\na_batch_intgrad = load(\"path\u002Fto\u002Fprecomputed\u002Fintgrad\u002Fexplanations\")\n```\n\n另一种方法是直接使用现有的多种 XAI 框架来获取归因值，例如 [Captum](https:\u002F\u002Fcaptum.ai\u002F)、\n[Zennit](https:\u002F\u002Fgithub.com\u002Fchr5tphr\u002Fzennit)、\n[tf.explain](https:\u002F\u002Fgithub.com\u002Fsicara\u002Ftf-explain)，\n或 [iNNvestigate](https:\u002F\u002Fgithub.com\u002Falbermax\u002Finnvestigate)。以下代码示例展示了如何使用 [Captum](https:\u002F\u002Fcaptum.ai\u002F) 获取解释（具体来说是 [显著性图](https:\u002F\u002Farxiv.org\u002Fabs\u002F1312.6034) 和 [积分梯度](https:\u002F\u002Farxiv.org\u002Fabs\u002F1703.01365))：\n\n```python\nimport captum\nfrom captum.attr import Saliency, IntegratedGradients\n\n# 生成测试集中第一批样本的积分梯度归因值。\na_batch_saliency = Saliency(model).attribute(inputs=torch.tensor(x_batch, dtype=torch.float32), target=torch.tensor(y_batch, dtype=torch.int64), abs=True).sum(axis=1).cpu().cpu().numpy()\na_batch_intgrad = IntegratedGradients(model).attribute(inputs=torch.tensor(x_batch, dtype=torch.float32), target=torch.tensor(y_batch, dtype=torch.int64)).sum(axis=1).cpu().numpy()\n```\n\n# 快速断言。\nassert [isinstance(obj, np.ndarray) for obj in [x_batch, y_batch, a_batch_saliency, a_batch_intgrad]]\n```\n\n#### 传递解释函数\n\n如果你没有预先计算好的解释集合，而是希望传递一个任意的解释函数来使用 Quantus 进行评估，那么这也是可行的。\n\n为此，你可以例如依赖内置的 `quantus.explain` 函数来开始，该函数包含了一些流行的解释方法（请运行 `quantus.available_methods()` 查看具体有哪些）。如何使用 `quantus.explain` 或你自定义的解释函数的示例将在下一节中给出。\n\n\u003Cimg class=\"center\" width=\"500\" alt=\"drawing\"  src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Funderstandable-machine-intelligence-lab_Quantus_readme_2cb44192d8d8.png\"\u002F>\n\n如上图所示，从定性角度来看，这些解释可能显得相当难以理解——因为我们缺乏关于解释应呈现何种样式的地面真相，因此很难对可解释性证据得出结论。为了收集不同解释方法质量的定量证据，我们可以应用 Quantus。\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>\u003Cbig>步骤3. 使用 Quantus 进行评估\u003C\u002Fbig>\u003C\u002Fb>\u003C\u002Fsummary> \n\nQuantus 实现了来自不同类别的 XAI 评估指标，例如忠实性、局部性和鲁棒性等，它们都继承自基础的 `quantus.Metric` 类。要将某个指标应用于你的场景中（比如 [最大敏感性](https:\u002F\u002Farxiv.org\u002Fabs\u002F1901.09392)），首先需要实例化它：\n\n```python\nmetric = quantus.MaxSensitivity(nr_samples=10,\n                                lower_bound=0.2,\n                                norm_numerator=quantus.norm_func.fro_norm,\n                                norm_denominator=quantus.norm_func.fro_norm,\n                                similarity_func=quantus.difference,\n                                abs=True,\n                                normalise=True)\n```\n\n然后将其应用于你的模型、数据以及（预先计算的）解释：\n\n```python\nscores = metric(\n    model=model,\n    x_batch=x_batch,\n    y_batch=y_batch,\n    a_batch=a_batch_saliency,\n    device=device,\n    explain_func=quantus.explain,\n    explain_func_kwargs={\"method\": \"Saliency\"},\n)\n```\n\n#### 使用 quantus.explain\n\n由于在进行鲁棒性评估时需要重新计算解释，因此在这个示例中，我们还将一个解释函数 (`explain_func`) 传递给指标调用。这里我们依赖内置的 `quantus.explain` 函数来重新计算解释。超参数通过 `explain_func_kwargs` 字典进行设置。有关如何使用 `quantus.explain` 的更多详细信息，请参阅 [API 文档](https:\u002F\u002Fquantus.readthedocs.io\u002Fen\u002Flatest\u002Fdocs_api\u002Fquantus.functions.explanation_func.html)。\n\n#### 使用自定义函数\n\n你也可以选择使用自己的自定义解释函数（假设它返回一个与输入 `x_batch` 形状匹配的 `np.ndarray`）。实现方式如下：\n\n```python\ndef your_own_callable(model, models, targets, **kwargs) -> np.ndarray\n  \"\"\"此处编写逻辑以计算归因，并返回与 x_batch 形状相同的解释（np.array），必要时可展平通道。\"\"\"\n  return explanation(model, x_batch, y_batch)\n\nscores = metric(\n    model=model,\n    x_batch=x_batch,\n    y_batch=y_batch,\n    device=device,\n    explain_func=your_own_callable\n)\n```\n#### 运行大规模评估\n\nQuantus 还提供了高级功能来支持大规模评估，例如同时评估多种 XAI 方法、通过多个指标进行多方面评估，或者两者的结合。要使用 `quantus.evaluate()`，你只需定义两件事：\n\n1. 你希望用于评估的 **指标**（每个 `__init__` 参数配置都被视为一个独立的指标）：\n    ```python\n    metrics = {\n        \"max-sensitivity-10\": quantus.MaxSensitivity(nr_samples=10),\n        \"max-sensitivity-20\": quantus.MaxSensitivity(nr_samples=20),\n        \"region-perturbation\": quantus.RegionPerturbation(),\n    }\n    ```\n   \n2. 你希望评估的 **XAI 方法**，例如一个包含预先计算好的归因的字典：\n    ```python\n    xai_methods = {\n        \"Saliency\": a_batch_saliency,\n        \"IntegratedGradients\": a_batch_intgrad\n    }\n    ```\n\n之后，你可以简单地运行大规模评估（结果会通过 `np.mean` 取平均值）：\n\n```python\nimport numpy as np\nresults = quantus.evaluate(\n      metrics=metrics,\n      xai_methods=xai_methods,\n      agg_func=np.mean,\n      model=model,\n      x_batch=x_batch,\n      y_batch=y_batch,\n      **{\"softmax\": False,}\n)\n```\n\u003C\u002Fdetails>\n\n请参阅 [\n入门教程](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002Fquantus\u002Fblob\u002Fmain\u002Ftutorials\u002FTutorial_Getting_Started.ipynb) 来运行与此示例类似的代码。有关如何自定义指标并扩展 Quantus 功能的更多信息，请参阅 [入门指南](https:\u002F\u002Fquantus.readthedocs.io\u002Fen\u002Flatest\u002Fgetting_started\u002Fgetting_started_example.html)。\n\n\n## 教程\n\n还有其他教程展示了使用 Quantus 可以进行的各种分析类型。为此，请查看位于 [tutorials](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002Ftutorials\u002F) 文件夹中的笔记本，其中包括以下示例：\n* [所有指标 ImageNet 示例](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002Ftutorials\u002FTutorial_ImageNet_Example_All_Metrics.ipynb)：展示如何为 ImageNet 数据集实例化不同的指标\n* [指标参数化分析](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002Ftutorials\u002FTutorial_Metric_Parameterisation_Analysis.ipynb)：探讨指标对其超参数的敏感程度\n* [鲁棒性分析模型训练](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002Ftutorials\u002FTutorial_XAI_Sensitivity_Model_Training.ipynb)：衡量随着模型准确率提高，解释的鲁棒性变化情况\n* [使用 Quantus 完全量化](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002Ftutorials\u002FTutorial_ImageNet_Quantification_with_Quantus.ipynb)：解释方法基准测试示例\n* [表格数据示例](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002Ftutorials\u002FTutorial_Getting_Started_with_Tabular_Data.ipynb)：说明如何将 Quantus 应用于表格数据\n* [Quantus 与 TensorFlow 数据示例](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002Ftutorials\u002FTutorial_Getting_Started_with_Tensorflow.ipynb)：展示如何将 Quantus 与 TensorFlow 结合使用\n\n……以及更多内容。\n\n## 贡献\n\n我们欢迎对 Quantus 的任何形式的贡献！如需详细的贡献指南，请先参阅 [Contributing](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fblob\u002Fmain\u002FCONTRIBUTING.md) 文档。\n\n如果您有任何与开发相关的问题，请[提交一个问题](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fissues\u002Fnew\u002Fchoose)，或发送邮件至 [hedstroem.anna@gmail.com](mailto:hedstroem.anna@gmail.com)。","# Quantus 快速上手指南\n\nQuantus 是一个用于定量评估神经网络解释（XAI）方法的工具包，支持 PyTorch 和 TensorFlow 框架。它提供了超过 35 种评估指标，涵盖忠实度、鲁棒性、定位、复杂度等六大类别，帮助开发者科学地衡量解释方法的有效性。\n\n## 环境准备\n\n在开始之前，请确保您的开发环境满足以下要求：\n\n*   **操作系统**：Linux, macOS 或 Windows\n*   **Python 版本**：3.8, 3.9, 3.10 或 3.11\n*   **深度学习框架**（任选其一）：\n    *   PyTorch\n    *   TensorFlow\n*   **前置依赖**：建议先安装好对应的深度学习框架及其依赖项。\n\n## 安装步骤\n\n您可以使用 `pip` 直接安装 Quantus。为了获得更快的下载速度，推荐使用国内镜像源（如清华大学开源软件镜像站）。\n\n**使用默认源安装：**\n```bash\npip install quantus\n```\n\n**使用清华镜像源加速安装（推荐）：**\n```bash\npip install quantus -i https:\u002F\u002Fpypi.tuna.tsinghua.edu.cn\u002Fsimple\n```\n\n安装完成后，您可以通过以下命令验证安装是否成功：\n```bash\npython -c \"import quantus; print(quantus.__version__)\"\n```\n\n## 基本使用\n\nQuantus 的核心用法非常直观：实例化一个评估指标对象，然后传入模型、输入数据、目标标签以及生成的解释图（Attribution Map）即可得到评分。\n\n以下是一个基于 **PyTorch** 和 **忠实度相关性（Faithfulness Correlation）** 指标的最简使用示例：\n\n```python\nimport torch\nimport quantus\n\n# 1. 准备数据 (假设已有模型、输入数据、标签和解释图)\n# model: 训练好的 PyTorch 模型\n# x_batch: 输入图像张量 [batch_size, channels, height, width]\n# y_batch: 真实标签张量 [batch_size]\n# attributions: 解释方法生成的注意力图\u002F归因图 [batch_size, channels, height, width]\n\n# 2. 初始化评估指标\n# 这里以 Faithfulness Correlation 为例，设置替换策略为黑色像素\nmetric = quantus.FaithfulnessCorrelation(\n    pixel_replace_value=0.0, \n    abs=False\n)\n\n# 3. 执行评估\nscore = metric(\n    model=model,\n    x_batch=x_batch,\n    y_batch=y_batch,\n    attributions=attributions,\n    channel_first=True,  # 如果数据格式是 [B, C, H, W] 则设为 True\n    disable_warnings=True\n)\n\n# 4. 输出结果\nprint(f\"Faithfulness Correlation Score: {score}\")\n```\n\n**关键点说明：**\n*   **指标选择**：Quantus 提供了多种指标（如 `Robustness`, `Localization`, `Complexity` 等），只需将 `quantus.FaithfulnessCorrelation` 替换为其他类名即可切换评估维度。\n*   **数据格式**：确保 `channel_first` 参数与您的数据张量格式（NCHW 或 NHWC）匹配。\n*   **框架兼容**：如果您使用 TensorFlow，代码结构类似，只需传入 TF 模型和对应的张量格式。\n\n更多详细用法和针对不同类型数据（时间序列、表格数据等）的示例，请参考官方教程或文档。","某医疗 AI 团队正在开发基于深度学习的肺结节 CT 影像诊断系统，需要向医院专家证明模型是依据病灶特征而非背景噪声做出判断。\n\n### 没有 Quantus 时\n- 团队仅凭肉眼观察热力图（Heatmap）来主观臆断模型是否“看对了地方”，缺乏客观数据支撑，难以说服持疑的放射科医生。\n- 试图手动复现论文中的评估指标（如忠实度、鲁棒性），但因 PyTorch 与 TensorFlow 版本差异导致代码报错，耗费数周仍无法跑通基准测试。\n- 面对数十种解释性算法（如 Grad-CAM, LIME），不知道哪种最适合当前场景，只能盲目尝试，导致模型优化方向错误。\n- 缺乏统一的评估框架，不同成员使用的评估标准不一致，使得实验结果无法复现，严重阻碍了模型的合规审批进程。\n\n### 使用 Quantus 后\n- 直接调用 Quantus 内置的 35+ 种标准化指标（如 Faithfulness, Robustness），生成量化评分报告，用确凿数据向医生证明模型关注区域的可靠性。\n- 利用其对 PyTorch 和 TensorFlow 的原生支持及预置接口，几分钟内即可集成 Captum 或 tf-explain 生成的解释结果进行批量评估，效率提升十倍不止。\n- 通过对比不同解释方法在六大类别下的得分雷达图，快速筛选出最能反映病灶特征的算法组合，精准指导模型迭代。\n- 依托其严格的版本控制和可复现性设计，确保所有评估流程符合学术与行业标准，大幅缩短了医疗器械注册所需的验证周期。\n\nQuantus 将模糊的“模型直觉”转化为可信的“量化证据”，成为连接黑盒算法与高风险领域落地应用的关键桥梁。","https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Funderstandable-machine-intelligence-lab_Quantus_ad4d4c0e.png","understandable-machine-intelligence-lab","Understandable Machine Intelligence Lab","https:\u002F\u002Foss.gittoolsai.com\u002Favatars\u002Funderstandable-machine-intelligence-lab_b1239f49.jpg","Understandable Machine Intelligence Lab: We bring #explainable #AI to the next level. Part of  @TUBerlin  #ML Department, funded by  @BMBF_Bund  #XAI",null,"TUBerlin_UMI","https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab",[80,84],{"name":81,"color":82,"percentage":83},"Jupyter Notebook","#DA5B0B",89.5,{"name":85,"color":86,"percentage":87},"Python","#3572A5",10.5,658,89,"2026-04-14T11:31:39","NOASSERTION",1,"未说明","未说明 (支持 PyTorch 和 TensorFlow，具体取决于用户模型需求)",{"notes":96,"python":97,"dependencies":98},"该工具包主要用于评估神经网络解释（XAI），支持 PyTorch 和 TensorFlow 框架。扩展支持 Captum、tf-explain 和 Zennit 等解释方法库。由于处于活跃开发中，使用时需注意版本号以确保结果可复现。支持图像、时间序列和表格数据。","3.8, 3.9, 3.10, 3.11",[99,100,101,102,103,104,105,106],"torch","tensorflow","captum","tf-explain","zennit","numpy","scipy","matplotlib",[14,108],"其他",[110,111,112,113,114,100,115,116,117],"machine-learning","explainable-ai","quantification-evaluation-methods","deep-learning","pytorch","interpretability","xai","reproducibility","2026-03-27T02:49:30.150509","2026-04-17T10:19:23.806452",[121,126,131,136],{"id":122,"question_zh":123,"answer_zh":124,"source_url":125},37382,"Quantus 是否支持非图像数据（如 1D 信号）或可变形状的输入？","是的，Quantus 已更新以支持可变的输入形状。只要存在批次维度（axis=0），且通道位于开头（axis=1）或结尾（axis=-1）并显式传递 `channel_first` 参数，即可支持任意 `numpy.ndarray` 形状。但请注意，部分功能（如 `baseline_replacement_by_blur`、旋转\u002F平移变换、超像素生成以及 `IterativeRemovalOfFeatures` 指标）目前尚未完全支持 1D 输入，因为它们依赖仅适用于 2D 图像的库（如 cv2, skimage）。","https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fissues\u002F51",{"id":127,"question_zh":128,"answer_zh":129,"source_url":130},37383,"`perturb_baseline` 参数中的 'random' 和 'uniform' 选项有什么区别？应该用哪个？","`random` 和 `uniform` 选项在语义上是重叠的，都会返回相同的值，但 `random` 函数的边界限制更多，容易造成混淆。建议统一使用 `uniform` 选项。维护者已在后续代码中移除了 `random` 选项及相关导入，推荐用户在代码中直接使用 `uniform`。","https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fissues\u002F106",{"id":132,"question_zh":133,"answer_zh":134,"source_url":135},37384,"`IterativeRemovalOfFeatures` 指标计算的是 AOC 还是 AUC？","这是一个已修复的错误。该指标原本错误地计算了 AUC（曲线下面积），而实际上应该计算 AOC（Area Over the Perturbation Curve）。问题已在代码中修正（参见 `faithfulness.py` 第 708 行附近）。此外，AOPC\u002FAOC 分数并不像某些人误解的那样必须 bounded 在 [0,1] 范围内，其行为类似于 AUC。","https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fissues\u002F113",{"id":137,"question_zh":138,"answer_zh":139,"source_url":140},37385,"如何在回归任务中使用 Quantus 的可解释性方法？遇到 'Expected dtype int64 for index' 错误怎么办？","目前 Quantus 的许多方法主要针对分类任务设计，直接在回归任务（如 1D CNN 回归）上使用可能会报错（例如 `gather_out_cpu(): Expected dtype int64 for index` 或 TensorFlow 中的类型转换错误）。这是因为内部实现假设标签为整数类别索引。对于回归任务，当前没有官方直接支持的内置指标，用户可能需要自定义 `similarity_func` 或等待未来版本增加对回归任务的专门支持。目前建议检查传入的 `y_batch` 格式是否符合分类任务要求，或参考源码差异自行适配 PyTorch\u002FTensorFlow 的后端处理逻辑。","https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fissues\u002F174",[142,147,152,157,162,167,172,177,182,187,192,197,202,207,212,217,222,227,232,237],{"id":143,"version":144,"summary_zh":145,"released_at":146},297940,"v0.6.0","# v0.6.0  \n本次发布对 Quantus 进行了重大更新，重点提升性能、可扩展性及兼容性。核心新增功能是支持批处理的指标评估，这显著提高了运行效率。此外，还增加了对 HuggingFace 模型的初步支持、内部代码清理以及兼容性升级。\n\n---\n\n### 变更内容\n\n#### 解释性指标的批处理支持\n\n- 为以下指标添加了基于批处理的评估功能：\n  - `PixelFlipping`、`Monotonicity`、`MonotonicityCorrelation`、`FaithfulnessCorrelation`、`FaithfulnessEstimate`。\n- 适配批处理计算的工具函数：\n  - `correlation_pearson`、`correlation_spearman`、`correlation_kendall_tausimilarity`、`calculate_auc`、`get_baseline_dict`。\n- 为保持一致性和提升运行效率，移除了旧版的逐样本版本。\n\n通过每样本随机运行 30 次的方式，将批处理结果与先前输出进行并列对比，验证了批处理的正确性。对于确定性指标，使用 `np.allclose` 进行校验；而对于随机性指标，则采用统计检验方法（例如，t 检验显示超过 90% 的情况下输出高度一致）。\n\n*感谢 @davor10105 主导批处理重构与验证工作。*\n\n---\n\n#### HuggingFace Transformers 支持\n\n- 通过 `PyTorchModel` 封装器实现了对 `SequenceClassification` 任务的 `transformers` 初步集成。\n- `predict` 方法现可直接处理 HuggingFace 模型。\n- 增加了测试覆盖率，并通过 `tox` 和 `flake8` 进行代码检查。\n\n*感谢 @abarbosa94 完成 Transformer 实现。*\n\n---\n\n#### 其他改进\n\n- 放弃对 Python 3.7 的支持。\n- 清理并优化了 `pyproject.toml` 文件；提升了代码质量检查与模块验证（`mypy`、`flake8`、`isort`）。\n- 修复了 `base.py` 中的 `normalise_func_kwargs` 错误。\n- 解决了批处理输入下区域扰动可视化中的形状不匹配问题（#353）。\n- 更新了文档和 README 中的链接（例如 QUANDA 相关引用）。","2025-07-21T15:28:54",{"id":148,"version":149,"summary_zh":150,"released_at":151},297941,"v0.5.3","## 变更内容\n* 修复 bug：@annahedstroem 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F318 中为 SmoothMPRT 和 zennit 测试添加了 explain_func_kwargs 参数。\n* 改进：@annahedstroem 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F319 中添加了对已弃用参数的警告（并进行了优雅处理）。\n\n\n**完整变更日志**：https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcompare\u002Fv0.5.2...v0.5.3","2023-12-05T11:42:07",{"id":153,"version":154,"summary_zh":155,"released_at":156},297942,"v0.5.2","## 变更内容\n* 由 @annahedstroem 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F316 中修复了指标中的缩进错误\n* 由 @annahedstroem 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F317 中发布了 0.5.2 版本的错误修复\n\n**完整变更日志**: https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcompare\u002Fv0.5.1...v0.5.2","2023-12-01T14:26:53",{"id":158,"version":159,"summary_zh":160,"released_at":161},297943,"v0.5.1","## 变更内容\n* 修复了 `EfficientMPRT` 指标中的错误，并由 @annahedstroem 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F314 中扩展了 `quantus.evaluate` 函数。\n* 更新了 `pandas` 的版本控制（以便与 `Colab` 环境顺畅兼容），并由 @annahedstroem 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F315 中重新添加了模型参数随机化测试的警告打印信息。\n\n\n**完整变更日志**: https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcompare\u002Fv0.5.0...v0.5.1","2023-11-27T11:16:59",{"id":163,"version":164,"summary_zh":165,"released_at":166},297944,"v0.5.0","## 变更内容\n\n在本次发布中，我们引入了两个新的指标：SmoothMPRT 和 EfficientMPRT，它们是模型参数随机化检验（MPRT）的变体。这些实现源自 Hedström 等人于 2023 年在 NeurIPS XAIA 上发表的论文 _\"Sanity Checks Revisited: An Exploration to Repair the Model Parameter Randomisation Test\"_。\n\n* @annahedstroem 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F307 中完成了 PR 发布 v0.4.5 的统一工作。\n* @annahedstroem 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F308 中实现了 MPRT 指标：EfficientMPRT 和 SmoothMPRT。\n* 其他较小的修复：\n    * @annahedstroem 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F312 中进行了修复。\n    * @annahedstroem 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F313 中进行了修复。\n\n\n**完整变更日志**：https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcompare\u002Fv0.4.5...v0.5.0","2023-11-24T14:46:09",{"id":168,"version":169,"summary_zh":170,"released_at":171},297945,"v0.4.5","## 变更内容\n* 由 @aaarrti 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F294 中统一了指标 API\n\n\n**完整变更日志**: https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcompare\u002Fv0.4.4...v0.4.5","2023-11-17T11:09:32",{"id":173,"version":174,"summary_zh":175,"released_at":176},297946,"v0.4.4","## 变更内容\n* 移除冗余的 attributes_check，由 @aaarrti 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F287 中完成\n* 解决 pytorch_model.py 中的 softmax 问题，由 @dilyabareeva 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F283 中完成\n* 修复 channels first 问题，由 @annahedstroem 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F305 中完成\n* 新版本 0.4.4，由 @annahedstroem 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F306 中发布\n\n\n**完整变更日志**: https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcompare\u002Fv0.4.3...v0.4.4","2023-10-26T16:25:48",{"id":178,"version":179,"summary_zh":180,"released_at":181},297947,"v0.4.3","## 变更内容\n* 修复 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fissues\u002F292\n* @annahedstroem 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F291 中对 v0.4.2 的修复\n* @aaarrti 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F293 中对动态版本的修复\n\n\n**完整变更日志**: https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcompare\u002Fv0.4.2...v0.4.3","2023-08-10T10:58:00",{"id":183,"version":184,"summary_zh":185,"released_at":186},297948,"v0.4.2","## 变更内容\n* 增加了对量子功能的曝光度，包括 `model` 和 `utils`\n* 由 @annahedstroem 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F279 中提升了指标数据和模型适用性的透明度\n* @annahedstroem 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F282 中更新了 evaluation.py 文件\n* @annahedstroem 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F289 中修复了 max_sensitivity.py 中的问题\n\n\n**完整变更日志**: https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcompare\u002Fv0.4.1...v0.4.2","2023-08-09T15:04:45",{"id":188,"version":189,"summary_zh":190,"released_at":191},297949,"v0.4.1","## 变更内容\n* 审查距离度量，并由 @annahedstroem 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F264 中添加了平方距离作为度量方法。\n* 由 @annahedstroem 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F266 中更新了 requirements_test.txt 文件。\n* 由 @aaarrti 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F269 中添加了 py.typed 标记。\n* 由 @aaarrti 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F270 中完成了 Toml 和 Tox 的迁移。\n* 由 @aaarrti 在 https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F278 中更新了 README.md 文件。\n\n\n**完整变更日志**: https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcompare\u002Fv0.4.0...v0.4.1","2023-06-27T11:09:57",{"id":193,"version":194,"summary_zh":195,"released_at":196},297950,"v0.4.0","## What's Changed\r\n* Fix warn_perturbation_caused_no_change by @p-wojciechowski in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F250\r\n* Packaging by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F256\r\n    * Add 3.10 and 3.11 Python support\r\n    * Clean-up requirements_test.txt\r\n    * Update `torch`, `tensorflow` versions and fixing `protobuf` incompatibility\r\n    * Check that optional dependencies packages in `setup.py` are the same as in `requirements_test.txt`\r\n    * Clean-up `setup.py`, including adding required and removing unnecessary imports\r\n    * Clean-up `pytests`\r\n* Setuptools-bug fixed, extra_requires not correctly identified by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F258\r\n* Updated python publish script by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F259\r\n* Python package bug by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F260\r\n* Updated setup.py with absolute path by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F262\r\n* Add reqs via MANIFEST.ini by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F263\r\n\r\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcompare\u002Fv0.3.5...v0.4.0","2023-05-09T20:54:54",{"id":198,"version":199,"summary_zh":200,"released_at":201},297951,"v0.3.5","## What's Changed\r\n* #195 fix autodoc for special members by @aaarrti in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F239\r\n* #219 fix unstable tests by @aaarrti in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F241\r\n* Added implementation and tests for return_aggregate=True for BatchedMetric by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F244\r\n* Action fixes by @aaarrti in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F246\r\n* Fix bugs and provide flexibility for Softmax argument by @dilyabareeva in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F230\r\n* Tutorial addition and resolving fixture by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F245\r\n\r\n\r\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcompare\u002Fv0.3.4...v0.3.5","2023-04-05T09:47:19",{"id":203,"version":204,"summary_zh":205,"released_at":206},297952,"v0.3.4","## What's Changed\r\n* Add tests for tabular (2D) data by @aaarrti in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F218\r\n* Fixed TF explain bug by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F231\r\n\r\n\r\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcompare\u002Fv0.3.3...v0.3.4","2023-02-22T14:40:54",{"id":208,"version":209,"summary_zh":210,"released_at":211},297953,"v0.3.3","## What's Changed\r\n* Updated contact details by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F226\r\n* Issue 199 by @dilyabareeva in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F214\r\n* Fix minor misleading error message in Robustness metrics by @dilyabareeva in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F227\r\n* Input Invariance Metric (issue 169) by @dilyabareeva in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F225\r\n* exposed constants to user by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F228\r\n\r\n\r\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcompare\u002Fv0.3.2...v0.3.3","2023-02-14T12:04:33",{"id":213,"version":214,"summary_zh":215,"released_at":216},297954,"v0.3.2","## What's Changed\r\n* Added first draft of normalise_by_average_second_moment_estimate by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F197\r\n* Use mini batch for tests by @aaarrti in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F198\r\n* TF model refactorings by @aaarrti in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F209\r\n* Feature\u002Frelative stability metric by @aaarrti in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F210\r\n* Issue 122 by @dilyabareeva in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F205\r\n* add new methods to captum and tf-explain funcs by @dilyabareeva in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F206\r\n* Fixing issue with blur_at_indices (Issue #200) by @leanderweber in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F208\r\n* Fix IROF metric - perturbation accumulation and counting the segments by @p-wojciechowski in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F221\r\n\r\n## New Contributors\r\n* @p-wojciechowski made their first contribution in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F221\r\n\r\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcompare\u002Fv0.3.1...v0.3.2","2023-01-25T10:50:47",{"id":218,"version":219,"summary_zh":220,"released_at":221},297955,"v0.3.1","## What's Changed (smaller PRs)\r\n* Improve error message by @vedal in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F185\r\n* Default value of explain_func_kwargs cannot be unpacked. by @vedal in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F184\r\n* Added AUC calculation to Pixel-Flipping by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F189\r\n* Docs by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F190\r\n* #173 Fix GitHub heatmaps by @aaarrti in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F188\r\n* Validate predictions change robustness by @aaarrti in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F183\r\n* Citation fixes by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F192\r\n* Update requirements_test.txt by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F193\r\n* Tutorial fixes by @dilyabareeva in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F194\r\n\r\n\r\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcompare\u002Fv0.3.0...v0.3.1","2022-11-21T18:13:05",{"id":223,"version":224,"summary_zh":225,"released_at":226},297956,"v0.3.0","## What's Changed\r\n\r\n### Bigger PRs\r\n* **Refactor `custom_preprocess()` for more robustness** by @dkrako, reviewed by @annahedstroem, @leanderweber in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F167\r\n  - Make the implementation `custom_preprocess()` more flexible: instead of passing the complete tuple with `model, x_batch, y_batch, a_batch, s_batch, ..` we instead pass a dictionary with the variable names as keys.\r\n* **Batched evaluation** by @dkrako, reviewed by @annahedstroem, @leanderweber  in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F168\r\n   - This PR implements the classes BatchedMetric and BatchedPerturbationMetric\r\n   - This is a rewrite of this PR: https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F87\r\n   - It solves the following issue: https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fissues\u002F80\r\n   - This will speed up a lot of metrics.\r\n   - Not all metrics in Quantus are yet updated with BatchedPerturbationMetric. The efforts of deploying this new construct with more metrics for efficiency gains will continue.\r\n* **Change normalisation default, fix bug in normalise_by_negative, adapt citations, absolute imports** by @leanderweber, reviewed by @annahedstroem, @dkrako in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F166\r\n   - Make some Updates to the readthedocs\r\n   - Update and Unify Citations in all Metrics\r\n   - Update, unify, and sort all imports\r\n   - Change all relative imports to absolute\r\n   - Rethink star imports. Specifically, restructure helpers to distinguish internal functions and functions that should be user-accessible.\r\n   - Change the normalisation_func default to normalise_by_max instead of normalise_by_negative due to the latter treating pos\u002Fneg values differently\r\n   - Target some bugs within the normalisation functions and their application\r\n   - Merge installation options rework (see PR https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F172)\r\n   - Updated and cleaned-up README.md file\r\n* **Mypy: fix errors fix** by @dilyabareeva, reviewed by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F171\r\n   - An in-depth look at the typing, branched from mypy-static-type-checker\r\n   - Fix errors or reject with reason\r\n\r\n### Smaller PRs\r\n* **Installation option refactoring to be more concise** by @leanderweber, reviewed by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F172\r\n   - Refactored installation options, e.g., \"extras\" removed, instead singular XAI libraries are targetable.\r\n   - Updated Quantus package dependencies\r\n* **Bugfixes to Installation and Tutorials** by @leanderweber, reviewed by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F176\r\n* **Tabular data minimal example** by @dilyabareeva, reviewed by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F175\r\n   - Jupyter notebook\u002F tutorial based on Titanic data, showing with a minimal example how to use Quantus\r\n* **Add time series tutorial** by @dkrako, reviewed by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F177\r\n* Remove coverage>=6.2 by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F179\r\n\r\n\r\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcompare\u002Fv0.2.2...v0.3.0","2022-10-21T15:28:34",{"id":228,"version":229,"summary_zh":230,"released_at":231},297957,"v0.2.2","## What's Changed 🚀 \r\n\r\n### Bigger PRs:\r\n* Fixes, license, readthedocs and docstrings by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F156\r\n  - Removed coverage from requirements.txt\r\n  - Finished all docstrings accordingly to numpydoc\r\n  - Included API documentation\r\n  - Configured readthedocs.org\r\n  - Added license stubs to all src files\r\n  - Added codecov badge\r\n  - Added assumptions on data applicability\r\n  - Removed * imports\r\n* Fixes and enhancements tutorials, other fixes to tests, .gitignore and README.md by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F164\r\n  - Added ImageNet data as examples\r\n  - Tutorials clean-up and added data to assets\r\n  - Fixed all tutorials, removed colab dependency and simplified install\r\n  - Updated .gitignore\r\n  - Replaced an assert with a warning\r\n  - Moved PR template\r\n  - Cleaned up assets folder\r\n  - Removed term_color\r\n  - Updated docs, post-tutorial API docs fixes\r\n\r\n### Smaller PRs:\r\n* 146 treatment of slices in baseline replacement by indices is wrong by @leanderweber in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F148\r\n* Require `numpy>=1.19.5` for correct version solving by @vedal in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F151\r\n* Code formatting and documentation by @dilyabareeva in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F154\r\n* bug: tensorflow is required, even if not used by @vedal in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F158\r\n* Create pull_request_template.md by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F161\r\n* Create issue templates by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F160\r\n* CONTRIBUTING.md by @dilyabareeva in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F157\r\n* Fixing indentation of LLE metric by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F162\r\n* Create python-publish.yml by @annahedstroem in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F163\r\n\r\n\r\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcompare\u002Fv0.2.1...v0.2.2","2022-10-08T18:27:36",{"id":233,"version":234,"summary_zh":235,"released_at":236},297958,"v0.2.1","### Minor fixes\r\n\r\n- [Merge branch 'main' into 146-treatment-of-slices-in-baseline_replacem…](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcommit\u002F960e94969ab46590ff8b38ceed5800f3d4dea7b9) \r\n- [Merge pull request](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcommit\u002Fea0c41b7b2081766b69ddb2d9b38eef8bf7d9f23) \r\n- [Fixed bug Relevance Rank Accuracy - reversed line 140 to earlier vers…](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcommit\u002Faf98d01de5bb5a118e5f41f41c15619f7bfbdc05) \r\n\r\n\r\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcompare\u002Fv0.2.0...v0.2.1","2022-09-26T14:02:16",{"id":238,"version":239,"summary_zh":240,"released_at":241},297959,"v0.2.0","## What's Changed 🚀 \r\n\r\nFor more transparent kwargs-passing and error handling, we have updated the API of Quantus! The main changes are the following:\r\n\r\n- Add function-specific keyword-arguments to metric functions. by @dkrako in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F124\r\n\r\n**From implicit to explicit argument passing.** To mitigate the risk of user typos and other undefined behaviours, the new version `0.2.0` forces explicit argument passing of all metric initalisations and calls. Instead of passing all evaluation parameters in one single kwargs list, e.g., \r\n  ```python\r\n  kwargs = {\"n_perturb_samples\": 10, \"abs\": False, \"normalise\": True, \"explain_func\": quantus.explain, method: \"Saliency\"}}\r\n  metric = Infidelity(**kwargs)\r\n  metric(model=model, x_batch=x, y_batch=y, a_batch=a, s_batch=s, **kwargs)\r\n  ``` \r\n  which could for example result in typos being overlooked, we now separate the parameters into different dictionaries such as `normalise_func_kwargs`, `explain_func_kwargs`, `model_predict_kwargs`. With the new API, the same call looks like this:\r\n  ```python\r\n  metric = Infidelity(n_perturb_samples=10, abs=False, normalise=True)\r\n  metric(model=model, x_batch=x, y_batch=y, a_batch=a, s_batch=s, explain_func=quantus.explain, explain_func_kwargs={\"method\": \"Saliency\"})\r\n  ```\r\n  In this way, we assert better control over the parameters. If an unexpected argument is passed, an error will now be thrown. \r\n  \r\n**Extended the `base.Metric` class.** Many metrics share the same logic when it comes to general attributes and pre-processing. To remove duplicate code, we extended the base class, which now includes some key methods `general_preprocessing`, `custom_preprocessing`, `evaluate_instance`, `custom_postprocessing` all which could be used for various `Metric` implementations. \r\n\r\n**Additional functionality in helpers and increased general code-quality.** We have shortended and rewritten scripts where necessary for better code readability. Added new functions and refactored existing ones to adhere to [PEP-8](https:\u002F\u002Fwww.python.org\u002Fdev\u002Fpeps\u002Fpep-0008\u002F). \r\n\r\nIf these API changes are not suitable for your project's needs, please install a previous release version such as [v0.1.6](https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Freleases\u002Ftag\u002Fv0.1.6) via:\r\n\r\n```bash\r\npip install quantus==0.1.6\r\n```\r\n\r\nAlso, a minor PR was incorporated:\r\n- Use `auto.tqdm` by @aaarrti in https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fpull\u002F143\r\n\r\n**Full Changelog**: https:\u002F\u002Fgithub.com\u002Funderstandable-machine-intelligence-lab\u002FQuantus\u002Fcompare\u002Fv0.1.6...v0.2.0","2022-09-25T15:51:52"]