SelfCheckGPT

• 我们论文 "SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models" 的项目页面
• 我们研究了 selfcheck 方法的几种变体：BERTScore（基于BERT的相似度评分）、Question-Answering（问答）、n-gram（n元语法）、NLI（自然语言推理）和 LLM-Prompting（大语言模型提示）。
• [2023年11月] 感谢 Daniel Huynh 的 SelfCheckGPT-NLI 校准分析 [链接到文章]
• [2023年10月] 论文已被 EMNLP 2023 接收并即将发表 [海报]
• [2023年8月] ML Collective 演讲的幻灯片 [链接到幻灯片]

代码/包

安装

pip install selfcheckgpt

SelfCheckGPT 使用方法：BERTScore、QA、n-gram

本包中有三种 SelfCheck 分数变体，如论文所述：SelfCheckBERTScore()、SelfCheckMQAG()、SelfCheckNgram()。所有变体都有 predict() 方法，该方法会输出句子级别（sentence-level）的分数，用于衡量与采样段落（sampled passages）的一致性。您可以使用 spacy（一个自然语言处理库）等包将段落拆分为句子。为了可复现性（reproducibility），您可以在调用此函数前设置 torch.manual_seed。更多详情见 Jupyter Notebook（交互式笔记本）demo/SelfCheck_demo1.ipynb

# Include necessary packages (torch, spacy, ...)
from selfcheckgpt.modeling_selfcheck import SelfCheckMQAG, SelfCheckBERTScore, SelfCheckNgram
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
selfcheck_mqag = SelfCheckMQAG(device=device) # set device to 'cuda' if GPU is available
selfcheck_bertscore = SelfCheckBERTScore(rescale_with_baseline=True)
selfcheck_ngram = SelfCheckNgram(n=1) # n=1 means Unigram, n=2 means Bigram, etc.

# LLM's text (e.g. GPT-3 response) to be evaluated at the sentence level  & Split it into sentences
passage = "Michael Alan Weiner (born March 31, 1942) is an American radio host. He is the host of The Savage Nation."
sentences = [sent.text.strip() for sent in nlp(passage).sents] # spacy sentence tokenization
print(sentences)
['Michael Alan Weiner (born March 31, 1942) is an American radio host.', 'He is the host of The Savage Nation.']

# Other samples generated by the same LLM to perform self-check for consistency
sample1 = "Michael Alan Weiner (born March 31, 1942) is an American radio host. He is the host of The Savage Country."
sample2 = "Michael Alan Weiner (born January 13, 1960) is a Canadian radio host. He works at The New York Times."
sample3 = "Michael Alan Weiner (born March 31, 1942) is an American radio host. He obtained his PhD from MIT."

# --------------------------------------------------------------------------------------------------------------- #
# SelfCheck-MQAG: Score for each sentence where value is in [0.0, 1.0] and high value means non-factual
# Additional params for each scoring_method:
# -> counting: AT (answerability threshold, i.e. questions with answerability_score < AT are rejected)
# -> bayes: AT, beta1, beta2
# -> bayes_with_alpha: beta1, beta2
sent_scores_mqag = selfcheck_mqag.predict(
    sentences = sentences,               # list of sentences
    passage = passage,                   # passage (before sentence-split)
    sampled_passages = [sample1, sample2, sample3], # list of sampled passages
    num_questions_per_sent = 5,          # number of questions to be drawn  
    scoring_method = 'bayes_with_alpha', # options = 'counting', 'bayes', 'bayes_with_alpha'
    beta1 = 0.8, beta2 = 0.8,            # additional params depending on scoring_method
)
print(sent_scores_mqag)
# [0.30990949 0.42376232]

# --------------------------------------------------------------------------------------------------------------- #
# SelfCheck-BERTScore: Score for each sentence where value is in [0.0, 1.0] and high value means non-factual
sent_scores_bertscore = selfcheck_bertscore.predict(
    sentences = sentences,                          # list of sentences
    sampled_passages = [sample1, sample2, sample3], # list of sampled passages
)
print(sent_scores_bertscore)
# [0.0695562  0.45590915]

# --------------------------------------------------------------------------------------------------------------- #
# SelfCheck-Ngram: Score at sentence- and document-level where value is in [0.0, +inf) and high value means non-factual
# as opposed to SelfCheck-MQAG and SelfCheck-BERTScore, SelfCheck-Ngram's score is not bounded
sent_scores_ngram = selfcheck_ngram.predict(
    sentences = sentences,   
    passage = passage,
    sampled_passages = [sample1, sample2, sample3],
)
print(sent_scores_ngram)
# {'sent_level': { # sentence-level score similar to MQAG and BERTScore variant
#     'avg_neg_logprob': [3.184312, 3.279774],
#     'max_neg_logprob': [3.476098, 4.574710]
#     },
#  'doc_level': {  # document-level score such that avg_neg_logprob is computed over all tokens
#     'avg_neg_logprob': 3.218678904916201,
#     'avg_max_neg_logprob': 4.025404834169327
#     }
# }

SelfCheckGPT 使用方法：NLI（推荐）

以句子和采样段落作为输入的蕴含（Entailment）或矛盾（Contradiction）分数可以用作 selfcheck 分数。我们使用在 Multi-NLI（多体裁自然语言推理数据集）上微调的 DeBERTa-v3-large（一种预训练语言模型），对“蕴含”或“矛盾”类别的概率进行归一化（normalize），并将 Prob(contradiction) 作为分数。

from selfcheckgpt.modeling_selfcheck import SelfCheckNLI
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
selfcheck_nli = SelfCheckNLI(device=device) # set device to 'cuda' if GPU is available

sent_scores_nli = selfcheck_nli.predict(
    sentences = sentences,                          # list of sentences
    sampled_passages = [sample1, sample2, sample3], # list of sampled passages
)
print(sent_scores_nli)
# [0.334014 0.975106 ] -- based on the example above

SelfCheckGPT 使用方法：LLM 提示

提示 LLM（Llama2、Mistral、OpenAI 的 GPT）在 zero-shot（零样本）设置下评估信息一致性。我们查询 LLM 以评估第 i 个句子是否得到样本（作为上下文）的支持。与其他方法类似，分数越高表示越有可能是幻觉。下面是一个使用 Mistral 的示例：

# Option1: open-source model
from selfcheckgpt.modeling_selfcheck import SelfCheckLLMPrompt
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
llm_model = "mistralai/Mistral-7B-Instruct-v0.2"
selfcheck_prompt = SelfCheckLLMPrompt(llm_model, device)

# Option2: API access 
# (currently only support OpenAI and Groq)
# from selfcheckgpt.modeling_selfcheck_apiprompt import SelfCheckAPIPrompt
# selfcheck_prompt = SelfCheckAPIPrompt(client_type="openai", model="gpt-3.5-turbo")
# selfcheck_prompt = SelfCheckAPIPrompt(client_type="groq", model="llama3-70b-8192", api_key="your-api-key")

sent_scores_prompt = selfcheck_prompt.predict(
    sentences = sentences,                          # list of sentences
    sampled_passages = [sample1, sample2, sample3], # list of sampled passages
    verbose = True, # whether to show a progress bar
)
print(sent_scores_prompt)
# [0.33333333, 0.66666667] -- based on the example above

LLM（大型语言模型）可以是 HuggingFace 上可用的任何模型。默认的 prompt template（提示模板）为 Context: {context}\n\nSentence: {sentence}\n\nIs the sentence supported by the context above? Answer Yes or No.\n\nAnswer: ，但您可以使用 selfcheck_prompt.set_prompt_template(new_prompt) 进行更改。

大多数模型（gpt-3.5-turbo、Llama2、Mistral）>95% 的时间会输出 'Yes' 或 'No'，而任何剩余输出可设为 N/A。输出将转换为分数：Yes -> 0.0，No -> 1.0，N/A -> 0.5。inconsistency score（不一致性分数）随后通过取平均值计算得出。

数据集

wiki_bio_gpt3_hallucination 数据集目前包含 238 篇标注过的段落（v3）。您可以在论文中或我们的 HuggingFace 数据卡片上找到更多信息：https://huggingface.co/datasets/potsawee/wiki_bio_gpt3_hallucination。要使用此数据集，您可以通过 HuggingFace 数据集 API 加载，也可以直接从下方以 JSON 格式下载。

更新

我们进一步标注了 GPT-3 wikibio 段落，现在数据集包含 238 篇标注过的段落。以下是 v1 中前 65 篇段落的 ID 链接。

选项1：HuggingFace

from datasets import load_dataset
dataset = load_dataset("potsawee/wiki_bio_gpt3_hallucination")

选项2：手动下载

从我们的 Google Drive 下载，然后您可以在 Python 中加载：

import json
with open("dataset.json", "r") as f:
    content = f.read()
dataset = json.loads(content)

每个实例包含：

• gpt3_text：GPT-3 生成的段落
• wiki_bio_text：实际的 Wikipedia 段落（第一段）
• gpt3_sentences：使用 spacy 将 gpt3_text 拆分为句子
• annotation：句子级别的人工标注
• wiki_bio_test_idx：来自原始 wikibio 数据集（测试集）的概念/个体 ID
• gpt3_text_samples：采样段落列表（do_sample = True & temperature = 1.0）

实验

基于概率的基线方法（例如 GPT-3 的概率）

如我们的论文所述，生成式 LLM 的概率（和生成熵）可用于衡量其置信度。请查看我们在 demo/experiments/probability-based-baselines.ipynb 中的示例/实现。

实验结果

• 完整细节可在我们的论文中找到。
• 请注意，我们的新结果表明，LLM（如 GPT-3 (text-davinci-003) 或 ChatGPT (gpt-3.5-turbo)）擅长文本不一致性评估。基于这一发现，我们尝试了 SelfCheckGPT-Prompt，其中每个待评估的句子都与每个 sampled_passage 进行比较，通过提示 ChatGPT 完成。SelfCheckGPT-Prompt 是表现最佳的方法。

在 wiki_bio_gpt3_hallucination 数据集上的结果。

方法	非事实 (AUC-PR)	事实 (AUC-PR)	排序 (PCC)
随机猜测	72.96	27.04	-
GPT-3 Avg(-logP)	83.21	53.97	57.04
SelfCheck-BERTScore	81.96	44.23	58.18
SelfCheck-QA	84.26	48.14	61.07
SelfCheck-Unigram	85.63	58.47	64.71
SelfCheck-NLI	92.50	66.08	74.14
SelfCheck-Prompt (Llama2-7B-chat)	89.05	63.06	61.52
SelfCheck-Prompt (Llama2-13B-chat)	91.91	64.34	75.44
SelfCheck-Prompt (Mistral-7B-Instruct-v0.2)	91.31	62.76	74.46
SelfCheck-Prompt (gpt-3.5-turbo)	93.42	67.09	78.32

其他

MQAG（多项选择题问答与生成） 在我们之前的工作中提出。我们的 MQAG 实现包含在此包中，可用于：(1) 生成多项选择题，(2) 回答多项选择题，(3) 获取 MQAG 分数。

MQAG 用法

from selfcheckgpt.modeling_mqag import MQAG
mqag_model = MQAG()

它有三个主要功能：generate()、answer()、score()。我们在 demo/MQAG_demo1.ipynb 中展示了示例用法。

致谢

本工作由剑桥大学出版社与考评部（Cambridge University Press & Assessment, CUP&A）——剑桥大学的一个部门，以及剑桥英联邦、欧洲与国际信托基金支持。

引用

@article{manakul2023selfcheckgpt,
  title={Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models},
  author={Manakul, Potsawee and Liusie, Adian and Gales, Mark JF},
  journal={arXiv preprint arXiv:2303.08896},
  year={2023}
}

SelfCheckGPT 快速上手指南

环境准备

• Python 版本: 3.7 或更高版本
• 操作系统: Linux、macOS 或 Windows
• 硬件要求: 建议使用 GPU（CUDA）加速计算，CPU 也可运行
• 前置依赖: 需要安装 PyTorch

# 安装 PyTorch（根据你的 CUDA 版本选择）
# 访问 https://pytorch.org/get-started/locally/ 获取最新安装命令
pip install torch

安装步骤

# 安装 SelfCheckGPT 主包（推荐国内镜像源加速）
pip install selfcheckgpt -i https://pypi.tuna.tsinghua.edu.cn/simple

# 安装 SpaCy 及其语言模型（用于句子分割）
pip install spacy
python -m spacy download en_core_web_sm

基本使用（NLI 方法）

NLI（自然语言推理）是推荐的方法，使用 DeBERTa 模型检测文本一致性。

import torch
import spacy
from selfcheckgpt.modeling_selfcheck import SelfCheckNLI

# 1. 初始化模型和工具
nlp = spacy.load("en_core_web_sm")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
selfcheck_nli = SelfCheckNLI(device=device)

# 2. 准备待检测文本（GPT 生成内容）
passage = "Michael Alan Weiner (born March 31, 1942) is an American radio host. He is the host of The Savage Nation."
sentences = [sent.text.strip() for sent in nlp(passage).sents]
print("待检测句子:", sentences)

# 3. 准备同一模型生成的其他样本（用于一致性对比）
sample1 = "Michael Alan Weiner (born March 31, 1942) is an American radio host. He is the host of The Savage Country."
sample2 = "Michael Alan Weiner (born January 13, 1960) is a Canadian radio host. He works at The New York Times."
sample3 = "Michael Alan Weiner (born March 31, 1942) is an American radio host. He obtained his PhD from MIT."

# 4. 执行幻觉检测
sent_scores = selfcheck_nli.predict(
    sentences=sentences,
    sampled_passages=[sample1, sample2, sample3],
)

print("幻觉检测分数:", sent_scores)
# 输出示例: [0.334014, 0.975106]

结果解读

• 分数范围: [0.0, 1.0]
• 分数含义: 分数越高，表示该句子越可能是幻觉（非事实）
• 建议阈值: 通常 0.5 以上需要警惕，可根据实际场景调整

其他检测方法

快速切换其他检测方法：

from selfcheckgpt.modeling_selfcheck import SelfCheckBERTScore, SelfCheckLLMPrompt

# BERTScore 方法（无需 GPU）
selfcheck_bertscore = SelfCheckBERTScore(rescale_with_baseline=True)
scores = selfcheck_bertscore.predict(
    sentences=sentences, 
    sampled_passages=[sample1, sample2, sample3]
)

# LLM Prompt 方法（需要 GPU 或 API）
selfcheck_prompt = SelfCheckLLMPrompt("mistralai/Mistral-7B-Instruct-v0.2", device)
scores = selfcheck_prompt.predict(
    sentences=sentences, 
    sampled_passages=[sample1, sample2, sample3]
)

更多详细示例请参考：demo/SelfCheck_demo1.ipynb