Papers and notes on LLM factuality, decoding methods, and mechanistic interpretability.
This repository collects reading notes on improving factuality and mitigating hallucinations in large language models, with a focus on:
- inference-time intervention
- decoding methods
- hallucination mitigation
- internal representations
- mechanistic interpretability
- feature-level understanding and editing
The repository is organized so that each paper lives in its own folder inside papers/, with the PDF and note document together.
后层打假— DoLa / 中文尖激活预警— In-Context Sharpness as Alerts / 中文想太多会错— Overthinking the Truth / 中文看证据再说— Trusting your Evidence / 中文会说没把握— Enhancing Trust in Large Language Models with Uncertainty-Aware Fine-Tuning / 中文自提证据— SelfElicit / 中文语义轨迹验真— The Geometry of Truth / 中文无害提示也会骗— BEYOND PROMPT-INDUCED LIES / 中文算图验推理— VERIFYING CHAIN-OF-THOUGHT REASONING VIA ITS COMPUTATIONAL GRAPH / 中文推理RAG全景图— Towards Agentic RAG with Deep Reasoning / 中文
- DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models (ICLR 2024) / 中文
- In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation (2024) / 中文
- Trusting your Evidence: Hallucinate less with Context-Aware Decoding (NAACL 2024) / 中文
- Enhancing Trust in Large Language Models with Uncertainty-Aware Fine-Tuning (2024) / 中文
- SelfElicit (ACL 2025) / 中文
- Monitoring Decoding: Mitigating Hallucination via Evaluating the Factuality of Partial Response during Generation (ACL 2025) / 中文
- The Geometry of Truth: Layer-wise Semantic Dynamics for Hallucination Detection in Large Language Models (2025) / 中文
- BEYOND PROMPT-INDUCED LIES: Investigating LLM Deception on Benign Prompts (2026) / 中文
- VERIFYING CHAIN-OF-THOUGHT REASONING VIA ITS COMPUTATIONAL GRAPH (2026) / 中文
- OVERTHINKING THE TRUTH: UNDERSTANDING HOW LANGUAGE MODELS PROCESS FALSE DEMONSTRATIONS (ICLR 2024) / 中文
- Sparse Autoencoders Find Highly Interpretable Features in Language Models (2023) / 中文
- Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models (ICLR 2025) / 中文