PROJECT 1

Mechanistically-Aware Decoding for Ensuring Factuality During Retrieval Augmented Generation.

Project Leader: Robin Jia, Assistant Professor, Computer Science Department

Website: https://robinjia.github.io/

Abstract: Users increasingly rely on large language models to summarize, condense, and answer questions about documents retrieved from the web, a setting known as retrieval-augmented generation. However, models are known to "hallucinate" claims that are not supported by these documents, leading to serious reliability and trustworthiness concerns. We propose a novel method for reducing hallucinations called Mechanistically-Aware Decoding (MAD). Our method leverages recent insights into language model internal mechanisms to steer the generated text away from claims that are unlikely to be supported by the context documents. MAD is efficient, general-purpose, requires no task-specific training, and is complementary with other approaches to reduce hallucinations. We will demonstrate that MAD substantially reduces hallucinations for a wide variety of application domains and language models.