MEGA Hub

Building Auditable AI Text Detectors with Sparse Autoencoders

Authors

Do you know Liam Dugan?You can claim authorship or link another user.Do you know Callum McDougall?You can claim authorship or link another user.Do you know Matthew Denton?You can claim authorship or link another user.Do you know Philip Pham?You can claim authorship or link another user.Do you know Christine Kaeser-Chen?You can claim authorship or link another user.Do you know Chris Callison-Burch?You can claim authorship or link another user.

Abstract

AI-generated text detectors are increasingly deployed in consequential settings---peer review, education, admissions, journalism, and content moderation---where their outputs directly affect individuals. Yet the strongest detectors are fine-tuned neural classifiers whose decisions are opaque, while transparent alternatives (stylometric and linguistic-feature classifiers) underperform them by wide margins. We present a pipeline that converts a fine-tuned neural detector into a classifier that operates on human-readable concepts. We accomplish this by training sparse autoencoders (SAEs) directly on a detector's residual stream, assigning auto-interpretation labels to each SAE feature, and retaining only features whose generated descriptions pass a global fidelity threshold. We then train a lightweight classifier on just the interpretable features. We find that our SAEs recover latent concepts that correspond to highly specific and predictive linguistic "tells" of AI-generated text. Classifiers built on these extracted features are not only highly interpretable but exhibit stronger performance on out-of-distribution texts than either neural or stylometric alternatives.

Community

00