MEGA Hub

You don't need long context

Authors

Do you know Peter Belcak?You can claim authorship or link another user.Do you know Saurav Muralidharan?You can claim authorship or link another user.Do you know Jindong Jiang?You can claim authorship or link another user.Do you know Yonggan Fu?You can claim authorship or link another user.Do you know Wonmin Byeon?You can claim authorship or link another user.Do you know Hanrong Ye?You can claim authorship or link another user.Do you know Ryan Angilly?You can claim authorship or link another user.Do you know Ali Taghibakhshi?You can claim authorship or link another user.Do you know Shih-Yang Liu?You can claim authorship or link another user.Do you know Siru Ouyang?You can claim authorship or link another user.Do you know Greg Heinrich?You can claim authorship or link another user.Do you know Pavlo Molchanov?You can claim authorship or link another user.

Abstract

Later tokens are costlier to generate than earlier tokens. On frontier open weight models, the disparity in cost may rise to as much as 7x. Through frequent compaction, we show across a broad range of long-horizon and long-input tasks that frontier language models sacrifice little to no task accuracy when maintaining short contexts. We further demonstrate that LLM performance does not collapse on long-horizon software engineering tasks even when taking at most one action per context, and based on that we outline the concept of semantic state space models (SSSMs). Our findings question the need for long-context language models.

Community

00

Publication notes

Author note
Technical report. 31 pages, 8 figures