MEGA Hub

FrontierPhysics: Evaluating Agents for End-to-End Physics Research

Authors

Do you know Bingran You?You can claim authorship or link another user.Do you know Wenjun Ke?You can claim authorship or link another user.Do you know Ria Rosenauer?You can claim authorship or link another user.Do you know Bo Liu?You can claim authorship or link another user.Do you know Wenbo Chen?You can claim authorship or link another user.Do you know Min Chen?You can claim authorship or link another user.Do you know Shuqi Xu?You can claim authorship or link another user.Do you know Yue Ma?You can claim authorship or link another user.Do you know Jiechao Feng?You can claim authorship or link another user.Do you know Bo Chen?You can claim authorship or link another user.Do you know Xiangkai Sun?You can claim authorship or link another user.Do you know Jiarui Liu?You can claim authorship or link another user.Do you know Jinen Guo?You can claim authorship or link another user.Do you know Zeyu Zhou?You can claim authorship or link another user.Do you know Xu Huang?You can claim authorship or link another user.Do you know Zhuohan Li?You can claim authorship or link another user.Do you know Haoyu Li?You can claim authorship or link another user.Do you know Heguang Lin?You can claim authorship or link another user.Do you know Lijie Ding?You can claim authorship or link another user.Do you know Yadong Zeng?You can claim authorship or link another user.Do you know Kangqi Zhang?You can claim authorship or link another user.Do you know Xinge Liu?You can claim authorship or link another user.Do you know Hanwen Liu?You can claim authorship or link another user.Do you know Zhenghong Li?You can claim authorship or link another user.Do you know Alexander von Recum?You can claim authorship or link another user.Do you know Yuchen Zhu?You can claim authorship or link another user.Do you know Felix Schwarzfischer?You can claim authorship or link another user.Do you know Yizhou Zhao?You can claim authorship or link another user.Do you know Jicheng Wang?You can claim authorship or link another user.Do you know Binxu Li?You can claim authorship or link another user.Do you know Qian Yu?You can claim authorship or link another user.Do you know Chan Li?You can claim authorship or link another user.Do you know Cheng Dai?You can claim authorship or link another user.Do you know Di Wang?You can claim authorship or link another user.Do you know Haoyu Zhang?You can claim authorship or link another user.Do you know Jiankai Sun?You can claim authorship or link another user.Do you know Kaituo Feng?You can claim authorship or link another user.Do you know Mian Qin?You can claim authorship or link another user.Do you know Ruihong Yin?You can claim authorship or link another user.Do you know Yiqun Luo?You can claim authorship or link another user.Do you know Weiwei Wu?You can claim authorship or link another user.Do you know Shihong Pan?You can claim authorship or link another user.Do you know Tsung Yeh Hsieh?You can claim authorship or link another user.Do you know Xiao Huang?You can claim authorship or link another user.Do you know Yan Zhang?You can claim authorship or link another user.Do you know Sihan Li?You can claim authorship or link another user.Do you know Zirui Ren?You can claim authorship or link another user.Do you know Zengji Tu?You can claim authorship or link another user.Do you know Kyoung Whan Choe?You can claim authorship or link another user.Do you know Ke Sun?You can claim authorship or link another user.Do you know Vincenzo Galia?You can claim authorship or link another user.Do you know Vaisakhi Mishra?You can claim authorship or link another user.Do you know Jiashu He?You can claim authorship or link another user.Do you know Hongyang Wang?You can claim authorship or link another user.Do you know Zhiheng Yang?You can claim authorship or link another user.Do you know Shubham Gaur?You can claim authorship or link another user.Do you know Yifan Gu?You can claim authorship or link another user.Do you know Shuyi Wang?You can claim authorship or link another user.Do you know Zihao Zuo?You can claim authorship or link another user.Do you know Steven Dillmann?You can claim authorship or link another user.Do you know Yiyou Sun?You can claim authorship or link another user.Do you know Xinyang Han?You can claim authorship or link another user.Do you know Hanchung Lee?You can claim authorship or link another user.Do you know Molei Tao?You can claim authorship or link another user.Do you know Yue Zhao?You can claim authorship or link another user.Do you know Shilong Liu?You can claim authorship or link another user.Do you know Yuchen Wu?You can claim authorship or link another user.Do you know Xuandong Zhao?You can claim authorship or link another user.Do you know Xiangyi Li?You can claim authorship or link another user.Do you know Florian Brand?You can claim authorship or link another user.Do you know Gerbrand Ceder?You can claim authorship or link another user.Do you know Hartmut Haeffner?You can claim authorship or link another user.

Abstract

AI agents are playing an increasingly important role in physics research. However, existing physics benchmarks only evaluate fixed questions, isolated sub-tasks, or paper reproduction with given solutions, and rarely check research quality alongside correctness. We introduce FrontierPhysics, an end-to-end benchmark with 56 tasks made by 42 physicists based on their own research, covering nine domains across theoretical, computational, and experimental physics. Each task requires agents to review the literature, develop a research plan, conduct research and experiments, and produce structured outputs and scientific manuscripts. Combining deterministic verifiers with expert-authored rubrics, FrontierPhysics evaluates whether agents' research is correct and publishable. Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts. Manual review of agent trajectories shows that frontier agents' main weaknesses are insufficiently rigorous reasoning and limited expert intuition, rather than gaps in physics knowledge or execution errors.

Community

00

Publication notes

Author note
41 pages (10 pages main text), 19 figures, 11 tables