MEGA Hub

MMSearch-OPD: Easy Made Hard, Hard Made Learnable for Long-Horizon Multimodal Search Agents

Authors

Do you know Huanyao Zhang?You can claim authorship or link another user.Do you know Jiepeng Zhou?You can claim authorship or link another user.Do you know Jiaye Lin?You can claim authorship or link another user.Do you know Ontai Tang?You can claim authorship or link another user.Do you know Jingtian Hao?You can claim authorship or link another user.Do you know Mugeng Liu?You can claim authorship or link another user.Do you know Ru Peng?You can claim authorship or link another user.Do you know Lang Mei?You can claim authorship or link another user.Do you know Xiaohan Yu?You can claim authorship or link another user.Do you know Liyan Liu?You can claim authorship or link another user.Do you know Chong Chen?You can claim authorship or link another user.Do you know Wentao Zhang?You can claim authorship or link another user.

Abstract

Multimodal deep search is moving from short queries to long-horizon investigations that continually acquire evidence. However, training such agents remains constrained by limited challenging training questions and insufficient process feedback. We introduce MMSearch-OPD, combining controllable data refinement with joint reinforcement learning (RL) and on-policy distillation (OPD). Our Refine pipeline performs targeted expansion of reasoning chains derived from existing simple multimodal questions. Parameterized transformations control search depth and cross-modal evidence requirements while reusing existing image--question pairs. For training, answer rewards and lightweight behavior penalties guide RL, while the reward gap modulates token-level teacher guidance on imperfect student trajectories. An exponentially decaying distillation coefficient reduces teacher dependence as training proceeds. Experiments show gains from data refinement and further improvements from joint training after SFT, indicating stronger long-horizon multimodal deep search capabilities.

Community

00