MEGA Hub

MDSKILL: Learning to Internalize Skill Memory through Multi-View Distillation for Efficient Dynamic Agent Self-Evolution

Authors

Do you know Dian Jin?You can claim authorship or link another user.Do you know Jinluan Yang?You can claim authorship or link another user.Do you know Kairong Han?You can claim authorship or link another user.Do you know Baohong Li?You can claim authorship or link another user.Do you know Zijing Hu?You can claim authorship or link another user.Do you know Yi He?You can claim authorship or link another user.Do you know Wenlin Zhong?You can claim authorship or link another user.Do you know Yunze Tong?You can claim authorship or link another user.Do you know Minqin Zhu?You can claim authorship or link another user.Do you know Linchuan Li?You can claim authorship or link another user.Do you know Yanan Niu?You can claim authorship or link another user.Do you know Kun Kuang?You can claim authorship or link another user.

Abstract

Skill memories encode past experiences as textual prompts to improve reinforcement learning (RL) for self-evolving agents. Complementing RL, On-Policy Self-Distillation (OPSD) provides dense, fine-grained supervision to internalize knowledge from skill-guided trajectories and accelerate policy learning. However, self-generated memories often contain noisy supervision and functionally entangled, redundant, or conflicting knowledge, making direct distillation difficult. Under limited training budgets, these issues can hinder knowledge internalization and even degrade performance. To address this problem, we propose MDSKILL, a multi-view distillation approach for skill memory internalization. MDSKILL organizes skill memories into three complementary functional views (state grounding, action planning, and execution control) and selects high-quality skills using dynamic metrics to construct view-specific and integrated reasoning branches. OPSD then transfers and consolidates complementary experiential knowledge across these branches into a shared policy, enabling efficient skill internalization and continuous agent self-evolution. We evaluate MDSKILL on three agent benchmarks against RL, skill-augmented, and self-distillation baselines that do not rely on stronger external teacher models. Without additional computational overhead, MDSKILL improves upon SOTA performance by up to 7.1% and 8.1% on ALFWorld and WebShop. Extensive ablations and evolution analyses further validate its effectiveness in knowledge internalization and policy improvement. The code is available at https://anonymous.4open.science/r/MDSKILL-CF53.

Community

00