Search2Skill introduces an innovative framework for skill distillation that transcends the limitations of traditional LLM-based agents. This approach seeks to enhance autonomous skill acquisition through a rubric-based reinforcement learning strategy, enabling agents to not only identify their capability gaps but also effectively search external sources to bridge these gaps. By doing so, Search2Skill distills valuable information into structured and reusable skills applicable in real-world scenarios.
The framework addresses a significant challenge in the development of self-evolving models. Traditional methods of skill acquisition often rely on the pre-existing parametric knowledge of the model, restricting the scope of learned skills to what is already known. In contrast, Search2Skill enables the acquisition of procedural knowledge essential for various expert domains, which may not be readily available or easily inferred from the model's current capabilities. The methodology proves robust through experiments conducted across eight expert-level domains derived from three benchmarks, consistently outperforming existing search-augmented and trajectory-based skill learning techniques.
The optimization of this framework hinges on a systematic rubric-based reinforcement learning approach, enhancing the processes of determining when and how to search for external information, alongside effective skill generation. Notably, the benefits from Search2Skill arise from the abstraction of skills rather than relying solely on raw evidence retrieval. Furthermore, the skills acquired through this framework exhibit transferability across different model scales, suggesting a versatile application across varied AI systems.