A new approach, CRAFTER (Corrective Residual Agent with Feature-based Temporal Exploration and Reasoning), has been proposed to enhance the performance of frozen pretrained forecasters. This method addresses the limitations in traditional fine-tuning, which can be costly and often ineffective for repairing structured, recurring model failures. Instead of adjusting model parameters, CRAFTER identifies and utilizes corrective features from the residual errors of a frozen model, thus maintaining the integrity of the original forecaster while improving its predictive accuracy.
CRAFTER operates using two generators: one that conducts a compositional search over the raw input channels and another that employs a large language model (LLM) to suggest combinations of features, binary flags, and executable code. A unified validation gate system evaluates and accepts or rejects candidates based on their contribution to forecast performance, resulting in a focused corrective mechanism that leaves forecasts unchanged when no significant improvement is identified. This approach allows for an assessment of feature-engineering models under comparable conditions, providing insights into the effectiveness of various feature sources in enhancing model accuracy.
The results from extensive testing on six public datasets using six frozen model backbones demonstrate that CRAFTER outperforms all dedicated feature-engineering systems across varying feature budgets. Notably, it achieves improvements that nearly double the effectiveness of the corrective mechanisms alone, reducing errors in the weakest models by up to 27%. These results are consistent regardless of the LLM backend used and remain effective even when applied to fine-tuned models.