From Weeks to Hours: Fast and Principled SFT Data Generation for Reasoning LLMs
Abstract
Reasoning-oriented post-training enables large language models (LLMs) to solve complex tasks via multi-step inference. Supervised fine-tuning (SFT) on high-quality reasoning traces is a particularly efficient approach, but its effectiveness depends critically on careful data curation. Existing pipelines rely on a costly generate-then-filter paradigm: large pools of long reasoning traces are produced by multiple teacher models and subsequently filtered for difficulty and diversity using additional LLMs, often requiring weeks of computation. In this work, we propose SFTProbe, a fundamentally different approach that bypasses this pipeline by directly predicting the quality of reasoning data. We show that a brief LoRA-based adaptation, combined with evaluating loss on only 5% of each reasoning trace, suffices to estimate both difficulty and diversity. SFTProbe is grounded in a geometric view of post-training, where pretrained LLMs lie in a low-rank, anisotropic loss basin. By probing this basin via low-rank perturbations, we construct compact loss signatures whose clustering captures diversity, while loss at curvature inflection points provides a robust proxy for difficulty. We also provide a simple criterion for teacher selection. Empirically, our approach reduces data curation time from weeks to hours while maintaining or improving performance. Remarkably, on mathematical reasoning, 5k generated examples match the highly filtered 89k-example in OpenThoughts-Math dataset, and 7.5k generated examples outperforms it by 2.5% in accuracy. We further introduce a high-quality physics reasoning dataset, and use it to train a model that outperforms an IPhO 2025 silver-medal-winning model. The code and the dataset are released at https://bigml-cs-ucla.github.io/SFTProbe-page/.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
SFTProbe introduces a principled, geometric approach to SFT data curation that significantly reduces computational costs while maintaining high performance in reasoning tasks. By leveraging low-rank perturbations to estimate data difficulty and diversity, the method offers a scalable alternative to expensive LLM-based filtering, with strong empirical validation on mathematical and physics reasoning benchmarks.
Methodology
The paper proposes SFTProbe, a method to bypass the standard "generate-then-filter" paradigm for SFT data curation. Instead of using LLMs to judge quality, it utilizes a brief LoRA-based adaptation on a small subset of data to predict the quality of the full dataset. The core theoretical contribution is a geometric view of the loss landscape, arguing that pretrained LLMs reside in a low-rank, anisotropic basin. By probing this basin with low-rank perturbations, the authors construct "loss signatures" where clustering indicates diversity and loss at curvature inflection points indicates difficulty. This is a novel shift from heuristic filtering to a principled, optimization-based estimation of data utility.
Results & evaluation
The empirical results are strong and directly address the efficiency bottleneck of current pipelines. The claim that 5k generated examples match the performance of 89k highly filtered examples from OpenThoughts-Math is significant, suggesting a massive reduction in compute requirements for data generation. Furthermore, the introduction of a high-quality physics reasoning dataset and the training of a model that outperforms an IPhO 2025 silver-medal-winning model provides concrete evidence of the method's effectiveness in complex reasoning domains. The reduction of curation time from weeks to hours is a practical win for the community.
Reproducibility
The paper states that code and datasets are released at the provided URL. The methodology relies on standard techniques (LoRA, loss evaluation) combined with specific probing strategies. While the geometric theory is abstract, the practical implementation appears straightforward enough for reproduction given the released artifacts. The specific hyperparameters for the "brief LoRA-based adaptation" and the definition of "5% of each reasoning trace" are critical details that must be present in the code release for full reproducibility.
Limitations
The primary limitation is the reliance on the geometric assumption of a "low-rank, anisotropic loss basin." If this assumption fails for certain model architectures or domains, the probing method may lose efficacy. Additionally, the comparison is primarily against the OpenThoughts-Math dataset; broader validation across diverse reasoning tasks (e.g., coding, general QA) would strengthen the claim. The "weeks to hours" claim depends heavily on the scale of the original pipeline being compared against, which may vary across labs.
Broader impact
This work has high potential for adoption in the LLM post-training community. As reasoning models become the standard for complex tasks, the cost of data curation is a major bottleneck. A method that reduces this cost by an order of magnitude while maintaining or improving quality is highly valuable. The geometric perspective on loss landscapes for data selection could also inspire further research into the interplay between model optimization and data quality. SFTProbe introduces a principled, geometric approach to SFT data curation that significantly reduces computational costs while maintaining high performance in reasoning tasks. By leveraging low-rank perturbations to estimate data difficulty and diversity, the method offers a scalable alternative to expensive LLM-based filtering, with strong empirical validation on mathematical and physics reasoning benchmarks.
Author of this paper? Send a correction.