SLIC: Reinforcement Fine-Tuning Small LMs for Multi-Turn Analog Circuit Optimization
Abstract
Analog integrated circuit (IC) design is a multi-turn, simulator-grounded engineering task that demands iterative reasoning over coupled performance tradeoffs, yet proprietary process design kits (PDKs) make hosted large language model workflows difficult to deploy in practical design settings. We present SLIC (Small LM Iterating on Circuits), a reinforcement fine-tuning framework that trains a 4B-scale, locally deployable language model for closed-loop analog circuit optimization. SLIC pairs structured circuit representations with discretized relative-update actions, domain- and success-gated multiobjective rewards, and Reward-Guided Rollout Steering (RGRS), which turns per-step simulator feedback into signals for both policy learning and online rollout construction in this sparse, highly constrained domain. We further release an open, goal-conditioned benchmark of 41 amplifier topologies with standardized protocols for in-distribution validation, held-out topology evaluation, and cross-technology-node transfer. Trained solely with open source 65 nm CMOS predictive technology models (PTM), SLIC-Random achieves 87.1% in-distribution four-specification Pass@1 within eight turns, surpassing the 763B DeepSeek-V4.1-Flash Think by 48.5 percentage points. Beyond the training distribution, SLIC-Softmax reaches 72.6% Pass@1 on held-out topologies, and the trained policies consistently transfer across five unseen technology nodes without additional fine-tuning. This transfer extends to a commercial TSMC 65 nm CMOS PDK, where SLIC-Random achieves 49.9% held-out-topology Pass@1, demonstrating practical deployment through local inference without transmitting proprietary data to external LLM services. Performance further scales with test-time simulator interactions, reaching up to 83.2% held-out-topology Pass@1 at a 32-turn budget.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
SLIC introduces a reinforcement fine-tuning framework that enables a 4B locally deployable language model to outperform a 763B general-purpose model in multi-turn analog circuit optimization by leveraging simulator-grounded rewards and reward-guided rollout steering. The paper makes a strong case for domain-specific small models in high-stakes, data-sensitive engineering applications, providing a rigorous benchmark and a reproducible pipeline that bridges the gap between LLM reasoning and physical simulation constraints.
Methodology
The paper proposes SLIC, a framework for training small language models (4B parameters) to perform multi-turn analog circuit optimization using Reinforcement Learning with Verifiable Rewards (RLVR). The methodology is sound and well-motivated by the constraints of the domain: proprietary PDKs prevent the use of large hosted LLMs, necessitating a small, locally deployable model. The core technical contributions are the formulation of circuit design as a multi-turn RL problem with structured actions (relative parameter updates) and the introduction of Reward-Guided Rollout Steering (RGRS). RGRS is a clever adaptation of group-relative policy optimization (GRPO) to handle sparse, high-dimensional feedback by using simulator results to steer the rollout trajectory and provide dense credit assignment. The reward shaping, which includes domain-validity gates (e.g., checking for saturation) and success-conditional slack bonuses, is physically grounded and addresses the specific failure modes of circuit optimization (e.g., non-functional circuits getting high scores for low power).
Results & evaluation
The experimental evaluation is rigorous and comprehensive. The authors construct a novel benchmark of 41 amplifier topologies, which is a significant contribution in itself given the scarcity of open data in this field. The evaluation covers in-distribution performance, out-of-distribution topology generalization, and cross-technology-node transfer (22-130nm). The results are impressive: the 4B model outperforms a 763B model (DeepSeek-V4.1-Flash) by a large margin (48.5 percentage points) on in-distribution tasks, demonstrating the efficacy of domain-specific RL fine-tuning over general-purpose scaling. The transfer to a commercial TSMC PDK, while showing a performance drop, validates the practical utility of the local deployment approach. Ablations clearly isolate the contributions of the reward gates and RGRS.
Reproducibility
The paper is highly reproducible. The authors release an open benchmark and use open-source base models (Qwen3.5-4B) and simulation tools (ngspice, PTM models). The training configuration (39 hours on 2x B200 GPUs) is clearly stated. The use of standard RL algorithms (GRPO/PPO variants) with specific hyperparameters provided in the appendix ensures that other researchers can replicate the results. The project page is provided.
Limitations
The primary limitation is the scope of the task: it focuses on amplifier topologies, which are a subset of analog circuits. While the framework is general, the specific action space and reward functions are tailored to amplifiers. The performance drop on the commercial PDK (49.9% vs 87.1%) highlights the gap between predictive models (PTM) and real foundry models, suggesting that while the method is promising, it may require fine-tuning or more robust generalization strategies for full industrial adoption without any adaptation. The reliance on simulator feedback means the method is computationally intensive during training, though the test-time scaling is efficient.
Broader impact
This work has significant potential impact on the EDA (Electronic Design Automation) industry. By demonstrating that small, local LLMs can outperform massive general-purpose models in specialized engineering tasks, it challenges the "bigger is better" paradigm in LLMs. It enables secure, on-premise AI assistance for circuit design, which is critical for companies with strict IP protection policies. The open benchmark will likely become a standard testbed for future AI-EDA research. SLIC introduces a reinforcement fine-tuning framework that enables a 4B locally deployable language model to outperform a 763B general-purpose model in multi-turn analog circuit optimization by leveraging simulator-grounded rewards and reward-guided rollout steering. The paper makes a strong case for domain-specific small models in high-stakes, data-sensitive engineering applications, providing a rigorous benchmark and a reproducible pipeline that bridges the gap between LLM reasoning and physical simulation constraints.
Author of this paper? Send a correction.