NextGAN: Towards Simple and Scalable GAN Training for Image and Video Generation
Abstract
Adversarial training enables one- and few-step image and video generation, but conventional adversarial losses supervise critic values without directly specifying the gradients used to update the generator. We present NEXTGAN, which trains a scalar critic by regressing its input gradient onto displacements between independently paired reference and generated features at interpolated locations in frozen representation spaces. For a fixed generator, this regression is equivalent to an adversarial objective with a gradient-energy penalty, while the generator retains the standard potential-based GAN update. The same objective adapts pre-trained image and video generators with up to 14B parameters, without auxiliary losses, online teacher scores, or trajectories. On ImageNet $256 \times 256$, NEXTGAN achieves one-step FIDs of 2.87 with pixel-space JiT-B and 1.54 with latent-space DiT-XL/1; its adversarial form further reaches 1.36. For text-to-video generation, a single trained Wan2.1-1.3B model supports one-, two-, and four-step sampling, achieving the highest VBench total scores among the compared 1.3B methods at one and two steps. At two steps, NEXTGAN scores 83.13, exceeding rCM and AnyFlow by 1.25 and 0.47 points, respectively. Code is available at https://github.com/LINs-lab/NextGAN.git.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
The paper introduces NEXTGAN, a simplified adversarial training objective that regresses critic gradients onto feature displacements, enabling efficient one-step image and video generation with competitive performance on ImageNet and VBench benchmarks.
Methodology
The paper proposes NEXTGAN, a training objective for Generative Adversarial Networks (GANs) that addresses the instability and complexity of conventional adversarial training. The core innovation is regressing the input gradient of a scalar critic onto displacements between independently paired reference and generated features in a frozen representation space. The authors argue that for a fixed generator, this regression is equivalent to an adversarial objective with a gradient-energy penalty. This approach allows for the adaptation of pre-trained image and video generators (up to 14B parameters) without auxiliary losses, online teacher scores, or complex trajectories. The method is presented as a simplification of the adversarial process, enabling one-step or few-step generation.
Results & evaluation
The experimental results are strong, particularly for image generation. On ImageNet 256x256, the method achieves one-step FIDs of 2.87 (pixel-space JiT-B) and 1.54 (latent-space DiT-XL/1), with an adversarial form reaching 1.36. These are competitive numbers, especially given the "one-step" constraint which is typically difficult to achieve with high fidelity. For text-to-video generation using Wan2.1-1.3B, the method supports 1, 2, and 4-step sampling. It claims the highest VBench total scores among compared 1.3B methods at one and two steps, specifically scoring 83.13 at two steps, outperforming rCM and AnyFlow. The evaluation covers both static images and dynamic video, demonstrating versatility.
Reproducibility
The paper provides a GitHub repository link (https://github.com/LINs-lab/NextGAN.git), which is a positive indicator for reproducibility. The use of standard benchmarks (ImageNet, VBench) and established baseline models (DiT, Wan2.1) further aids in reproducibility. However, the specific details of the "frozen representation spaces" and the pairing mechanism for features would require careful implementation to replicate the exact results, especially given the scale of the models used (14B parameters).
Limitations
The primary limitation is the reliance on pre-trained generators and frozen representation spaces, which may limit the method's applicability to novel architectures that do not have such pre-trained backbones. Additionally, the computational cost of training on 14B parameter models is significant, though the method claims to be simpler than conventional adversarial training. The paper does not extensively discuss the failure modes of the gradient regression approach or how it handles mode collapse compared to traditional GANs.
Broader impact
This work has significant potential to simplify the training of high-fidelity generative models. By enabling one-step generation with competitive FID scores, it could reduce the inference cost of GAN-based models, making them more practical for real-time applications. The adaptation to video generation is particularly impactful, as video GANs are notoriously difficult to train. If the method scales well and is robust, it could become a standard technique for fine-tuning or adapting large-scale pre-trained generative models. The paper introduces NEXTGAN, a simplified adversarial training objective that regresses critic gradients onto feature displacements, enabling efficient one-step image and video generation with competitive performance on ImageNet and VBench benchmarks.
Author of this paper? Send a correction.