---
title: FiT: Flexible Vision Transformer for Diffusion Model
url: https://www.ml-quant.com/papers/arxiv/2402.12376/
site: ML-Quant (https://www.ml-quant.com)
updated: 2026-09-26
license: Summaries CC BY 4.0; links go to the original sources
index: https://www.ml-quant.com/llms.txt
identifier: arXiv:2402.12376
source_url: https://arxiv.org/abs/2402.12376
featured: 2024-10-03
citations: 95
topic: ML & AI Methods
---


# FiT: Flexible Vision Transformer for Diffusion Model

Improved Vision Transformer: The Flexible Vision Transformer (FiTv2) is presented, a design that generates images with unrestricted resolutions and aspect ratios, showing excellent performance across various resolutions.

- Source: https://arxiv.org/abs/2402.12376
- Identifier: arXiv:2402.12376
- Released: 2024-02-19
- First featured: Quant Letter No. 68 (2024-10-03): https://www.ml-quant.com/issues/2024-10-03/
- Citations (Semantic Scholar): 95
- Published in: International Conference on Machine Learning
- Topic: ML & AI Methods

## Related

- [Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models](https://www.ml-quant.com/papers/arxiv/2501.01423/): The paper proposes a new model, VA-VAE, that aligns the latent space with pre-trained vision foundation models, enabling faster convergence of Diffusion Transformers in high-dimensional latent spaces and achieving top performance on ImageNet 256x256 generation.
- [xGen-VideoSyn-1: High-fidelity Text-to-Video Synthesis with Compressed Representations](https://www.ml-quant.com/papers/arxiv/2408.12590/): Video Synthesis: xGen-VideoSyn-1 is a text-to-video generation model that creates realistic scenes from text descriptions using a video variational autoencoder and a Diffusion Transformer model.
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://www.ml-quant.com/papers/arxiv/2312.00752/): Sequence Modeling: Mamba, a neural network architecture that doesn't use attention or MLP blocks, provides faster inference and better performance in language, audio, and genomics than Transformers.
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://www.ml-quant.com/papers/arxiv/2405.21060/): The research identifies a link between state-space models and Transformers in deep learning, leading to the creation of a faster language modeling architecture, Mamba-2.
- [Octo: An Open-Source Generalist Robot Policy](https://www.ml-quant.com/papers/arxiv/2405.12213/): Octo is a large transformer-based policy for robotic manipulation, trained on a vast dataset, that can be instructed via language or images and adapted to new domains.
- [Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction](https://www.ml-quant.com/papers/arxiv/2404.02905/): The article discusses Visual AutoRegressive modeling (VAR), a new image learning method that outperforms diffusion transformers in terms of speed, image quality, and scalability.