MaxViT: Multi-Axis Vision Transformer (ECCV 2022)

⚠️ DISCLAIMER: This implementation is still under development.

[TOC]

MaxViT is a family of hybrid (CNN + ViT) vision backbone models, that achieves better performances across the board for both parameter and FLOPs efficiency than both state-of-the-art ConvNets and Transformers (Blog). They can also scale well on large dataset sizes like ImageNet-21K. Notably, due to the linear-complexity of the grid attention used, MaxViT scales well on tasks requiring large image sizes, such as object detection and segmentation.

MaxViT meta-architecture: a homogeneously stacked backbone, wherein each MaxViT block contains MBConv, block attention (window-based local attention), and grid attention (dilated global attention).

Results on ImageNet-1k standard train and test:

Results on ImageNet-21k and JFT pre-trained models:

Model Performance

Note: Deit ImageNet pretrain experimental settings are different from the paper. These experiments follows the pre-training hyperparameters in paper and only run pre-training for similar number of steps. The paper suggested a short fine-tuning with different hyper-parameters and EMA.

Deit ImageNet pretrain {.new-tab}

Cascade RCNN models {.new-tab}

JFT-300M supervised pretrain {.new-tab}

ImageNet Finetuning {.new-tab}

Cascade RCNN models {.new-tab}

Citation

Should you find this repository useful, please consider citing:

@article{tu2022maxvit,
  title={MaxViT: Multi-Axis Vision Transformer},
  author={Tu, Zhengzhong and Talebi, Hossein and Zhang, Han and Yang, Feng and Milanfar, Peyman and Bovik, Alan and Li, Yinxiao},
  journal={ECCV},
  year={2022},
}