projects/ViTDet/README.md
Yanghao Li, Hanzi Mao, Ross Girshick†, Kaiming He†
In this repository, we provide configs and models in Detectron2 for ViTDet as well as MViTv2 and Swin backbones with our implementation and settings as described in ViTDet paper.
Note: Unlike the system-level comparisons in the paper, these models use a lower resolution (1024 instead of 1280) and standard NMS (instead of soft NMS). As a result, they have slightly lower box and mask AP.
We observed higher variance on LVIS evalution results compared to COCO. For example, the standard deviations of box AP and mask AP were 0.30% (compared to 0.10% on COCO) when we trained ViTDet, ViT-B five times with varying random seeds.
The above models were trained and measured on 8-node with 64 NVIDIA A100 GPUs in total. *: Activation checkpointing is used.
All configs can be trained with:
../../tools/lazyconfig_train_net.py --config-file configs/path/to/config.py
By default, we use 64 GPUs with batch size as 64 for training.
Model evaluation can be done similarly:
../../tools/lazyconfig_train_net.py --config-file configs/path/to/config.py --eval-only train.init_checkpoint=/path/to/model_checkpoint
If you use ViTDet, please use the following BibTeX entry.
@article{li2022exploring,
title={Exploring plain vision transformer backbones for object detection},
author={Li, Yanghao and Mao, Hanzi and Girshick, Ross and He, Kaiming},
journal={arXiv preprint arXiv:2203.16527},
year={2022}
}