transformer_pipeline

• Attention is all you need
• ViT: AN IMAGE IS WORTH 16X16 WORDS: TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE
• Swin: Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
• CSwin: CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows
• DETR: End-to-end Object Detection with Transformers
• iRPE: Rethinking and Improving Relative Position Encoding for Vision Transformer
• DAT: Vision Transformer with Deformable Attention
• CvT: CvT: Introducing Convolutions to Vision Transformers
• CrossViT: CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification
• SwinTrack: SwinTrack: A Simple and Strong Baseline for Transformer Tracking
• Stark: Learning Spatio-Temporal Transformer for Visual Tracking
• [] Swin-V2: coming soon

Performance comparisons on ImageNet1K

Attention is all you need

Vision Transformer

SwinTransformer

CSwinTransformer

DETR

iRPE: Rethinking Position Encoding

Deformable Attention Transformer

CvT: Introducing Convolutions to Vision Transformers

CrossViT SwinTrack Stark