subinium / deep-papers Goto Github PK

• https://arxiv.org/abs/2005.12872
• DETR

• 읽기에 앞서 기존 object detection 방법을 먼저 알고 보는 게 좋을 것 같아서 (오디알못...)
• Jusin Johnson : Lecture 15: Object Detection
  • 기존 방법론(1-stage, 2-stage), 용어(RoI, IoU, NMS, mAP), 한계점을 살펴보고 가자.
  • RCNN
  • Fast R-CNN
  • Faster R-CNN
  • 위 과정이 귀찮다면 Incredible.AI 블로그가 쉽게 설명해주고 있음.

• https://arxiv.org/abs/2007.10379
• GH-Feat

• https://papers.nips.cc/paper/2018/file/69386f6bb1dfed68692a24c8686939b9-Paper.pdf
• Neural ODE

• https://dl.acm.org/doi/10.1145/3290605.3300831

• https://arxiv.org/abs/2112.10741
• GLIDE

• https://arxiv.org/pdf/1905.01164.pdf
• SinGAN
• https://github.com/FriedRonaldo/SinGAN

• https://arxiv.org/abs/1802.05365
• a.k.a ELMo

• https://arxiv.org/abs/1812.04948
• StyleGAN

• https://arxiv.org/abs/2006.11371
• XAI Survey

• https://arxiv.org/abs/1805.08318
• SAGAN

• https://arxiv.org/abs/1907.11692
• RoBERTa

• https://arxiv.org/pdf/1912.09024.pdf
• SCS

• https://arxiv.org/abs/1511.05644
• Adversarial Autoencoders

• https://arxiv.org/abs/2003.10555
• ELECTRA

• https://arxiv.org/abs/1909.11942
• ALBERT

• https://arxiv.org/abs/1611.07004
• pix2pix

• https://arxiv.org/abs/1810.04805
• BERT

• https://arxiv.org/abs/1804.07461
• GLUE

• https://arxiv.org/pdf/1706.03762.pdf
• Transformer
• Seq2Seq Model

Concept

Structure

• Multi-head self-attention
  • Scaled Dot-Product Attention
• Residual connection
• Layer nomalization
• Masked
• Positional Encoding

Optimizer

• Adam (warm up steps = 4000)

Regularization

• Residual Dropout
• Label Smoothing

• https://arxiv.org/abs/2004.02546
• GANSpace

• https://arxiv.org/pdf/1801.00631.pdf

• https://arxiv.org/abs/1907.07171

• https://arxiv.org/abs/1806.00069

• https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0130140
• LRP

• https://arxiv.org/abs/2002.05709
• SimCLR

• https://arxiv.org/abs/1710.10196
• PGGAN

• https://arxiv.org/abs/1911.05722
• MoCo

• https://arxiv.org/abs/1610.02391

• https://arxiv.org/abs/1907.10830
• U-GAT-IT

• https://arxiv.org/pdf/2102.04803.pdf
• DetCo

• https://arxiv.org/pdf/1811.11839.pdf
• https://github.com/SinaMohseni/Awesome-XAI-Evaluation

• https://arxiv.org/abs/2010.11929
• ViT
• 오픈리뷰에 있었지만 스케일을 보고 모두가 구글이라고 예측한 논문.

• https://arxiv.org/pdf/1509.06321.pdf

• https://arxiv.org/abs/1809.11096
• BigGAN

• https://arxiv.org/abs/1703.10593
• CycleGAN

• https://arxiv.org/abs/2007.00653
• NeurlPS 2020
• Swapping Autoencoder

• https://github.com/NVlabs/stylegan3
• Alias-Free GAN (StyleGAN3)
• NeurIPS 2021
• https://nvlabs-fi-cdn.nvidia.com/stylegan3/stylegan3-paper.pdf

• https://arxiv.org/pdf/1906.08237.pdf
• XLNet
• Transformer-XL의 확장이라고 함
  • 이 친구는 리젝되었지만 성능이 좋아 허깅페이스, 구글 블로그에 올라옴 (저자 같음. 리젝 후 XLNET으로 업그레이드)
  • PR12 박성남님 발표로 살펴보기
  • transformer 문제점
    • long-term dependency (고정 길이라 긴 길이 불가)
    • context fragmentations (이전 시퀀스에 대한 문맥 정보 전달 불가)
  • 해결책
    • Segment-level Recurrence (cache로 다음 시퀀스에게 정보 전달)
      • 메모리만 가능하다면 이전 시퀀스 정보 사용 가능
    • Relative Positional Embeddings
      • 연속적인 시퀀스 학습을 할 수 없기 때문에 기존 positional embedding 불가
  • autoregressive + receptive field로 이해하면 될 듯.
  • evaluation : PPL, BPC, RECL
  • limit : downstream task에서 안해봄, documentation generation 결과 없음, GPT2가 이김

• https://arxiv.org/abs/1701.07875
• WGAN

• https://arxiv.org/abs/1602.04938
• LIME

• https://arxiv.org/abs/1912.04958
• StyleGAN2

• https://arxiv.org/abs/1908.07442
• Tabnet