Each week, the Hugging Face team has a science day where one team member presents an awesome NLP paper. We've decided to share this discussion with the community. See planned future discussions below.

• Topic: Overview of recent work on: Indexing and Retrieval for Open Domain Question Answering
• Presenter: Yacine Jernite
• Discussion: Slides

• Paper: Scaling Laws for Neural Language Models
• Authors: Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, Dario Amodei
• Presenter: Teven Le Scao
• Discussion: Google doc paper tutorial

• Paper: Representation Learning with Contrastive Predictive Coding
• Authors: Aaron van den Oord, Yazhe Li, Oriol Vinyals
• Presenter Patrick von Platen
• Discussion: Slides

• Paper: Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
• Authors: R. Thomas McCoy, Ellie Pavlick, Tal Linzen
• Presenter: Victor Sanh
• Discussion: Slides

• Paper: REALM: Retrieval-Augmented Language Model Pre-Training
• Authors: Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, Ming-Wei Chang
• Presenter: Joe Davison
• Discussion: Write-up

• Paper: Adaptively Sparse Transformers
• Authors: Gonçalo M. Correia, Vlad Niculae, André F.T. Martins
• Presenter: Sasha Rush
• Discussion: Colab notebook

• Paper: Compositionality decomposed: how do neural networks generalise?
• Authors: Dieuwke Hupkes, Verna Dankers, Mathijs Mul, Elia Bruni
• Presenter: Thomas Wolf