Code and assignment repository for the Imperial College Mathematics department Deep Learning course

Deep Learning is a fast-evolving field in artificial intelligence that has been driving breakthrough advances in many application areas in recent years. It has become one of the most in-demand skillsets in machine learning and AI, far exceeding the supply of people with an expertise in this field. This course is aimed at PhD students within the Mathematics department at Imperial College who have no prior knowledge or experience of the field. It will cover the foundations of Deep Learning, including the various types of neural networks used for supervised and unsupervised learning. Practical tutorials in TensorFlow/PyTorch are an integral part of the course, and will enable students to build and train their own deep neural networks for a range of applications. The course also aims to describe the current state-of-the-art in various areas of Deep Learning, theoretical underpinnings and outstanding problems.

Topics covered in this course will include:

• Convolutional and recurrent neural networks
• Reinforcement Learning
• Generative Adversarial Networks (GANs)
• Variational autoencoders (VAEs)
• Theoretical foundations of Deep Learning

There is a course website where registrations can be made and further logistical details can be found here.

Kevin obtained his PhD in 2003 from the Department of Mathematics at Imperial College, in the area of dynamical systems. He has also held postdoctorate positions at Imperial College, and was awarded a Marie Curie Individual Fellowship, which he spent at the Potsdam Institute for Climate Impact Modelling in Germany. During these positions his research interests became more focused on machine learning, and specifically adapting ML technologies for numerical analysis problems in dynamical systems. He was the Head of Research at the London music AI startup Jukedeck, where he oversaw the development of the deep learning framework for automatic music composition. In 2018 he set up his own machine learning consultancy, FeedForward, with a focus on the music & the creative industries. His particular interest in the field of deep learning is generative modelling. @kn_webster / [email protected]

Pierre is currently researching his PhD in deep reinforcement learning at the Data Science Institute of Imperial College. He also helps run the Deep Learning Network and organize thematic reading groups there. Prior to that, he has worked in electronics as a research engineer and in quantitative finance as a trader. He has studied electrical engineering at ENST, probability theory and stochastic processes at Universite Paris VI - Ecole Polytechnique, and business management at HEC. His other research interests in the field of deep learning include neural network theory, as well as stochastic optimization methods. @KloudStrife / [email protected]

We are grateful to Kai Arulkumaran for providing PyTorch notebooks for the course and teaching two of the demonstration classes on PyTorch.

Kai is currently researching his PhD in deep learning at the Department of Bioengineering at Imperial College. During his PhD he has been a research intern at Microsoft Research, Twitter Magic Pony, Facebook AI Research and DeepMind. He also founded the Deep Learning Network at Imperial College to organise guest lectures and a reading group on the topic of deep learning. He is an advocate for open-source software and a well-known contributor to the Torch/PyTorch ecosystems. Before his PhD he studied computer science at the University of Cambridge and worked as a web developer. @KaiLashArul / [email protected]

This repository contains the notebooks for the TensorFlow/PyTorch tutorials as well as details for the coursework, for students that wish to take this course for credit.

Students are recommended to fork this repository and add their solutions to the assignments (as python scripts) in their forked repository. The coursework will be assessed orally following completion of the course.

To complete the coursework and run the notebooks you will need to install Tensorflow and PyTorch (as well as other scientific packages, especially numpy). These can be installed using pip; alternatively Tensorflow/PyTorch can be installed using Anaconda (preferred for PyTorch). Jupyter is conveniently installed with Anaconda, or it can also be installed using pip. Relevant links are given below:

• Installing anaconda: https://www.anaconda.com/download/
• Installing Jupyter: https://jupyter.readthedocs.io/en/latest/install.html
• Installing Tensorflow via Anaconda: https://www.anaconda.com/blog/developer-blog/tensorflow-in-anaconda/
• Installing Tensorflow using pip: https://www.tensorflow.org/install/
• Installing PyTorch via Anaconda/pip: https://pytorch.org/get-started/locally/
• Installing OpenAI Gym using pip (Anaconda install unavailable): https://gym.openai.com/docs/