In this github repo, I share materials corresponding to two courses that I etach at the University of Calgary:

• ENEL645 - Data Mining & Machine Learning
• ENSF 619.02 - Advanced Image Analysis and Machine Learning

This repo is constantly being updated. Initially, we used TensorFlow as the deep learning, but we will be adding PyTorch examples since it is the dominant deep learning framework at the moment.

For all teamwork assigments, including the final project, your team needs an additional 1-page pdf document as described here.

-Assignment 01

-Assignment 02

-Final Project

Templates for final project:

• Overleaf
• MS Word

Week 0

• Python recap
• NumPy recap
• L: Python Bootcamp
• T: Avoid Loops

Week 01 (09-15 January)

• L: Course Overview
• L: Fundamentals ML
• L: Overfitting and Regularization
• T: Overfitting and Regularization

Week 02 (16-22 January)

• L: Deep Learning Intuition
• T: Softmax, cross-entropy, etc.
• L: Data Normalization (self-study)
• L: Fully Connected Neural Networks
• T: Fully Connected NN: 2D Synthetic Example
• T: Fully Connected NN: Image Classification - PyTorch
• T: Fully Connected NN: Image Classification - PyTorch - explanation of the code in the tutorial above

Week 03 (23-29 January)

• T: Fully Connected NN: Image Classification
• L: Convolutional Neural Networks
• T: Different ways to define NNs
• Quiz: 27 January

Week 04 (30 January - 05 February)

• Assignment 01 due 30 January at midnight
• T: CNN: Image Classification
• T: Fully Connected NN - Revisited
• L: Transfer Learning
• T: Transfer Learning
• Discussion about assignment 01 - Friday class

Week 05 (06-12 February)

• T: Garbage classifier - images

Week 06 (13-19 February)

• T: Garbage classifier - images (SLURM)
• L: Sex Prediction from Brain Images
• Quiz: 17 February

Week 07 (20-26 February)

• Reading week

Week 08 (27 February - 05 March)

• Quiz: 27 February
• T: Garbage Classifier - Transfer Learning on Text
• L: The U-net model
• T: U-net Denoising 1D signals

Week 09 (06-12 March)

• Lectures TBD
• Assignment 02 due 06 March at midnight

Week 10 (13-19 March)

• Lectures TBD

Week 11 (20-26 March)

• Lectures TBD
• Wednesday and Friday - support for finalizing the projects

Weeks 12 and 13 (27 March - 12 April )

• Project report and 10 minute presentation recording due date: 27 March at 9 am
• Final projects' presentations

-Assignment 01

-Assignment 02

-Final Project

Templates for final project:

• Overleaf
• MS Word

Week 0

• Python recap
• NumPy recap
• L: Python Bootcamp
• T: Avoid Loops

Week 01 (09-15 January)

• L: Course Overview
• L: Fundamentals ML
• L: Overfitting and Regularization
• T: Overfitting and Regularization

Week 02 (16-22 January)

• L: Deep Learning Intuition
• L: Fully Connected Neural Networks
• T: Softmax, cross-entropy, etc.
• L: Convolutional Neural Networks
• T: Fully Connected NN: Image Classification
• T: CNN: Image Classification

Week 03 (23-29 January) Monday - paper discussion Paper discussion

[1] K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition.” arXiv, Apr. 10, 2015. Accessed: Dec. 08, 2022. [Online]. Available: http://arxiv.org/abs/1409.1556

[2] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition.” arXiv, Dec. 10, 2015. Accessed: Dec. 08, 2022. [Online]. Available: http://arxiv.org/abs/1512.03385

Presenters: Nisha Mansuri and Yashkumar Trada

• Friday (27 January) - short quiz and paper discussion*

[3] M. Tan and Q. V. Le, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,” p. 10.

Presenters: Anik Das

[4] Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A ConvNet for the 2020s.” arXiv, Mar. 02, 2022. Accessed: Dec. 08, 2022. [Online]. Available: http://arxiv.org/abs/2201.03545

Presenters: Mouri Zakir

Week 04 (30 January - 05 February)

• T: Fully Connected NN: Image Classification - PyTorch
• T: Fully Connected NN: Image Classification - PyTorch - explanation of the code in the tutorial above
• L: Transfer Learning
• T: Transfer Learning

[5] A. Dosovitskiy et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.” arXiv, Jun. 03, 2021. Accessed: Dec. 08, 2022. [Online]. Available: http://arxiv.org/abs/2010.11929

[6] Z. Liu et al., “Swin Transformer: Hierarchical Vision Transformer using Shifted Windows,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, Oct. 2021, pp. 9992–10002. doi: 10.1109/ICCV48922.2021.00986.

Presenters: Hadi Heidarirad and Reet Ghosh

[7] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation.” arXiv, May 18, 2015. Accessed: Jan. 07, 2023. [Online]. Available: http://arxiv.org/abs/1505.04597

[8] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 4, pp. 834–848, Apr. 2018, doi: 10.1109/TPAMI.2017.2699184.

Presenters: Tariq Al Shoura

Week 05 (06-12 February)

• Models imagenet

[9] A. Hatamizadeh et al., “UNETR: Transformers for 3D Medical Image Segmentation,” in 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, Jan. 2022, pp. 1748–1758. doi: 10.1109/WACV51458.2022.00181.

Presenters: Philip Ciunkiewicz

[10] J. Bassey, L. Qian, and X. Li, “A Survey of Complex-Valued Neural Networks.” arXiv, Jan. 28, 2021. Accessed: Dec. 15, 2022. [Online]. Available: http://arxiv.org/abs/2101.12249

Presenters: Natalia Dubljevic and Paula Brandt

Week 06 (13-19 February)

• Monday lecture TBD

[11] A. Diaz-Pinto et al., “MONAI Label: A framework for AI-assisted Interactive Labeling of 3D Medical Images.” arXiv, Mar. 23, 2022. Accessed: Dec. 08, 2022. [Online]. Available: http://arxiv.org/abs/2203.12362

[12] M. J. Cardoso et al., “MONAI: An open-source framework for deep learning in healthcare.” arXiv, Nov. 04, 2022. Accessed: Dec. 08, 2022. [Online]. Available: http://arxiv.org/abs/2211.02701

Presenters: Mahsa Dibaji and Aashka Mohite

[13] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-CAM: Visual Explanations From Deep Networks via Gradient-Based Localization”.

[14] S. Bach, A. Binder, G. Montavon, F. Klauschen, K.-R. Müller, and W. Samek, “On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation,” PLoS ONE, vol. 10, no. 7, p. e0130140, Jul. 2015, doi: 10.1371/journal.pone.0130140.

Presenters: Sepideh Afshar

Week 07 (20-26 February)

• Reading week

Week 08 (27 February - 05 March)

• Quiz: 27 February
• Assignment 01 due 27 February at midnight
• T: GRAD-CAM - Alzheimer's Classification

[15] Y. Ganin et al., “Domain-Adversarial Training of Neural Networks,” in Domain Adaptation in Computer Vision Applications, G. Csurka, Ed. Cham: Springer International Publishing, 2017, pp. 189–209. doi: 10.1007/978-3-319-58347-1_10.

[16] N. K. Dinsdale, M. Jenkinson, and A. I. L. Namburete, “Deep learning-based unlearning of dataset bias for MRI harmonisation and confound removal,” NeuroImage, vol. 228, p. 117689, Mar. 2021, doi: 10.1016/j.neuroimage.2020.117689.

Presenters: Salma Begum Tamanna and Rubya Afrin

[17] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, Jun. 2022, pp. 10674–10685. doi: 10.1109/CVPR52688.2022.01042.

Presenters: Mehregan Biglarbeik

Week 09 (06-12 March)

• Monday lecture TBD

[18] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A Simple Framework for Contrastive Learning of Visual Representations”.

[19] I. Misra and L. van der Maaten, “Self-Supervised Learning of Pretext-Invariant Representations,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, Jun. 2020, pp. 6706–6716. doi: 10.1109/CVPR42600.2020.00674.

[20] X. Liang, L. Lee, W. Dai, and E. P. Xing, “Dual Motion GAN for Future-Flow Embedded Video Prediction,” in 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Oct. 2017, pp. 1762–1770. doi: 10.1109/ICCV.2017.194.

Presenters: Abbas Omidi and Aida Mohammadshahi

Week 10 (13-19 March)

• Monday lecture TBD

[21] Y. Wang, Q. Yao, J. T. Kwok, and L. M. Ni, “Generalizing from a Few Examples: A Survey on Few-shot Learning,” ACM Comput. Surv., vol. 53, no. 3, pp. 1–34, May 2021, doi: 10.1145/3386252.

[22] Y. Xian, C. H. Lampert, B. Schiele, and Z. Akata, “Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, no. 9, pp. 2251–2265, Sep. 2019, doi: 10.1109/TPAMI.2018.2857768.

Presenters: Jose Cazarin and Mohammed Adnan

Week 11 (20-26 March)

• Monday lecture TBD
• Friday - support for finalizing the projects

Weeks 12 and 13 (27 March - 03 April )

• Project report and 10 minute presentation recording due date: 27 March at 9 am
• Final projects' presentations