A deep learning primer

A collection of topics in the area of deep learning

These set of notes started as a three-day mini-course that I delivered at the Institute of Space Technology AI Lecture Series in September 2022. The goal of the course was to introduce students to techniques, methods, and practices needed to start working on deep learning. The course has two objectives: 1) introduce students to theoretical concepts in deep learning, including autoencoders, tSNE for data visualization, visual object detection, and recurrent neural networks; and 2) provide hands on training on how to develop a deep learning system using Python+PyTorch ecosystem. The selected topics provide an opportunity to discuss common concepts, such as unsupervised learning and generative models, deep features, techniques for understanding the inner working of deep networks, and sequence modeling.

Faisal Qureshi

Email: faisal.qureshi@ontariotechu.ca
Web: http://vclab.science.ontariotechu.ca

Forward

I have collected a set of topics that I cover in my upper year and graduate courses at Ontario Tech U.

Notes

Convolutional networks for object detection
  • A series of lectures on convolutional networks for object detection.
    • A brief history of neural networks
    • Machine learning
    • Linear regression
    • Neural networks
    • Visual object detection
On MNIST Dataset
On CIFAR Dataset
Gradient Descent
  • Minimizing loss and the need for numerical techniques
  • Gredient desent
    • Recipe
    • Update rule
  • Batch update
  • Mini-batch update
  • Stochastic (or online) gradient descent
  • Learning rate
    • Changing learning rate to achieve faster convergence
  • Newton’s method
    • How to choose a step size?
  • Momentum
tSNE
  • Stochastic Neighbour Embedding (SNE)
  • t-Distributed Stochastic Neighbour Embeding (t-SNE)
Autoencoders
  • Autoencoders
  • Variational autoencoders
  • Class-conditional autoencoders
  • PyTorch implementations using MNIST dataset
Recurrent Neural Networks
  • Sequential processing of fixed inputs
  • Recurrent neural networks
  • LSTM
Diffusion Models
  • Diffusion Models
Python development
  • A series of sketches on how to setup a Python development environment.
    • Python development hello world
    • Setting up Python development environments
    • Docker setup for Python development

Copyright and License

© Faisal Qureshi

Creative Commons Licence
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Last update: 2025-02-23 21:19
Webify version: 4.1
© Faisal Qureshi