Introduction to Deep Learning with PyTorch
Learn to build, train, and optimize neural networks for data analysis using PyTorch
Authors
This course provides a hands-on introduction to deep learning using the PyTorch framework. Starting from foundational concepts like loss functions, gradient descent, and automatic differentiation, you will progressively build your understanding of how neural network models learn from data. The course covers topics like activation functions for modeling nonlinear relationships, data preparation techniques like normalization and train-test splitting, and strategies for enhancing model performance through regularization and architectural choices. You will also learn to build custom neural network architectures using object-oriented programming and explore convolutional neural networks (CNNs) for processing grid-structured data like images. Through practical exercises with real-world datasets, including neuroscience data for decoding neural activity, you will gain the skills needed to design, train, and evaluate deep learning models for your own research and applications.
Credits
Installation
To run the course materials on your own machine:
- Install VSCode as your editor
- Install pixi or alternatively conda to create virtual Python environments (see the lessons on environment and package management)
- Download the materials for a lesson using the "Download Materials" button
- Extract the zip file and open the notebook in VSCode
- In VSCode, open a new terminal and install the environment:
pixi installconda env create -f environment.yml
conda activate dl_with_pytorchCourse Contents
Loss and Learning with PyTorch
Introduction to Data Modeling with PyTorch
The loss is used to find the model that best represents the data
Learning in Neural Networks
Create and train neural networks with PyTorch
Nonlinear Data, Activation Functions, and Classification
Nonlinear Data and Models
Nonlinear relationships can be modelled by adding activation function to the network
Classification with Neural Networks
Identify clusters or classes that data belongs to with neural network models
Prepare Data for Training
Scale Data to Prepare it for Model Training
Scaling data with normalization or standardization, encoding categorical data
Split Data into Train, Validation, and Test Subdatasets
Split dataset into train, validation, and test subsets to tune hyperparameters and test how well the trained model generalizes
Enhance Model Performance
Network Architecture: Depth vs Width
How network architecture choices affect training: depth vs width of network
Improve Network Through Regularization
Prevent overfitting through L1 and L2 regularization, dropout, and early stopping