Understand neural networks and train small deep-learning models.
Deep learning
Status: Advanced outline. Begin after classical machine learning and linear algebra.
Prerequisites
- A complete classical ML project with evaluation
- Arrays, vectors, matrix multiplication, and gradients
- Python functions and basic debugging
- Ability to explain overfitting and validation
Learning objectives
- explain how tensors, parameters, loss, and gradients fit together;
- train a small model while separating training and evaluation;
- recognize overfitting from training and validation behavior; and
- inspect errors and state where the model should not be used.
Ordered subtopics
- Tensors, neurons, weights, and biases
- Activation functions and forward passes
- Loss functions and gradient descent
- Backpropagation and training loops
- Validation, regularization, and checkpoints
- PyTorch datasets and data loaders
- Convolutional neural networks
- Image augmentation and transfer learning
- Sequences, embeddings, and attention
- Experiment tracking, interpretability, and limitations
Planned project: train and evaluate an image classifier with a documented error analysis.
Completion evidence
Submit a training notebook with a baseline, train/validation curves, a saved model, at least five inspected errors, and a clear statement of where the model should not be used.
Best practices
- Start with a simple baseline and a small dataset.
- Fix random seeds when comparing experiments, but do not confuse reproducibility with generalization.
- Keep training, validation, and test data separate.
- Save the data version, model settings, and dependency versions.
- Do not use a model with people or safety-critical decisions without appropriate review and testing.