Mathematics for Machine Learning

About this course

The mathematics a practitioner actually needs to read papers, debug training runs and reason about models. Linear algebra is taught through embeddings and attention, calculus through autodiff and optimisers, probability through likelihood and experiment design, and numerics through the floating-point formats used to train today's models. Every idea is worked by hand for small cases and in NumPy for realistic ones.

For: student, software engineer, data professional.

Learning outcomes

On successful completion you will be able to:

1. 1Explain the linear-algebra, calculus, probability and information-theory concepts that underpin modern machine learning, including vector spaces, matrix decompositions, gradients, distributions and entropy.
2. 2Apply matrix and tensor operations and decompositions to machine-learning problems, by hand for small cases and in NumPy for realistic ones.
3. 3Derive the gradients of common loss functions and implement gradient-based optimisation, including momentum and adaptive methods.
4. 4Analyse data and model results with probabilistic and statistical reasoning, including estimation, uncertainty quantification, hypothesis testing and experiment design.
5. 5Diagnose numerical problems in ML code, such as overflow, underflow, ill-conditioning and precision loss, and choose numerically stable formulations.
6. 6Judge independently whether a mathematical or statistical claim in an ML paper, benchmark or product report is supported by the evidence presented.

Modules and lessons

1. Week 1Vectors, matrices and geometry⌄

   • Vectors, norms and similarity
   • Matrices as linear transformationslab
2. Week 2Decompositions and tensors⌄

   • Eigendecomposition and the singular value decomposition
   • Tensors, broadcasting and einsumlab
3. Week 3Calculus for learning⌄

   • Derivatives, gradients and Jacobians
   • The chain rule, computational graphs and automatic differentiationlab
4. Week 4Optimisation⌄

   • Gradient descent and its geometry
   • Adaptive optimisers and learning-rate scheduleslab
5. Week 5Probability⌄

   • Random variables and distributions
   • Likelihood, maximum likelihood and MAP estimation
6. Week 6Statistics and experiments⌄

   • Statistical inference and uncertaintylab
   • Designing experiments and A/B tests
7. Week 7Information theory⌄

   • Entropy, cross-entropy and KL divergence
   • Mutual information and the compression view of learning
8. Week 8Numerical computing and reading papers⌄

   • Floating point and numerical stabilitylab
   • Reading the mathematics of an ML paper

Assessment

• Quiz (20%)

  Eight weekly quizzes drawn from the lesson quiz banks; best seven count.

• Lab (35%)

  Six notebook labs with automated checks, covering least squares, einsum, autodiff, optimisers, the bootstrap and numerical stability.

• Project (25%)

  Written statistical critique of a published ML benchmark claim, reproducing its key numbers and assessing whether the evidence supports the conclusion.

• Exam (20%)

  Ninety-minute online written exam of derivations and short problems, taken on camera.

Pass mark 60%. Grades: Distinction, Merit, Pass.