Mathematics for Machine Learning

The course begins with foundational topics in set theory and logic. Students develop a precise mathematical language for describing sets, functions, and predicates, and are introduced to key ideas such as supremum and infimum, as well as argmax and argmin notation. Logical reasoning and Boolean functions are also studied, providing important conceptual motivation for understanding the expressive limitations of some simple machine learning models, such as why certain functions (e.g. XOR) cannot be represented by a single-layer neural network.

Students then explore matrices in-depth. They study Gaussian elimination, solve systems of equations, learn about determinants and their properties, and compute inverse matrices. These tools form the computational backbone of many machine learning algorithms.

As part of this course, students perform a deep dive into vector spaces, exploring linear independence, subspaces, bases, dimension, rank, and nullity. They also study operations such as the Hadamard product, which appears naturally in element-wise computations in neural networks. Students generalize key concepts to abstract vector spaces and inner product spaces, and study orthogonality, projections, and the Gram–Schmidt process.

Students will learn how to find eigenvectors of a matrix, compute matrix diagonalizations, and extend these ideas to orthogonal diagonalization of symmetric matrices. These techniques underpin dimensionality reduction methods and are central to understanding the structure of data.

The course then develops key applications of linear algebra in machine learning. Students study singular value decomposition and its relationship to principal component analysis, and learn how these tools are used for dimensionality reduction, feature extraction, and identifying structure in high-dimensional datasets. Linear least-squares problems and regression techniques are also studied as foundational methods for fitting models to data.

A solid grasp of multivariable calculus is essential for understanding modern machine learning algorithms. In this course, students become well-versed in partial derivatives and gradient vectors (used in gradient-based optimization), the multivariable chain rule (essential for backpropagation), and vector-valued functions. Students extend differentiation to maps between multi-dimensional vector spaces and develop the tools of vector and matrix calculus used to compute gradients of loss functions in neural networks.

Students also study optimization of multivariable functions, including critical points, second derivative tests, and constrained optimization using Lagrange multipliers. These techniques form the mathematical foundation of algorithms such as support vector machines and constrained optimization problems in machine learning.

Students work with standard multivariable surfaces to build intuition for loss surfaces, and study double integrals as a tool for understanding continuous probability distributions.

On the probability and statistics side, students study discrete and continuous random variables, including probability density functions, cumulative distribution functions, transformations, expectation, moments, variance, and Bayes' theorem. Important probability distributions are explored in detail.

Students extend their understanding to joint, marginal, and conditional distributions, as well as sums and products of random variables. Topics such as covariance, correlation, and multivariate normal distributions are introduced, which are essential in probabilistic modeling.

The course concludes with parametric inference, focusing on point estimation, maximum likelihood estimation, and confidence intervals, which form the statistical foundation for many machine learning methods.

• Develop a solid understanding of sets, logical reasoning, Boolean functions, and vector geometry.
• Understand supremum, infimum, and apply argmax and argmin notation in optimization contexts.
• Master determinants, Gaussian elimination, and learn how to compute inverse matrices efficiently.
• Develop a thorough understanding of vector spaces, including bases, dimension, rank, nullity, and the rank-nullity theorem.
• Understand and apply operations such as the Hadamard product in vector and matrix computations.
• Understand the concept of diagonalization and how to apply it.
• Develop knowledge of orthogonality in vector spaces, including orthogonal sets, complements, and orthogonal projections.
• Learn bilinear and quadratic forms, including positive-definite and negative-definite quadratic forms.
• Understand singular value decomposition and its applications, including the pseudoinverse matrix.
• Understand and apply principal component analysis for dimensionality reduction and feature extraction.
• Solve linear least-squares problems, both with and without collinearity.
• Perform linear, polynomial, and multiple linear regressions.
• Compute partial derivatives, gradient vectors, Jacobian matrices, Hessian matrices, and understand their geometric interpretations.
• Apply vector and matrix calculus to compute derivatives of functions between multi-dimensional vector spaces.
• Understand vector-valued functions and their properties.
• Analyze and optimize multivariable functions, including constrained optimization using Lagrange multipliers.
• Interpret and analyze loss surfaces arising in machine learning models.
• Use Riemann sums to approximate volumes.
• Calculate double integrals over rectangular and non-rectangular domains.
• Understand and apply fundamental probability concepts such as the law of total probability and Bayes' theorem.
• Work with discrete and continuous random variables.
• Apply transformations to random variables and compute the associated CDFs and PDFs.
• Understand and compute expectation, variance, and higher moments.
• Work with joint distributions, marginal distributions, and conditional distributions.
• Compute covariance and correlation for random variables.
• Work with normally distributed random variables and their combinations.
• Apply point estimation and maximum likelihood estimation.
• Construct and interpret confidence intervals in statistical models.