A PDF copy of the course program can be found here: IMT_3420.pdf

November 21: We defined the Wasserstein distance and used a coupling argument to derive the convergence rate of the Langevin Monte Carlo algorithm.

Nov 14 - Nov 16: We reviewed the basics of stochastic calculus in preparation for discussing the following paper by Durmus & Moulines:

Alain Durmus and Éric Moulines, High-dimensional Bayesian inference via the unadjusted Langevin algorithm Bernoulli, 25(4A), pp. 2854-2882, 2019.

In the final two weeks, we will focus on the analysis of the Langevin Monte Carlo algorithm. This topic brings together various elements of diffusion processes, optimal transport theory, and concentration of measure results.

November 7: We introduced the symmetrization method and proved the Vapnik-Chervonenkis theorem. The latter allowed us to control the excess risk of certain statistical learning procedures.

November 2: We defined the Vapnik-Chervonenkis dimension and discussed its relation to the notion of metric entropy.

The fourth homework assignment has been posted.

Notes: VC_dimension.pdf

October 28: We used Dudley's entropy bound to derive a uniform law of large numbers over the class of Lipschitz functions.

October 24: We discussed the chaining technique and proved Sudakov’s minoration inequality along with Dudley’s entropy bound.

Notes: random_fields.pdf

October 19: We defined the notion of a Gaussian process and proved the Slepian and Sudakov-Fernique inequalities. Along the way, we also derived some results of independent interest on Gaussian interpolation.

Oct 12 - Oct 17: We discussed the decoupling technique for random quadratic forms and proved the Hanson-Wright inequality.

Notes: Hanson-Wright.pdf

October 3: We derived the matrix Bernstein inequality and revisited the problem of covariance estimation.

Notes: Bernstein.pdf

September 28: We introduced the Grassmann manifold and proved the Johnson-Lindenstrauss lemma.

September 26: We discussed the use of isoperimetric inequalities in deriving concentration of measure results. As an example, we proved the concentration of Lipschitz functions in certain metric spaces.

Notes: isoperimetry.pdf

September 21: We refined the ε-net argument of the previous lecture and proved a concentration inequality for the singular values of matrices with sub-Gaussian rows. We concluded the lecture by applying these results to the problem of covariance estimation.

The third homework assignment has been posted.

September 14: We used an ε-net argument to derive an estimate for the operator norm of random matrices with sub-Gaussian entries.

September 12: We introduced the notions of ε-nets and covering numbers along with the closely related notion of metric entropy.

Notes: random_matrices.pdf

September 7: We proved Grothendieck's inequality and discussed its role in establishing good semidefinite relaxations for integer programming problems.

September 5: We discussed some basic results on isotropic distributions and sub-Gaussian random vectors, and we proved that random unit vectors in high dimensions are with high probability almost orthogonal.

Notes: high_dimensional_vectors.pdf

August 31: We derived the concentration of the Euclidean norm for random vectors with sub-Gaussian coordinates. We concluded the lecture with the Gaussian annulus theorem.

The second homework assignment has been posted.

August 29: We turned our attention to sub-exponential distributions and derived Bernstein's inequality. Along the way, we also proved various properties of the sub-exponential and sub-Gaussian norms.

August 24: We defined the Orlicz space of sub-Gaussian random variables and proved the general Hoeffding inequality.

August 22: We derived some special cases of the Hoeffding and Chernoff inequalities. As an application, we proved that dense Erdös-Rényi graphs are almost regular with high probability.

Notes: concentration_inequalities.pdf

August 17: We defined the characteristic function (Fourier transform) of a random variable, and we sketched the proof of the central limit theorem due to Lindeberg and Lévy.

The first homework assignment has been posted.

August 10: We proved the Borel-Cantelli lemma and the strong law of large numbers.

August 8: We introduced several measure theoretic concepts and defined the Lebesgue integral with respect to a probability measure.

My lecture notes on probability theory can be found here: probability.pdf