Topico Apren Estad Cienc Datos

Stochastic Differential Equations and Score-Based Generative AI

Class notes, announcements, and other information can be found here.

Instructor: Anastasios Matzavinos, amatzavinos@uc.cl 

Teaching assistant: Lucas Buvinic Meira, lbuvinic@uc.cl 

Class meeting times: Mon & Wed 2:50 pm - 4:00 pm in room AE207.

TA's office hours: By appointment.

Course description: The focus of this course will be twofold: First, to develop competency in the theoretical foundations of stochastic analysis, and second, to apply this knowledge to understand recent developments in the intertwined fields of score-based generative modeling, generative AI, and machine learning. The first part of the course will cover material on Brownian motion and white noise, stochastic integration, Itô calculus, and the existence and uniqueness of solutions to stochastic differential equations, including the Feynman-Kac formula. In the second part, topics will revolve around score-based generative models in machine learning, such as score matching for diffusion processes, time reversals, the convergence of diffusion-based samplers, and mixing times. Diffusion-based generative models have been central to several recent generative AI technologies, including OpenAI’s DALL·E and Google’s Imagen, among others.

Grading policy: The final grade will be based on attendance (5% of the grade), homework assignments (35%), a mid-term exam (30%),  and a final take-home exam (30%).

Homework assignments: Homework problems will be handed out on a regular basis. Discussion of homework assignments with other students is encouraged, but what is handed in should be your own work. 

References: The following references will be used in different parts of the course. 

• L.C. Evans, An Introduction to Stochastic Differential Equations. American Mathematical Society, 2013.
• W. Tang and H. Zhao, Score-based Diffusion Models via Stochastic Differential Equations – A Technical Tutorial. arXiv: 2402.07487 2024.
• P. Dhariwal and A. Nichol. Diffusion models beat GANs on image synthesis. In Neurips, volume 34, 2021.
• M. Bishop and H. Bishop, Deep Learning: Foundations and Concepts. Springer, 2024.
• Sanseviero, P. Cuenca, A. Passos, and J. Whitaker. Hands-On Generative AI with Transformers and Diffusion Models. O’Reilly, 2024.
• R. Durrett, Stochastic Calculus: A Practical Introduction. CRC Press, 1996.

Announcements and other information about the class can be found here. A PDF copy of the syllabus can be found here: IMT_3450.pdf