From Boltzmann to Diffusion: Statistical Physics of Machine Learning (MVSpec)

Materialien

• week02_handout.pdf
• week03_handout.pdf

Übungsgruppen

• Gruppe 1
  1 Teilnehmer/innen
  Phil 12 106A, Mi 16:15 - 18:00
• Gruppe 02
  3 Teilnehmer/innen
  Phil 12, SR 060, Mi 16:15 - 18:00

Readings for the warm-ups

One reading per warm-up, Weeks 2 to 13. Every link is free to read except Week 8, which needs the university network (on campus, eduroam, or VPN). The sections to read, together with the priming task, are in each week's handout under Material.

• Week 2: J. J. Hopfield (1982), Neural networks and physical systems with emergent collective computational abilities, PNAS 79, 2554.
• Week 3: G. E. Hinton (2002), Training products of experts by minimizing contrastive divergence, Neural Computation 14, 1771.
• Week 4: H. Ramsauer et al. (2021), Hopfield networks is all you need, ICLR. Refresher: A. Vaswani et al. (2017), Attention is all you need, Section 3.2.
• Week 5: M. Mézard and A. Montanari (2009), Information, Physics, and Computation, Chapter 2 (Part A of the authors' draft).
• Week 6: Mézard and Montanari, Chapter 14 (Part D of the draft). Optional: D. L. Donoho, A. Maleki and A. Montanari (2009), Message-passing algorithms for compressed sensing, PNAS 106, 18914.
• Week 7: D. P. Kingma and M. Welling (2019), An introduction to variational autoencoders, Foundations and Trends in Machine Learning 12, 307.
• Week 8: S. Kirkpatrick, C. D. Gelatt and M. P. Vecchi (1983), Optimization by simulated annealing, Science 220, 671. University network required.
• Week 9: S. Mandt, M. D. Hoffman and D. M. Blei (2017), Stochastic gradient descent as approximate Bayesian inference, JMLR 18(134), 1.
• Week 10: A. Hyvärinen (2005), Estimation of non-normalized statistical models by score matching, JMLR 6, 695.
• Week 11: C. Jarzynski (1997), Nonequilibrium equality for free energy differences, Phys. Rev. Lett. 78, 2690.
• Week 12: Y. Song et al. (2021), Score-based generative modeling through stochastic differential equations, ICLR.
• Week 13: N. Tishby, F. C. Pereira and W. Bialek (1999), The information bottleneck method, Proc. 37th Allerton Conference, 368.