Singular Learning Theory: A Mathematical Introduction

Speaker: Kamillo Ferry, Imperial College London

Title: Singular Learning Theory: A Mathematical Introduction

Abstract: Classical statistics rests on an assumption that modern machine learning almost never satisfies: that a model’s parameters can be identified from data, so that near the truth the model looks locally like a Euclidean space of known dimension. In neural networks, mixture models and matrix factorisations, many different parameters describe the same model. The set of optimal parameters is then a singular real algebraic variety, and familiar tools such as the Fisher information and the BIC break down. Watanabe’s singular learning theory shows that the asymptotics of Bayesian learning are nevertheless governed by a single invariant of this singularity, the real log canonical threshold, which can be read off from a resolution of singularities. This talk is an introduction to the mathematics of the theory: why algebraic geometry enters, how resolution of singularities turns an analytic question into a combinatorial one, and what can be computed exactly. I will work through examples, from toy models to reduced-rank regression. No background in algebraic geometry is assumed.

Join: https://teams.microsoft.com/meet/3395756456794587?p=KXola0KUUnCaaSx1QX

We ask that you please familiarise yourself with the Hub’s code of conduct prior to attendance. Please note, this talk will be recorded.