Seminar: Patnaik-Pearson intrinsic dimension for internal representations of neural networks

Thursday 15^th October 14:00-15:00

Speaker: Dr. Tom Hadfield, Imperial College London

Title: Patnaik-Pearson intrinsic dimension for internal representations of neural networks

Abstract: We define a new measure of intrinsic dimension of a data manifold, which we call the Patnaik-Pearson dimension, and apply this to internal representations of neural networks, in particular transformers. The inspiration for this comes from the HTSR and SETOL work of Martin, Mahoney and Hinrichs, combined with the TwoNN intrinsic dimension estimator of Facco et al. We prove various properties of this intrinsic dimension estimator. Treating weight matrices of neural networks as data manifolds, for weight matrices whose empirical spectral density follows a Pareto (power law) distribution, we relate the Patnaik-Pearson dimension to the HTSR and SETOL analysis, and show that critical values of the tail exponent coincide for the two approaches. Using a combination of theoretical and numerical techniques, including Voiculescu’s free probability, we study the behaviour of the Patnaik-Pearson dimension of a data manifold under the transformations typical to neural networks. https://arxiv.org/abs/2606.19268

https://teams.microsoft.com/l/meetup-join/19%3ameeting_MzY2ZDkwMTAtNGE5OC00Yzc1LWFjNzItODU4YmYyMjMwZjc3%40thread.v2/0?context=%7b%22Tid%22%3a%222b897507-ee8c-4575-830b-4f8267c3d307%22%2c%22Oid%22%3a%22311f0e52-bbf7-47a9-8cc6-752fcc16adcc%22%7d

We ask that you please familiarise yourself with the Hub’s code of conduct prior to attendance.
Please note, this talk will be recorded.