Shanon Memorial Lecture

Yale University Professor of Statistics and Data Science Andrew Barron will speak on “Information Theory and High-Dimensional Bayes Computation.” 

Abstract: Notable advances in information theory and statistics are associated with efficient procedures for Bayesian computation. We review the history of such developments starting with Laplace and Gauss and their developments of predictive distributions, normal approximations, and least squares. Modern counterparts include Kalman filtering, Bayesian belief propagation, the context tree weighting algorithm, LDPC codes, and sparse regression codes. Markov chain Monte Carlo procedures broaden the Bayesian computation landscape, and rapid information-theoretic convergence is available for sampling log-concave posterior distributions. However, artificial neural network estimation is made challenging by the multi-modality of the posterior. We overcome this challenge by introducing auxiliary parameters for which their distribution is log-concave and likewise for the conditional distribution of the network parameters.   An associated procedure computes Bayes optimal neural net estimates.

Biography: Andrew Barron is the Charles C. and Dorothea S. Dilley Professor of Statistics & Data Science at Yale University. He works at the interplay of Information Theory with Probability and Statistics. Prior to joining Yale in 1992, Barron was at the University of Illinois at Urbana-Champaign. He obtained his Ph.D. from Stanford in 1985 and Bachelor’s from Rice University in 1981. He received the Shannon Award and provided the 2024 Shannon Lecture at the International Symposium on Information Theory. He is a Fellow of the IEEE and a winner of the IMS Medallion Prize and the IEEE Thompson Prize. He served as secretary and elected member of the Board of Governors of the Information Theory Society, as associate editor of the Transactions on Information Theory and the Annals of Statistics, and as chair of the Thomas Cover Dissertation Prize Committee. Research accomplishments include generalizing the AEP to continuous-valued ergodic processes, proving an information-theoretic central limit theorem, formulating the index of resolvability, characterizing Minimum Description Length performance, determining asymptotics of universal data compression, characterizing concentration of Bayesian posteriors, information-theoretic determination of minimax rates of function estimation, information-theoretic characterization of statistical efficiency, approximation and estimation bounds for artificial neural networks,  information-theoretic aggregation of least squares regressions, and capacity-achieving sparse regression codes for Gaussian channels.  

Organizers encourage people to attend this event in-person. However, if you are unable to attend in-person, you are invited to tune in to the lecture virtually via Zoom.

Zoom link: https://ucsd.zoom.us/j/92297527906?pwd=aYS3PbJNOwkqDXBFEKlOChmxDHqbHf.1
Meeting ID: 922 9752 7906
Password: SML2024

For more information, please email Octavio Ochoa at oochoa@ucsd.edu.