Data Assimilation and Machine Learning as Statistical Physics Problems: Deepest Learning
Guest Speaker: Henry D. I. Abarbanel
Host: Gert Cauwenberghs
Transferring information from observations to models of those observations has long been a core practice in numerical weather prediction; it is called data assimilation. It is a problem in statistical physics. The same problem is posed in many machine learning settings. It is actually equivalent to data assimilation, and thus is also a statistical physics problem.
We will discuss both problems and show their equivalence. Examples from each, using instructive models, will be presented. In variational formulations of the problem, we will show how to use annealing in the precision of the model to identify the minima of the cost function (action). These depend strongly on the amount of information in the data and the structure of the model. Similar precision annealing in Monte Carlo analyses of the problem will be discussed.
We will show how one can identify models that are consistent with the data and which may be expected to give good predictions (generalization). The method also identifies just how much data is required for the task a machine learning models required to perform.
Each challenge in data assimilation or machine learning requires: (1) well curated data, (2) appropriate models, and (3) accurate and efficient learning/training methods. This talk focuses on the third element as a tool to validating and improving the model given the data. Both problems take on enhanced importance when the available data is very rich in information and the computational challenges in making and training models is substantial.