Model Reduction by Manifold Boundaries

Understanding the collective behavior of complex systems from their basic components is a difficult yet fundamental problem in science. Existing model reduction techniques are either applicable under limited circumstances or produce “black boxes” disconnected from the microscopic physics. We propose a new approach by translating the model reduction problem for an arbitrary statistical model into a geometric problem of constructing a low-dimensional, submanifold approximation to a high-dimensional manifold. When models are overly complex, we use the observation that the model manifold is bounded with a hierarchy of widths and propose using the boundaries as submanifold approximations. We refer to this approach as the manifold boundary approximation method. We apply this method to several models, including a sum of exponentials, a dynamical systems model of protein signaling, and a generalized Ising model. By focusing on parameters rather than physical degrees of freedom, the approach unifies many other model reduction techniques, such as singular limits, equilibrium approximations, and the renormalization group, while expanding the domain of tractable models. The method produces a series of approximations that decrease the complexity of the model and reveal how microscopic parameters are systematically “compressed” into a few macroscopic degrees of freedom, effectively building a bridge between the microscopic and the macroscopic descriptions.

Figure 1

Identifying the boundary limit. For the exponential model in the text with eight parameters, initially, the least sensitive parameter combination involves many parameters and is difficult to remove (inset, top left). By following a geodesic to the manifold boundary (solid line), the combination rotates to reveal a limiting behavior; here, only one parameter (an exponential rate) becomes zero (inset, bottom left). As the boundary is approached, one eigenvalue of the FIM approaches zero (inset, right). Once the smallest eigenvalue becomes well separated from the other eigenvalues, the limiting behavior becomes apparent. This limit is largely independent of the starting point; nearby parameter values all map to the same simplified model (dashed lines).

Figure 2

Approximating the manifold by its boundary. A high-dimensional, bounded manifold may be approximated by a low-dimensional manifold. Parameter degrees of freedom are systematically removed, one at a time, by approximating the full manifold by its boundary. After several approximations, the reduced model is represented by a hypercorner of the original manifold that preserves most of the original model’s behavior.

Figure 3

Original and reduced EGFR models. The interactions of the EGFR signaling pathway [32, 33] are summarized in the leftmost network. Solid circles are chemical species for which the experimental data were available to fit. Manifold boundaries reduce the model to a form (right) capable of fitting the same data and making the same predictions as in Refs. [32, 33]. The FIM eigenvalues (center) indicate that the simplified model has removed the irrelevant parameters identified as eigenvalues less than 1 (dotted line) while retaining the original model’s predictive power.