Skip to yearly menu bar Skip to main content


Poster
in
Workshop: AI for Science: Scaling in AI for Scientific Discovery

Spectrum-Informed Initialization for Multistage Neural Networks

Jakin Ng · Yongji Wang · Ching-Yao Lai

Keywords: [ multistage neural networks ] [ scientific machine learning ] [ precision machine learning ]


Abstract: Deep learning frameworks have become powerful tools for approaching scientific problems such as turbulent flow, which has wide-ranging applications. In practice, however, existing scientific machine learning approaches have difficulty fitting complex, multi-scale dynamical systems to very high precision, as required in scientific contexts. We propose using the novel multi-stage neural net approach with a spectrum-informed initialization to learn the residue from the previous stage, utilizing the spectral biases associated with neural nets to capture high frequency features in the residue, and successfully tackle the spectral bias of neural nets. This approach allows the neural net to fit target functions to double floating-point machine precision $O(10^{-16})$.

Chat is not available.