In this paper, we demonstrate the utility of Amortized Neural Posterior Estimation (ANPE) for the problem of stellar spectra fitting. We introduce an effective approach to handle the measurement noise properties inherent in spectral data. This allows training-time data to resemble actual observed data, an aspect that is crucial for ANPE applications. We apply this approach to train an ANPE model for the APOGEE survey that observed over 2 million spectra, and demonstrate its efficacy on both mock and real APOGEE spectra.To train the ANPE model, we applied our new NPE framework---Neural Bayesian Inference (\textit{nbi})---that is concurrently submitted to this workshop as an NPE framework optimized for stress-free astronomical applications. Application of this framework allowed us to train the ANPE model with minimal customization and coding efforts.Given the association of spectral data properties with the observing instrument, we propose the idea of an ANPE ``model zoo,'' where models are trained for specific instruments and distributed with the \textit{nbi} framework to facilitate real-time stellar parameter inference.