In the realm of electronic and electrical engineering, the automation of analog circuit design is increasingly vital, given the complexity and customized requirements of modern applications. However, existing methods only develop search-based algorithms that require lots of simulation iterations to design one custom circuit, which is usually a time-consuming process.To this end, we introduce LaMAGIC, a pioneering language model (LM)-based topology generation model that leverages supervised fine-tuning (SFT) for automated analog circuit design.LaMAGIC can efficiently generate optimized circuit designs from custom specifications in a single pass.Our approach involves a meticulous development and analysis of various input and output formulations for circuit.These formulations can ensure canonical representations of circuits and align with the autoregressive nature of LMs to effectively addressing the challenges of representing analog circuits as graphs. Experimental results show that LaMAGIC achieves a success rate of up to 96\% under a tolerance threshold of 0.01. We also examine the scalability and adaptability of LaMAGIC, specifically testing its performance on more complex circuits. Findings reveal the enhanced effectiveness of our adjacency-matrix-based circuit formulation with floating-point inputs, suggesting its suitability for handling intricate circuit designs.This research not only demonstrates the potential of LMs in graph generation but also builds a foundational framework for future explorations in automated analog circuit design.