Scaling ECG-Language Models: A Comparative Study of PEFT-Based Adaptation Across Backbone Architectures

We present preliminary findings from an ongoing systematic study of scaling ECG language models via Parameter-Efficient Fine-Tuning (PEFT). We compare four language models spanning two families and two size tiers - Qwen3.5-9B and Gemma4-E4B (4.5B effective) as small models, Qwen3.6-27B and Gemma4-31B as large models - each connected to a pretrained wav2vec2 ECG encoder via a projection layer. We compare three PEFT strategies: projection-only (LLM frozen), joint PEFT (connector + LoRA on LLM), and few-shot connector PEFT (with in-context examples). Projection-only fine-tuning consistently outperforms joint PEFT for large quantized models, which destabilize under cosine LR restarts. Scaling is non-monotonic: the 9B model achieves 46.79% exact-match accuracy on PTB-XL (+14.3% over prior ELMs), outperforming the 27B by 9.7 points, while Gemma4-31B achieves 74.19 ROUGE-L (+32.3%). Crucially, 20K-scale results show in-context learning unlocks latent capabilities in quantized models: the 27B with few-shot outperforms its projection-only baseline across all metrics (e.g., +2.9% ACC, +4.2 F1 on PULSE), while the 9B with few-shot trades precision for fluency. The 31B with few-shot shows more modest gains (+0.6% ACC on PTB-XL), suggesting the effect is modulated by both quantization and architecture. Failure-mode analysis reveals a clinical dichotomy: Qwen excels at rule-out (93-95% specificity) while Gemma4 excels at screening (66-75% sensitivity).