Abstract: Quasars play a crucial role in studying various important physical processes. We propose a cross-modal contrast learning method for estimating the photometric redshifts of quasars. Our model utilizes adversarial training to enable the conversion between photometric data features (magnitudes, colors, etc.) and photometric image features in five bands (u, g, r, i, z), in order to extract modality-invariant features. We used $|\Delta z|=|(z_{photo}-z_{spec})/(1+z_{spec})|$ as evaluation metric. The latest SOTA method, which implements cross-modal generation of simulated spectra from photometric data, has been chosen as the baseline. Firstly the proposed method was tested on the same SDSS DR17 dataset of 415,930 quasars$(1 \le z_{spec} \le 5)$ as the baseline method. Compared to the baseline, the RMSE of our $\Delta z$ decreased from 0.1235 to 0.1031. Further evaluation on a larger dataset of 465,292 quasars achieved a lower RMSE of $\Delta z$ of 0.0861. This method also can be generalized to other tasks such as galaxy classification and redshift estimation.