Abstract: Recent studies showed the possibility of extracting SoS information from pulse-echo ultrasound raw data (a.k.a. RF data) using deep neural networks that are fully trained on simulated data.These methods take sensor domain data, i.e., RF data, as input and train a network in an end-to-end fashion to learn the implicit mapping between the RF data domain and SoS domain. However, such networks are prone to overfitting to simulated data which results in poor performance and instability when tested on measured data. We propose a novel method for SoS mapping employing learned representations from two linked autoencoders. We test our approach on simulated and measured data acquired from human breast mimicking phantoms. We show that SoS mapping is possible using the learned representations by linked autoencoders. The proposed method has a Mean Absolute Percentage Error (MAPE) of $2.39\%$ on the simulated data.On the measured data, the predictions of the proposed method are close to the expected values (MAPE of 1.1%).Compared to an end-to-end trained network, the proposed method shows higher stability and reproducibility.