Learning the ESG Geometry with Domain Aware Language Models

Responsible investing aims to generate positive impact across Environment (E), Society (S), and Governance (G), and rating companies along these dimensions is now widespread, making ESG scores highly popular. Allocating retail capital with sustainability in mind could be transformational, yet it remains unclear how individual investors can do so in practice. Current ESG solutions cannot model high-dimensional, multi-modal time series capturing the joint evolution of ESG risks, financial returns, news, and sentiment, even though this domain requires jointly reasoning over distinct numerical signals where both numerical proximity and semantic type must be preserved. To bridge this gap, we introduce a novel domain-aware $\textbf{representation learning framework}$ that learns geometry-preserving representations for heterogeneous time series using value-aware tokens with block-wise $\textbf{orthogonal embeddings}$. To capture trajectory-level structure, we introduce $\textbf{FACET}$ tokens and train the model using a geometry-preserving loss. The resulting model jointly learns to forecast future values and to organize entities in a representation space that reflects their temporal evolution. Trained on ESG, returns, news, and sentiment, the domain-aware LLM learns a representation space that enables accurate ESG forecasting, trajectory-based grouping, and latent-space search for superior asset selection and downstream application like portfolio rebalancing

This paper introduces an AI system that helps investors make more sustainable financial decisions by better understanding how companies perform on environmental, social, and governance (ESG) factors. Existing investment tools often struggle to combine many different types of information at once, such as ESG scores, stock returns, company news, and public sentiment over time. Our work proposes a new “domain-aware” language model that learns patterns from all of these signals together while preserving both their numerical meaning and their financial context. The system creates specialized representations for different kinds of financial information so the model can understand that two close ESG scores are related, while an ESG score and a stock return with the same number represent very different concepts. It also learns how companies evolve over time, enabling it to forecast future ESG risks and identify companies with similar financial behavior but stronger sustainability performance. The learned representations can also support practical applications such as ESG forecasting and portfolio rebalancing, where investors replace less sustainable companies with financially similar but more sustainable alternatives. More broadly, this work aims to make responsible investing more accessible by helping both individual and institutional investors better evaluate sustainability risks and opportunities. In the future, the system could evolve into an interactive AI assistant that helps users build and maintain sustainable investment portfolios through natural language conversations.