Abstract: Galaxies are theorized to form and co-evolve with their dark matter halos, such that their stellar masses and halo masses should be well-correlated. However, it is not known whether other observable galaxy features, such as their morphologies or large-scale environments, can be used to tighten the correlation between galaxy properties and halo masses. In this work, we train a baseline random forest model to predict halo mass using galaxy features from the Illustris TNG50 hydrodynamical simulation, and compare with convolutional neural networks (CNNs) and graph neural networks (GNNs) trained respectively using galaxy image cutouts and galaxy point clouds. The best baseline model has a root mean squared error ($\text{RMSE}$) of $0.310$ and mean absolute error ($\text{MAE}$) of $0.220$, compared to the CNN ($\text{RSME}=0.359$, $\text{MAE}=0.238$), GNN ($\text{RMSE}=0.248$, $\text{MAE}=0.158$), and a novel combined CNN+GNN ($\text{RMSE}=0.248$, $\text{MAE}=0.144$). The CNN is likely limited by our small data set, and we anticipate that the CNN and CNN+GNN would benefit from training on larger cosmological simulations. We conclude that deep learning models can leverage information from galaxy appearances and environment, beyond commonly used summary statistics, in order to better predict the halo mass.