Contrary to classical statistical theory, machine learning models with enormous sizes have shown high performances even when they interpolate the data. Such a phenomenon is called ``benign overfitting'' and has attracted much attention in the theoretical literature. Recent studies have clarified its theoretical perspective mostly in linear models, and there are yet only a few results for neural networks with feature learning. To address this issue, we theoretically investigate the statistical property of two-layer neural networks trained by noisy gradient descent in the setting where inputs have an intrinsic structure with lower dimensionality. We show when a true model is given by another neural network, the trained network can obtain the intrinsic feature of the true model through the gradient based training and eventually achieve benign overfitting.