Deep generative models (DGMs) have shown great potential in structured-based drug design (SBDD). However, existing methods overlook a crucial physical constraint during both the learning and inference processes. That is, due to the attractive and repulsive forces, two atoms need to maintain a minimum distance as defined by their atomic radii. We refer to cases that violate this principle as \textit{atomic collisions}. To address this problem, we first introduce three novel metrics to measure the atomic collisions at three granularities. We then demonstrate that existing DGMs for SBDD can generate ligands exhibiting atomic collisions. To mitigate such an issue, we further devise NucleusDiff. It jointly models the distribution of atomic nuclei and surrounding electrons on a manifold, ensuring adherence to physical laws by constraining the distance between the nucleus and the manifold. Empirical findings demonstrate that NucleusDiff not only achieves superior performance on four out of seven metrics for stability and potency but also circumvents collision issues by up to 30\% on the three novel metrics, leading to a more efficient and effective drug design pipeline.