Watershed and global-scale nitrogen (N) budgets indicate that the majority of the N surplus in anthropogenic landscapes does not reach the coastal oceans. While there is general consensus that this 'missing' N either exits the landscape via denitrification or is retained within watersheds as nitrate or organic N, the relative magnitudes of these pools and fluxes are subject to considerable uncertainty. Our study, for the first time, provides direct, large-scale evidence of N accumulation in the root zones of agricultural soils that may account for much of the 'missing N' identified in mass balance studies. We analyzed long-term soil data (1957–2010) from 2069 sites throughout the Mississippi River Basin (MRB) to reveal N accumulation in cropland of 25–70 kg ha−1 yr−1, a total of 3.8 ± 1.8 Mt yr−1 at the watershed scale. We then developed a simple modeling framework to capture N depletion and accumulation dynamics under intensive agriculture. Using the model, we show that the observed accumulation of soil organic N (SON) in the MRB over a 30 year period (142 Tg N) would lead to a biogeochemical lag time of 35 years for 99% of legacy SON, even with complete cessation of fertilizer application. By demonstrating that agricultural soils can act as a net N sink, the present work makes a critical contribution towards the closing of watershed N budgets.