Vegetation restoration plays an important role in soil carbon storage, with the relatively lacking information in either the deep soil or the soil inorganic processes. The study aims to investigate the effect of returning farmland to the forest on deep soil carbon in the hilly and gully region of the Chinese Loess Plateau. Four sampling sites were selected: PO (Platycladus orientalis (Linn.) Franco forest; oriental arborvitae), PT (Pinus tabulaeformis Carr. Forest; Chinese red pine), AO (apple orchard) and FL (farmland, as a control treatment). Soil organic carbon (SOC) and soil inorganic carbon (SIC) content were analysed in 20-m soil profiles, along with the responsible factors. The mean SOC content slightly increased in the 1-5 m layer in the order FL < PO =AO < PT (p < 0.05) and in the 5-10 m layer in the order FL < PO < PT < AO, but no differences were found in the 0–1 m layer. Compared with FL, the SOC storages of PO, PT and AO increased by 6.71%, 16.28% and 2.50%, respectively. SIC content was relatively uniform throughout the profile, regardless of land-use types. The soil profiles in the study area presented vast SIC storage that was 8.8–10.2 times higher than SOC storage. After the 36-yrs implementation of the “Grain for Green” Project, SOC in 20 m soil profiles insignificantly increased as a cost of water depletion while SIC did not change. PT is the preferred tree species for soil carbon sequestration on afforested farmland in this region.