Isopycnal mixing of tracers is important for ocean dynamics and biogeochemistry. Previous studies have primarily focused on the horizontal structure of mixing, but what controls its vertical structure is still unclear. This study investigates the vertical structure of the isopycnal tracer diffusivity diagnosed by a multiple-tracer inversion method in an idealized basin circulation model. The first two eigenvalues of the symmetric part of the 3D diffusivity tensor are approximately tangent to isopycnal surfaces. The isopycnal mixing is anisotropic, with principal directions of the large and small diffusivities generally oriented along and across the mean flow direction. The cross-stream diffusivity can be reconstructed from the along-stream diffusivity after accounting for suppression of mixing by the mean flow. In the circumpolar channel and the upper ocean in the gyres, the vertical structure of the along-stream diffusivity follows that of the rms eddy velocity times a depth-independent local energy-containing scale estimated from the sea surface height. The diffusivity in the deep ocean in the gyres instead follows the profile of the eddy kinetic energy times a depth-independent mixing time scale. The transition between the two mixing regimes is attributed to the dominance of nonlinear interactions and linear waves in the upper and deep ocean, respectively, distinguished by a nonlinearity parameter. A formula is proposed that accounts for both regimes and captures the vertical variation of diffusivities better than extant theories. These results inform efforts to parameterize the vertical structure of isopycnal mixing in coarse-resolution ocean models.