Radiation damage in zircon directly impacts diffusion of He from the crystal lattice and is a key factor in defining the kinetics of the zircon (U-Th)/He system. Damage accumulates within a crystal as a function of time and U and Th concentration, but can be thermally annealed as well. The total level of radiation damage in a zircon crystal is governed by a thermally-activated, kinetic process, which in turn influences the interpretation of zircon (U-Th)/He dates for thermal histories. Several annealing models have been defined for the zircon system based on measurements in natural crystals; however, few studies have investigated how multiple levels of radiation damage due to zonation of actinides within a crystal may influence the annealing process. Here we use Raman spectroscopy to map the full width half maximum (FWHM) of then (SiO) band, a proxy for radiation damage, in zircon crystals from the Lucerne pluton (Maine, USA) with heterogeneous distributions of U and Th. We compare FWHM maps before and after annealing these crystals at laboratory times and temperatures. These maps show that each damage zone within a single zircon acts as an isolated domain that is dictated by an independent set of annealing kinetics. Thermally activated annealing decreases radiation damage in all radiation damage zones; however, the rate of annealing is not consistent across all zones. We identify specific modes of damage in probability density plots of all measured FWHM in a crystal that are not present in pre-annealed imagery, but persist in post-annealing Raman maps, regardless of laboratory time temperature conditions: FWHM modes at 2-5 cm, 10-15 cm, and25-30 cm. We attribute these persistent damage modes to variable annealing kinetics that are partially dependent on the level of pre-annealing damage, combined with the inability of high-damage crystals, or zones within crystals, to fully recover their crystallinity. That is, some damage is permanent. These findings therefore show that zircon crystals with non-uniform distributions of U and Th can anneal to create long-lived damage zones at specific damage levels, which has implications for treating the zircon (U-Th)/He chronometer as a multi-domain diffusion system.