Under proper loading conditions, micro-to-nanoscale heterogeneities (i.e., the bond system) that are commonly found within the materials of a system can coalesce until causing macroscopic alterations of the system properties. The bond system is responsible for atypical and invariant-scale non-linear elastic processes in granular media, from laboratory-tested materials (mm) to the Earth’s crust (km). The unusual observed behavior involves slow recovery, or relaxation, of the elastic properties after dynamic loading. Several models have been designed to explain non-linear elasticity, although their physics is still partially unknown. Here, we show that recovery processes are also observed at intermediary scales (m) in civil engineering structures, and that they might be related to structural health due to the healing of cracks. For Japanese buildings subjected to earthquakes, we observe rapid co-seismic reductions of their resonance frequency, followed by fascinating recoveries over different time-scales: over short times (i.e. seconds) for a single earthquake; over intermediate times (i.e. months) for a sequence of aftershocks; and over long times (i.e. years) for a series of earthquakes. By comparing two buildings with different damage levels after the 2011 Tohoku earthquake, we show how relaxation models can characterize the level of cracking caused by damaging events. Our results bridge the gap between the laboratory and seismological observation scales, verifying in this way the universality of recovery processes, and demonstrating their value for the detection and characterization of damage.