Fatigue characterization of asphalt mixtures is imperative for predicting pavement performance. Moreover, since cracking begins in asphalt binders, obtaining fatigue properties in these materials makes it possible to establish cause and effect relationships with cracking of mixtures and pavement structures. In this paper, 12 Brazilian asphalt binders and mixtures were fatigue characterized, and correlations among scales were evaluated to verify the potential for predicting fatigue from binder scale. The characterization protocols include Linear Amplitude Sweep (LAS) test for binders and Direct Tension Cyclic Fatigue test for mixtures. Simplified viscoelastic continuous damage theory (S-VECD) was used to interpret the results, allowing the simulation of accumulated damage (N/Nf) in a pavement using FlexPAVE ^TM software. The main findings are that the type of binder alone is not able to predict fatigue class by the fatigue factor of binder (FFB), requiring rheological damage analysis. Most of the mixtures performed well in pavement simulations, and only fatigue factor of mixtures (FFM) and fatigue classes (FFM along with stiffness) are not enough to predict performance. There is a promising increasing linear correlation between FFB and FFM, but not between FFB and N/Nf. Despite this, excellent FFB values tend to dominate mixture’s behavior, generating low simulated damage.