Stir zone (SZ) of AA6061-T6 double-side friction stir welding (DS-FSW) joint with a thickness of 6 mm had a poor fatigue performance compared with that of base metal (BM). The rolling process was employed to improve the fatigue performance of SZ. The results showed that rolling process increased the dislocation density in SZ and remarkably enhanced its fatigue performance. The crack initiation played a key role in the fatigue performance of SZ. It relied on the debonding of lamellar structure in SZ. The debonding of lamellar structure depended on the cracking of fine grain layer in lamellar structure.