Successive observation of fatigue crack initiation process was conducted on six types of thermally simulated HAZ microstructures to clarify the physical significance of fatigue life difference between coarse grain (CG) and fine grain (FG) heat-affected zone (HAZ) microstructures. The results showed the fatigue life-decreasing mechanism in the CGHAZ to be successfully explained by acceleration of the shear mode fatigue crack growth rate provided the crack is smaller than the prior austenite grain size, based on the following experimental results. In other words, the fatigue cracks initiated along the shear stress plane. In addition, the prior austenite grain size of crack initiation life with the FG and CGHAZ were comparable, since the shear mode fatigue crack growth rate of the CGHAZ was faster than that of the FGHAZ. As a result, the remaining fatigue life after the grain size crack initiation of the CGHAZ become shorter than that of the FGHAZ as a function of grain size difference.