Reliability of future global warming projections depends on how well climate models reproduce the observed climate change over the twentieth century. In this regard, deviations of the model simulated climate change from observations, such as a recent “pause” in global warming, have received considerable attention. Such decadal mismatches between model simulated and observed climate trends are common throughout the twentieth century, and their causes are still poorly understood. Here we show that the discrepancies between the observed and simulated climate variability on decadal and longer time scale have a coherent structure suggestive of a pronounced global multidecadal oscillation. Surface temperature anomalies associated with this variability originate in the North Atlantic and spread out to the Pacific and Southern oceans and Antarctica, with Arctic following suit in about 25–35 years. While climate models exhibit various levels of decadal climate variability and some regional similarities to observations, noneof the model simulations considered match the observed signal in terms of its magnitude, spatial patterns and their sequential time development. These results highlight a substantial degree of uncertainty in our interpretation of the observed climate change using current generation of climate models.