After a year of work our paper on evaluating performance of historical climate models is finally out! We found that 14 of 17 the climate projections released between 1970 and 2001 effectively matched observations after they were published. doi.org/10.1029/2019GL… 1/19 
While modern climate models are commonly evaluated by predicting the past through "hindcasts", modelers have knowledge of past temperatures and matching historical warming is not always an independent test of model skill. 2/19
Projecting future warming after the model is published is a better test of model skill. However, it poses a problem: a model might get the physics of the climate system perfect, but might have a poor estimate of future warming if it gets future emissions wrong. 3/19
For example, a number of climate models published in the 1970s and 1980s estimated that present-day CO2 levels would be around 450 ppm, rather than the 410 ppm that actually occurred. 4/19
Emissions arise from human decisions, not physical processes, and are in many ways much more difficult to actually predict. We wanted to evaluate climate models based on how good their climate physics performed, not their future emissions crystal ball. 5/19
We searched for all the old climate models we could find that had published projections of future warming and future CO2 concentrations and other climate forcings. We identified 14 models with 17 different projections (some models had multiple scenarios). 6/19
These include the first three IPCC assessment reports; we looked at the fourth as well, but the period after publication – 11 years – was too short to effectively say much about how well its future projection performed. 7/19
Many of these models had future climate forcings – increases in greenhouse gases, aerosols, and other factors influencing the climate – that differed notably from what actually happened in the real world: 8/19
We evaluated models on two metrics: how the change in temperature over time compared to observations (top panel), and how the change in temperature over the change in forcings compared to observations (bottom panel). 9/19
This latter approach controls for mismatches between modeled and observed radiative forcing; it effectively looks at how much the model warms for a standardized change in radiative forcing (in this case, that associated with a doubling of CO2). 10/19
10 of 17 model projections were statistically indistinguishable from observations in the years after they were published on a temperature vs time basis. Controlling for mismatches in future forcings, 14 of the 17 projections match observations. 11/19
Hansen's famous 1988 model is a good example of why this temperature vs. forcing approach is a better way to evaluate models. While his most likely "Scenario B" overestimated warming by around 50% after it was published, it also overestimated increase in forcings by 27%. 12/19
While Hansen's Scenario B actually got future CO2 spot on, he failed to anticipate the Montreal protocol cutting CFC emissions and overestimated future methane. 13/19
We can see the overestimate of future temperatures when we plot Hansen's projections vs observations over time (top panel). However, when we plot both Hansen's model and observations on a temperature vs forcing basis (bottom panel), they match up rather well. 14/19
Because we can't directly observe the actual changes in radiative forcing that occurred, we used a 1000-member ensemble of forcing estimates to account for uncertainties in things like aerosols (as well as 5 different observational temperature records). 15/19
The main takeaways are that past climate models were quite skillful at projecting future warming that occurred after they were published. This is particularly true for the models published in the 1970s, when there was limited evidence at the time that the Earth was warming. 16/19
The warming thats occurred over the past 50 years is pretty much exactly what our models projected it would be. Its not worse than we thought, and models have not overestimate future warming. This improves our confidence that modern climate models are getting things right. 17/19
The authors of the paper were @henrifdrake, Tristan Abbott, @ClimateOfGavin, and myself.
For more details, see our new post at RealClimate: realclimate.org (should be up shortly!) 18/19
For more details, see our new post at RealClimate: realclimate.org (should be up shortly!) 18/19
The article was inspired by this (much simpler) @CarbonBrief piece I did back in 2017: carbonbrief.org/analysis-how-w… 19/19
@CarbonBrief Its also worth mentioning that there were some non-peer-reviewed projections, such as this one from Exxon researchers in the early 1980s, that also performed well but were not included in our paper (as they were not academic publications): earther.gizmodo.com/exxon-predicte… 20/19
@CarbonBrief A non-paywalled copy of our paper can be found here: pubs.giss.nasa.gov/abs/ha08910q.h…
