Thread by @marcodelnegro

@pilossopher @paulgp Usually when I see some attention-seeking garbage on twitter I just move on. But since you ask, @pilossopher, and since you find this stuff interesting, @paulgp ... 1/n

@pilossopher @paulgp first, they generate data from the SW model and then use an R-pckge to find the mode. The results are not that surprising—it takes a lot of data to nail the true parameters, and for some you may not succeed at all even w lots of data because the model is not well identified 2/n

@pilossopher @paulgp Which is why reasonable people would never just look at the mode but explore the entirety of the posterior distribution—if the data are uninformative the posterior uncertainty would tell you just that. But I guess this subtlety escapes our esteemed authors 3/n

@pilossopher @paulgp Two, if one actually tried to find the mode for SW or any other largish DSGE, you would know that it is a tedious and error prone process. could be wrong but the idea that an R-pckge left to its own device finds the global mode for each of the 000s simulations seems unlikely 4/n

@pilossopher @paulgp Bottom line, my guess is that what sucks is actually their optimizer and that because of that the results of the paper are junk 5/n

@pilossopher @paulgp And btw, even exploring the posterior in SW is no trivial task, and one that standard packages may not do that well as shown in this paper
onlinelibrary.wiley.com/doi/abs/10.100…
6/n

@pilossopher @paulgp The second part of the paper does the series swapping exercise. First comment, it’s just a publicity stunt, as noticed by @OtiliaBoldea—why would you ever do something like that? 7/n

@pilossopher @paulgp @OtiliaBoldea Second, the exercise tells us absolutely nothing how good the DSGE is. The data are generated by SW! So it’s not like you can forecast actual GDP better by swapping it with inflation. All it tells us is that these guys can’t find the mode—which we knew already 8/n

@pilossopher @paulgp @OtiliaBoldea Third, because they can’t find the mode the resulting model is misspecified. Since a linear DSGE is just a bunch of cross equation restrictions on the VAR representation of the data …
pages.stern.nyu.edu/~dbackus/GE_as… 9/n

@pilossopher @paulgp @OtiliaBoldea if the model is misspecified it may well be that if you swap them across equations you may actually do better at forecasting. 10/n

@pilossopher @paulgp @OtiliaBoldea Big surprise? The two statisticians think it is. I am less sure. So, @PaulaAureo, I would not compare this to the serious work of Keane, Wolpin, Blundell, …you cite 11/n

@pilossopher @paulgp @OtiliaBoldea @PaulaAureo Last point, the authors do not seem to have read the literature they cite. The paper by Edge and @RefetGurkaynak actually shows that SW performs better than VARs (the exact opposite of what they write 🤣) and even the Greenbook 12/n

@pilossopher @paulgp @OtiliaBoldea @PaulaAureo @RefetGurkaynak In this Handbook of Forecasting chapter Frank Schorfheide and I show that they do as well as Blue Chip in a pseudo real time forecasting exercise that *includes* the Great Recession. But I guess this is all news to our experts
sciencedirect.com/science/articl…
13/n

@pilossopher @paulgp @OtiliaBoldea @PaulaAureo @RefetGurkaynak And in this paper we show that in *real real time*—that is, with *published forecasts*—the same is true
sciencedirect.com/science/articl… 14/n

@pilossopher @paulgp @OtiliaBoldea @PaulaAureo @RefetGurkaynak In fact, quarter after quarter, at the NY Fed we stick our neck out and publish our DSGE forecast. We do that not because we think our DSGE model is particularly good at forecasting—we agree with the authors on this (although it is as bad as others)..
libertystreeteconomics.newyorkfed.org/2022/06/the-ne… 15/n

@pilossopher @paulgp @OtiliaBoldea @PaulaAureo @RefetGurkaynak .. but precisely to figure out where and when it fails. So if you want to “check your model”, as the two statisticians patronizingly suggest we should do, then confront it with reality as opposed to run some nonsense series-swapping exercises 16/n

@pilossopher @paulgp @OtiliaBoldea @PaulaAureo @RefetGurkaynak A quick comment on @ben_moll’s comment: any economic model, including HA, being a simplified description of reality, can always be caricatured. Especially by people who do not understand it. 17/n

@pilossopher @paulgp @OtiliaBoldea @PaulaAureo @RefetGurkaynak @ben_moll we have learned a lot from representative agent models after all, in spite of their limitations. Maybe worth pointing out given that one of its main contributors just passed away. Perhaps let’s not give a pass to statisticians who mock them. 18/n

@pilossopher @paulgp @OtiliaBoldea @PaulaAureo @RefetGurkaynak @ben_moll So @pilossopher, I am afraid you got a lot more than you wished for :-). And apologies for not answering before but I was busy running the #TCSNYCMarathon this weekend! 19/n