Thread by @tomstafford

During my PhD I nearly lost my mind thinking about why we build computational models in cognitive science. {thread}

Modelling defines what makes cognitive science different from psychology. All cognitive scientists know that formal, computable models are good, but we don't always say exactly *why*

And when we do say, we find we don't exactly agree

After my PhD (which I completed without completely losing my mind), the issue of articulating exactly why we put so much effort into modelling still bugged me. Eventually I wrote something up, organised around answering the charge that models are just tautological

The accusation being that models just produce answers that depend on how you've set them up - the answers are "built in" from the beginning.

My answer to this was to say, in effect, "Yes, absolutely models are tautological, but we can learn a lot about the role they can play in science by exploring the implications of that"

In "What use are computational models of cognitive processes?" tomstafford.staff.shef.ac.uk/docs/stafford0… (presented at the Neural Computation and Psychology Workshop) I tried to set out how modelling could be understood as both tautological and scientifically essential

Models can be used for many different fundamental scientific motives, I argued, and proposed this taxonomy:

But, my claim was, only one class of motives could provide ultimate justification for model building - those relating to explanation; when models played a role in establishing existence proofs (or lack thereof), made or tested predictions

I also said, whether you accept the previous argument or not, that it is essential that modellers say why they are building models explicitly. If we don't know why they present a model, how can the model be evaluated?

(this seemed obvious to me. There's a prize at this point if you can guess where this is going)

"Models aid explanation in the same way as mathematics: by enhancing our perception beyond the horizon of individual reason and intuition." tomstafford.staff.shef.ac.uk/docs/stafford0…

So that was that paper, and I experienced temporary relief from the itch of not having systematised my thinking about models, but *then* I got to thinking... Hey, you've declared this neat schema, I wonder what the modelling literature in cognitive science actually looks like?

Are the most successful models (modellers, modelling papers) those which play an explanatory role? That test a theory? Prove some claim impossible? Demonstrate an existence proof? Are the best modelling papers those which explicitly declare the purpose for building a model?

I realised I could take a look: "How do we use computational models of cognitive processes?" tomstafford.staff.shef.ac.uk/docs/stafford_…

I did some searches and pulled out the most highly cited modelling papers from the journals Nature, Nature Neuroscience, Neural Computation, Cognitive Science and Connection Science, and that years NCPW conference.

Did being explicit about the scientific motivation for building your model predict higher citation rates? NO

Were the most highly cited modelling papers about models which focussed on explanatory purposes (explicitly or implicitly)? NO

Most modelling papers were not of my "explanatory type"

Model modelling papers were not of any single type - all types in my taxonomy (and a few not in it) were represented.

So much for my hope of finding a prescriptive formula for scientifically productive modelling in cognitive science

These two papers have languished in the conference proceeding for ten years (ten!) available to download with a £35 charge (£35!) but today a PhD student asked why we should build models and I remembered that I had an opinion about this written down

So I did a bit of formatting (yay LaTeX! Thanks @overleaf !) and uploaded them as post-prints to my publications page tomstafford.staff.shef.ac.uk/?page_id=8

Stafford, T. (2009). What use are computational models of cognitive processes? tomstafford.staff.shef.ac.uk/docs/stafford0…

Stafford, T. (2010). How do we use computational models of cognitive processes? tomstafford.staff.shef.ac.uk/docs/stafford_…