Thread by @drbensearle

I will try to answer this question about how psychology researchers assess #creativity. Two common methods are rating systems like Novelty + Usefulness, and divergent thinking tasks like the Alternative Uses Test. The study discussed in this WaPo article used both.
A thread...🧵

https://twitter.com/marshallmaresca/status/1649096974283702272

(I should explain at this point that I’m not one of the authors of the article – people wanting more information should contact them. Nor am I in a competing research group. I don't have beef with the researchers or their paper, I’m merely an Interested Party!)

The Novelty + Usefulness approach to rating creativity comes from the notion of innovation being something new (novel) but with potential value (useful). This might be of interest if you’re studying things that could facilitate, or interfere with, innovative problem-solving.

This approach may frustrate those who believe art shouldn’t be judged on utility - that's a valid critique! But “useful” can vary by context. A storyteller may want to surprise readers with something unexpected (novel) that won’t disrupt the story structure (useful).

Anyway, researchers give people a task with creative potential (e.g. problems to solve), and have raters evaluate the solutions. Sometimes they use separate scales for novelty & usefulness, but more often it’s a single scale of low to high novelty+usefulness, as in this study.

(This itself is an issue. Other studies show that when giving a single rating score for multiple criteria, we’re influenced by our beliefs about their relative importance. But reviewers don’t like deviations from established methods unless there’s a very strong case for change.)

Then there’s the question of what task to use. You can try to replicate the kind of problems that need to be solved in a given context (so if you're studying engineers, you might use engineering problems). But in such cases, performance can be affected by prior knowledge.

Alternative Uses takes a different approach, and captures a different kind of divergent thinking. In this task, participants get a picture of an everyday object (a brick, a tin, a plank of wood) and are given a limited time to list all the ways the object could be used.

Responses can be scored on several criteria, such as total number of uses & number of use domains. For example, a brick could be used as part of a wall, a step, or a tiled patio - all from one use domain. Paperweight, weapon, and animal home are 3 uses from different domains.

(The cited study did not report on these other scoring criteria, just on the raters’ evaluations of Novelty+Usefulness of the responses to the Alternative Uses task. Not a criticism, I'm just trying to describe different ways of assessing creativity)

This approach to measuring creativity clearly has its own limitations. It captures whether, under time pressure, someone can identify applications of objects to a variety of practical situations. Not all people who excel in creative roles would perform well.

But remember that psychology is not mathematics. We cannot expect to develop perfect tools to measure constructs like creativity that have different meanings in different contexts. We find ways to measure things for certain reasons/contexts, and then try to refine these methods.

What we try to do is to use approaches that are comparative rather than descriptive. So the question is not "Is this a perfect measure of X?", but "Is this an acceptable way to turn X into numbers so that we can make sensible comparisons?"

The paper in question uses these tools to investigate effects of cannabis use. People were randomly assigned to take cannabis or not shortly before doing these tasks. The question was: for this approach to measuring creativity, did cannabis users perform better than non-users?

Spoiler: They didn't. But they did score higher on self-perceptions of creativity. So maybe that's why some people believe cannabis boosts creativity? As I've suggested, there may be aspects of creativity that went unmeasured, so its not clear.

But you might reasonably expect that if cannabis use had a powerfully beneficial effect on creativity, it might have influenced performance on this task.

I should mention that the cited paper (Heng et al., 2023) included one study using the Alternative Uses task and a second that involved conventional problem-solving (developing ideas for fitness equipment), both of which showed the same results.

These methods are only used for experimental research, where naïve participants perform tasks under controlled conditions. I don't believe that these methods could be used to determine which was the more creative of two artworks, two novels, or two films. /end

@threadreaderapp unroll