One of the, to me, more annoying points that is often made with respect to #ChatGPT within my field of biblical studies is that it “lacks creativity.”
The reason for why I take issue with such simplistic assertions is that they do not seem to reflect any awareness of the literature on the subject. I’ve made creativity a core value in my own research and I am thus very interested in ChatGPT’s potential in this regard.
Unfortunately, so far there seems to have been carried out very little research on the topic by researchers who work on the psychology of creativity – if you know studies that I have missed, please let me know below!
What follows is mostly based on my more or less intuitive impressions of hundreds of hours of chatting with ChatGPT and a couple of simple tests.
Let us first begin with divergent thinking, the capability of generating a lot of different ideas from a single stimulus. This is a step in the creative process that lies behind “brain storming” sessions etc.
Many of those who think that ChatGPT has a role to play in research seem to use the chatbot along these lines. For example, check out the helpful threads by @MushtaqBilalPhD. Note, however, that ChatGPT aims at *probable* sequences of words.
If you want to think outside the box, so to speak, the heuristic value is therefore limited. However, in some cases getting a series of ideas that are expectable is of course precisely what you want – for example if you are preparing for a job interview or ...
... are wondering about possible objections to an article that you have written. In general, however, it seems to me that ChatGPT’s tendency to come up with rather obvious associations might be the main reason for why people are skeptical of its ability ...
... to generate creative proposals – even if they don’t know about the concept of divergent thinking. Their intuitive impression certainly has some justification. However, some qualifications are in order.
First, if ChatGPT seems uncreative to you with respect to coming up, for example, with good ideas about PhD topics, the reason for that might be that ideas in your field are all over the place, i.e. *too* divergent.
For example, the UK system of producing PhD theses requires students to come up with “innovative” ideas for applications, often during the first half of a one-year masters degree.
Then, after one year you are already supposed to submit the first chapter. In my opinion, the result is that many of these projects just don’t begin to make sense.
ChatGPT is programmed to avoid such nonsensical proposals – if it displayed the same kind of tendency for innovativeness, it would hallucinate even more and critics would be happy to point out to this alleged weakness even more.
If you want to play around with this, use the new #Bing chat (bing.com/chat) and try both the “creative” and the “precise” mode. More importantly, I want to note that it is indeed possible to increase the divergent capabilities of ChatGPT by means of simple tools.
For example, if you ask ChatGPT to come up with a list of ten possibilities for your question, this will of course lead to a greater variety of responses. Moreover, you can of course instruct ChatGPT to come up with “unusual” ideas.
I ran a very simple test that requires you to come up with a list of nouns that are as different from each other as possible: datcreativity.com
Cf. also pnas.org/doi/10.1073/pn… .
On its first attempt, ChatGPT was slightly above the average score, within the top 30% of subjects. I then told it that this was an exercise in divergent thinking and it did slightly better. The result was tainted by the pair “harmony” and “liberty.”
I made ChatGPT aware of that rather obvious mistake and with this feedback it managed a score in the top 10%. That’s not mind-blowing but ...
... it might be useful for you if you want to be creative and divergent thinking is not your greatest strength and/or you have to perform a whole series of creative tasks (such as coming up with the plot for a story) and/or you just have a dopamine low on a specific day.
(And let’s not forget it does all that in just a second or so.) Before moving on, I would also like to point out that you can make use of the probabilistic structure of ChatGPT’s inferences in order to find out how divergent a certain idea that you’ve had is ...
... just see how often you need to click the “regerate response”-button in order for the chatbot to come up with it. That’s like having a huge test audience! In general, I would like to see more studies on the divergent capabilities of ChatGPT.
I think in principle ChatGPT has one big advantage over human processes of divergent thinking. Divergent thinking requires us to access our memory to find possible links and then, in a second step, to dissolve actual connections, i.e., to do away with “autobiographical” stories.
To put it simply, ChatGPT does not have a bias for actual stories that are told in the simple past. It can take relationships that it finds in conditional sentences just as seriously without it requiring any effort.
Therefore, it seems to me that increasing the divergent capabilities of ChatGPT is mainly a matter of optimizing instruction.
In any case, it is not clear to me at all why anyone would think that when it comes to divergent thinking “the human brain wins hands down” (csm.tech/blog-details/6…).
Accordingly, suggestions to prioritize divergent thinking as a core skill in higher education due to perceived shortcomings of ChatGPT in this area may seem to be likewise based on a questionable assumption. (Against timeshighereducation.com/blog/thank-you…).
Unfortunately, many “studies” are currently published within a variety of different disciplines that are largely based on the authors chatting with ChatGPT for a very short time.
For example, this study makes the outlandish claim that “ChatGTP3 only promotes convergent thinking and prevent innovation” (researchgate.net/publication/36…).
This brings us to the second main step in the creative process (i.e., in the framework that was established by Joy Paul Guilford and that, despite some limitations, seems to me to be still quite useful in this context), convergent thinking.
Here, the issue is bringing together concepts and selecting one of the few correct answers in a pool of many possible combinations.
To use myself as an example, the creativity of my own research is limited almost exclusively to honing on what to me seems the only solution to a problem that makes sense to me.
For example, explicating a narratological definition of narratology or the notion of “implicit” proto-narratives by means of text-grammatical categories (degruyter.com/document/doi/1…).
Or using Bayes’s theorem to critique the structure of many arguments in biblical studies (theologie.unibas.ch/en/departments…).
Convergent thinking capabilities can be tested by means of the “remote associates test,” where you are presented with three cue words that are linked by a fourth word, which is the correct answer. You can do the test here: remote-associates-test.com .
You should be able to answer the easy problems. But even the medium ones are quite challenging. A typical person will only be able to solve half of these riddles – while failing to find the solution for most hard ones. I selected 10 random hard riddles.
ChatGPT got the correct answer 5 times at the first attempt and 3 times at the second attempt. Two times clicking on the “regenerate button” did not help as it continued to insist on its first suggestion.
However, when presented with the correct solution it immediately understood the underlying connection.
Even this rather superficial demonstration should show that ChatGPT has a lot of potential when it comes to convergent thinking in the creative process.
To be sure, convergent thinking requires knowledge and ChatGPT still does not have access to all scholarly literature (for better or worse – that’s another question).
So it is easy to trick yourself into thinking that ChatGPT is worthless by asking it very specific questions from your field of research.
Convergent thinking requires focus. I find it fascinating to see how much time it takes ChatGPT to come up with the responses for the RAT prompts (well, still only a couple of seconds … ).
If you want ChatGPT to solve one of the most fundamental issues in your field by drawing on a variety of other disciplines, it will simply not be able to do that in the time that most users would find acceptable ...
... which is why it takes a shortcut and gives you a rather general response that will most likely not satisfy you.
But if you provide ChatGPT with context or a preselection of options, you will be surprised about the kind of connections it can identify, connections you might otherwise have missed.
To give a very simple example not from research from everyday (academic life): I am currently receiving applications for doctoral positions in my research group. I did the experiment of running the job description, cover letters, and CVs through ChatGPT.
The chatbot was able to dramatically improve the cover letters by identifying just the right entries in the CV that contributed most toward the desired features as mentioned in the job description.
Another example, for those among you who are writing fiction. You are wondering about the cascade of events that would unfold in case of a specific event, e.g., a nuclear explosion? ChatGPT knows about architecture, laws, political dynamics and come up with fasciating scenarios.
I know this was a long 🧵and I barely scratched the surface of the subject matter. Still, I hope that I was able to demonstrate that the common idea that ChatGPT might be good in producing boring texts but is not capable of creativity is at best in need of further substantiation.
If you found it helpful in thinking about the role of ChatGPT in the creative process, I would be grateful if you shared it. And please let me know to what extent – or not – ChatGPT has helped you with divergent and convergent thinking.
@threadreaderapp unroll please.
Here is how ChatGPT summarized the thread:
https://twitter.com/ChristophHeilig/status/1633189770514898945
• • •
Missing some Tweet in this thread? You can try to
force a refresh
