Thread by @timbocop

At @CRADLEdeakin and @TEQSAGov webinar on #ChatGPTHigherEd - how should educators respond?

Firstly, @margaret_bea raises the need to include students in discussions and to look for a diverse range of voices. Yes!

Side note: I’m hearing a lot of people say “ChatGPT is just a tool”. Beware instrumentalism – technologies aren’t neutral things that humans use to their desired ends. They-and-we, in combination, reshape society and the world.

Margaret says machines don't construct quality and we need students to develop evaluative judgement. Complex, contextual and disciplinary judgements.

ChatGPT not good at this... yet

Adapting assessments for now:
- critically review rubrics (more nuanced ideas of quality)
- tasks where students make and defend contextualised judgements
- assess how students change their work based on evaluative judgements

Q's from panel
Lucinda: how to get around students using chatGPT to produce evaluative judgements

Margaret: focus isn't catching students out but working on rubrics to elicit human, contextualised quality. Trust educators to set parameters of assignments to promote complexity.

From Simon:
Can ask ChatGPT to critique from a conceptual framework (and results are impressive)

Margaret: it can help scaffold evaluative judgement but remains limited and needs students to build on its responses.

Rola: Is this just a matter of good assessment design?

Margaret: Yes and no. Hopefully a shift in focus towards human capabilities. We need to look at where our priorities in assessment might change.

Margaret now discussing "good writing" - we need to monitor how this unfolds, how writing is changing in relation to tech. This will change our notion of writing standards.

Now @LucindaMcKnigh8 on writing and AI.

Students (and everyone) have been using AI writers for years. We are way behind in our thinking about this issue.

ChatGPT is a user-friendly interface on an engine that was already pretty powerful.

Is the essay dead? Will humans just tinker with AI text?

ChatGPT is a distraction, just one form of AI writing that will be (more) intimately woven into our techs so that human/AI writing becomes indistinguishable. And anyway, editing takes great writing skill.

My interpretation of what Lucinda is saying: The idea of citing ChatGPT like any other source doesn’t make sense in this context. We need to build creativity into assessment to make good use of this integration of human / AI

Simon: Should we be asking students to do things ChatGPT can do or not?

Lucinda: both but where can we find the time for it all?

Lucinda says Academia's needs to align with what’s happening in industry and teach students to use these things..

and therefore teachers need to be experts in this tech (and in assessment design, adds Rola). Where do we find the time?

Hmm.. Not sure this is the most useful way to think about this.. I think we need to learn how to teach things we ourselves don't know.

By that I mean that we might move away from the model where the teacher is supposed to know the answers but instead is good at supporting students to ask questions and undertake inquiry. What do you think?

Now @sbuckshum on effective ethical engagement with ChatGPT. Wants to move from aspiration to evidence.

Again note: we’ve been working with AI for ages.
Mentions Salomon et al. “partners in cognition”. Partnership needs agency & effort.

Students aren’t pro’s – might not be able to make expert use of this tech in learning.

Need to equip students with knowledge, dispositions, integrity to support good use.

Raises the idea of automated feedback literacy - do students know how to engage with automated feedback?

It quickly become meaningless to ask students to declare each AI use. Simon is looking at process analytics to better understand this partnership.

Uni's need to work quickly on research in generative AI pedagogy.

Simon ends with 3 key things for educators:
- Scaffold variable student capacity to critique ChatGPT output
- Guide students on evidencing critical engagement with ChatGPT
- Fill evidence vacuum with quality research

Rola on relational and affective aspects of feedback. How does AI-generated feedback fit with these ideas? Is it about educator’s scaffolding this AI-generated information?

Simon: we might be overoptimistic about students’ critical engagement faculties. So it needs scaffolding

Now on whether ChatGPT can be a springboard for expressing ideas? Can it make things more inclusive?

Answer: potential benefits and risks 😂

Simon: do we still assume that students need to be able to do work without using technologies?

Lucinda mentions the possibility of removing meaningless labour, are we ok with that?

S: as tech gets more reliable the need to be able to do without it is reduced (right, Simon?)

Getting thirsty, just going to get ChatGPT to finish this thread for me...

Warnings from panel that using ChatGPT to summatively assess student work breaches OpenAI terms and conditions and the student’s data rights.

Teacher POV from @sahoward_uow. Education is slow but resilient. We reshape tech into our practices. But ChatGPT has caught our imagination– good interface, good performance. What AI competencies do educators need? We’re trying to figure out what we want to do with these tech’s.

Humans might be better at critical thinking but that doesn’t mean we’re doing it. How do we scaffold these human capacities alongside these tools?

(as @sbuckshum points out in the chat, some humans aren't good at critical thinking)

(The chat is also right about Sarah's dog, it really is very cute)

Audience Q: how to help students develop capacities for ethical, sustainable & effective use of AI?
MB: focus on coming to know what's ethical and what quality is irrespective of engagement with AI. Hold the line rather than reorienting to the new possibilities of AI?

Simon talks about some educators' refusal to engage with AI as a legitimate response given pressures and conditions. And there are disciplinary considerations. But we might go beyond worries about misconduct but potential to “damage learning”.

Q What kinds of assessment are most at risk?

MB Weaker ones – single answer, right / wrong, questions that allow general (not applied / personalised / contextualised) answers.

Sarah: to some extent, things we could have answered with an internet search already

Sarah says: but there are interesting implications for arts too - what new practices might arise from this situation?

Discussion of levels (standards) of assessment rising - but does that further marginalise those not able to make effective use of AI?

Lucinda mentions changes at school in writing (e.g. to include images) and maybe Higher Ed also needs to embrace multimodal assessment more (though that might not solve AI worries as things evolve).

Sarah: Developing new assessments is labour intensive and needs expertise. (Are we back to the resources / workforce / scalability problem that seems to be at the heart of worries about generative AI – yes, Lucinda now talking about casualisation, etc.)

The question is hinted at of whether effective, laborious prompt engineering is equivalent to doing the work of writing. But even if it is equivalent in terms of effort or even knowledge, does it fit with the kind of knowledge we want (and the purpose, as Margaret now says)?

Q. Does AI generated feedback information get in the way of educators knowing students?
Simon: if quality of info is good enough or equivalent, he can’t see the problem.

Is he missing (or isn’t worried about) the relational element of education? What will @r_ajjawi say?? 😂

Here's Rola suggesting there’s more going on than the information. Don’t teachers need to be involved in the process and dialogue?

Simon: No worries, they can have a conversation with the AI. It's about a system not about individual teachers.

Rola not delighted, I'm guessing.🤔

How to design teaching for a world of ubiquitous AI?

Sarah: The future world might not be about AI or any tool. We can’t focus on specific things but adaptive capacities for uncertainty, change and risk.

Lucinda: ethics will be a big part of this

Rola: what are the 3 next steps you’d take as an educator?

MB: thinking about assessing process vs product. Going back to that question “what is it that we need people to know?” For more immediate changes, critically look at your rubrics

Simon: use AI's capacity to reflect on itself to help you and students develop AI literacy. Learn the limits. Practice.

Lucinda: to stay safe, know basic principles, limitations, terms and conditions, copyright implications. Be careful and ethical.

There endeth another ChatGPT webinar that was very good but that also reinforced the complex, multifaceted and ever-changing nature of the challenges of education. 👋

If you read this and thought "I wish that thread was better", this one from @DrJoannaT is for you:

https://twitter.com/DrJoannaT/status/1630765581535412224

And here's the recording:

https://twitter.com/TEQSAGov/status/1631442255960756224?t=YaSRq6KEFH68qxJkRgoh0w&s=19