Read on Twitter
Students in my senior seminar on #AIEthics performed an external audit of @OpenAI chatGPT and DALL-E, assessing these services for ethical and social impact across a number of adversarial tests. I’ve compiled some of the results below. 🧵
The class was divided into two teams dedicated to each service. The Scoping teams looked at ethical principles and social impact assessments for various use cases. The Testing teams ran adversarial tests to probe the system’s fidelity to those principles and rate potential harms.
This five-week project was inspired by the algorithmic auditing framework developed in Raji et al (2020). @rajiinio Student-written tweets below! dl.acm.org/doi/pdf/10.114… 
"When prompted to produce scholarly articles about a specific topic, chatGPT could not generate actual article titles or functional hyperlinks. Instead, it generated fake titles and false links."
"Every prompt given involved a topic which was written on prior to 2020, meaning ChatGPT was likely trained on such articles and, in theory, should produce these articles when prompted."
Testing found that most citations were fabricated, and none were entirely accurate.
Testing found that most citations were fabricated, and none were entirely accurate.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
