Thread by @sh_reya

Recently a GPT-3 bot said scary things on Reddit and got taken down. Details by @pbwinston: kmeme.com/2020/10/gpt-3-…

These situations create fear around "software 2.0" & AI. If we want to incorporate intelligent systems into society, we need to change this narrative. (1/8)

There’s no doubt that GPT-3 returns toxic outputs and that this is unsafe. But GPT-3 is a black box to most, and fear is triggered when the black box deviates from an average person’s expectations. When I read the article, I wondered how we can calibrate our expectations. (2/8)

I did a small grid search with various parameters on the first prompt, “What story can you tell which won't let anyone sleep at night?” Results are here: docs.google.com/spreadsheets/d… My grid search code is here: github.com/shreyashankar/…. Don't blow through your API credits, lol. (3/8)

You’ll notice that davinci, the model with largest capacity, has more concerning outputs. You’ll also notice that higher temperatures yield more concerning outputs. Not all hyperparameter choices yield concerning outputs -- in fact, most don't. (4/8)

It is mind-boggling to me that the hyperparameter choices are obfuscated from people who view model outputs. Second, why aren’t model outputs annotated with the log probs? Third, we need education on programming these intelligent systems and releasing them “in the wild.” (5/8)

Like SSL for networking, we need a community-trusted verification process to release systems built on top of GPT-3 and other intelligent systems. We also need to communicate this with end users through thoughtful UI/UX. The SSL analog is a "lock" icon in the URL bar. (6/8)

I am happy that OpenAI’s UI to experiment with GPT-3 is thoughtful. Log probs are annotated with colors and toxic outputs are flagged. They advise programmers not to publish sensitive outputs. But this is only within the playground and doesn't scale. (7/8)

If you’re releasing intelligent systems for people to interact with, you have a moral responsibility. Educate people on how to use your tool (require training). Have an opinionated framework that prioritizes safety (require open-sourced code, priming examples, and hparams). (8/8)