Thread by @JanelleCShane

Need a Halloween costume idea? Here are some ideas generated by a neural net
aiweirdness.com/post/188342947…

In 2017 I tried to generate Halloween costumes with a neural net that didn't have any prior knowledge of English words. Its costumes often made no damn sense.
aiweirdness.com/post/166814009…

In 2018 I tried generating Halloween costumes with a slightly bigger dataset, but the neural net still couldn't do more than remix letter combinations.

But @susanthesquark of @nytimes arranged for a Professional Illustrator & Jessia Ma nailed it:
nytimes.com/interactive/20…

@susanthesquark @nytimes My 2019 Halloween costume attempt uses the same dataset, but a neural net (GPT-2-355M) that has picked up a lot of context from the internet.

This is most obvious here. It grabbed words like bitcoin & titmouse that weren't in the example costumes. Compare to 2017 (conchpaper??)

@susanthesquark @nytimes When I increase neural net sampling temperature, the output conforms less closely to the training data.

In 2017/2018 that meant more nonsense words.

The 2019 neural net reverts to its internet training data, introducing more Game of Thrones, Frozen, and racism. Oh, internet.

@susanthesquark @nytimes It is super interesting to see the form chaos takes in a small vs huge neural net.

Unfiltered text from 2017 char-rnn vs 2019 GPT-2, both trained on the same dataset of halloween costumes & sampled at chaotic 1.4 temperature.

reverting to nonsense vs to celebrity references