Riley Goodside Profile picture
Aug 17 13 tweets 10 min read
A novel(?), powerful method of complete-program synthesis using GPT-3, generalizing the “format trick” @BorisMPower showed me to combine instructions with contextually informative templates. This is a single temp=0 generation without cherry-picking, truncated here due to length. ImageImageImageImage
It’s like f-strings, except the English goes on the inside and the Python goes on the outside.
A second example, returning the lowest palindromic Fibonacci number greater than a given n: ImageImageImage
Example 3: GPT-3 generating a PyGame implementation of Tic-tac-toe using given class/module structure and libraries. Only selections of output are shown here due to length. ImageImageImageImage
Note the inclusion in Ex. 3 of “Implement all functions completely.” Otherwise for longer generations, it’s inclined to cheat by writing function bodies with “# TODO: Implement this” if directions are extremely high-level as in the end of Example 3.
Example 4: Synthesizing example CSV data and related Python code in a single generation. Playground link: beta.openai.com/playground/p/x… ImageImageImage
Example 5: Generating inter-related NDJSON, Python, and HTML simultaneously. Ringo Starr truncated due to length. Playground link: beta.openai.com/playground/p/7… ImageImageImageImage
Example 6: Generating a Python script that creates synthetic CSV data, following a prescribed class format. Some enum subclasses in output omitted due to length. Playground link: beta.openai.com/playground/p/b… ImageImageImageImage
Note in Example 6 we import an entirely imaginary function, `synthetic_name.get_name()`, and the generated output uses it as intended.
Example 7: Using the OpenAI API to generate code that uses the OpenAI API. Playground link: beta.openai.com/playground/p/R… ImageImageImage
Example 8: Generating hash-modulo binning algorithm from docstring description. General theme here is “I know exactly what I want, but I forget the import names.” Trust the model only to translate idioms/syntax. Playground link: beta.openai.com/playground/p/K… ImageImageImage
Example 9: Posing an assertion-satisfaction puzzle to GPT-3 using a trivial instructional template. Playground link: beta.openai.com/playground/p/R… ImageImage
Example 10: Cross-language synthesis in Python and R. It’s conceivable this works better than traditional translation, as code is generated from a template summarizing high-level intent. Playground link: beta.openai.com/playground/p/G… ImageImageImageImage

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Riley Goodside

Riley Goodside Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @goodside

Aug 11
A demo of GPT-3's ability to understand extremely long instructions. The prompt here is nearly 2,000 chars long, and every word is followed:
An even more extreme example of GPT-3 task-following with a ~5k-char prompt — searching for a needle in a haystack of "Don Quixote":
Another example, "Bobby the Dragon", a 2,238-character GPT-3 prompt testing reading comprehension and translation. This result is cherry-picked — small errors become common at this length. OpenAI Payground link: beta.openai.com/playground/p/p…
Read 4 tweets
Aug 10
A GPT-3 failure mode I call “speaker confusion”: instead of answering, the model unexpectedly interprets the instructions as incomplete and attempts to finish them. Note how subtly different phrasing resolves the issue in the third example. ImageImageImage
An easy fix for this is to end zero-shot prompts with “Answer:”, “Solution:”, etc.: Image
A prompt is only reliable to the extent it’s obvious that the prompt is complete, and that an answer immediately follows.
Read 4 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

:(