Share this page!

Enter URL or ID to Unroll

You can paste full URL like: https://x.com/threadreaderapp/status/1644127596119195649
or just the ID like: 1644127596119195649

How to get URL link on X (Twitter) App

  1. On the Twitter thread, click on or icon on the bottom
  2. Click again on or Share Via icon
  3. Click on Copy Link to Tweet
  4. Paste it above and click "Unroll Thread"!
  5. More info at Twitter Help
Ben Newhouse Profile picture
Ben Newhouse
@newhouseb
member of technical staff @openai, cofounder @aidkit_tech, former head of sync @dropbox, cofounded bubbli (acquired by dropbox), previously made yelp monocle.
Jun 19, 2023 • 4 tweets • 2 min read
I hadn't seen this paper before but it's pretty neat: it's a more formal version of "what can we do with a transformer" when we use (or don't use) Chain of Thought "reasoning" to traverse the layers of the transformer more than once to solve a problem.

I've been thinking... about tasks that are simple enough that each "step" could be done in one layer and where we could construct problems that require an arbitrary number of them. We could then use them to suss out the depth of a given network (read: GPT4) if we don't let it do CoT...
Jun 19, 2023 • 16 tweets • 4 min read
I've engineered a hand-crafted Transformer to do long-hand addition. All of the weights were hand-chosen by me. It is perfectly accurate.

How? Fall down the rabbit hole here: colab.research.google.com/github/newhous…

Short on time? Here's a summary 👇

1/ I'm fascinated by how LLMs do the magic they do.

A couple months ago, I reimplemented GPT-2 from scratch to understand Transformer architectures.

But with that behind me, I was left staring down 117M random numbers with little intuition as to what on earth they were doing.

2/
Mar 23, 2021 • 9 tweets • 5 min read
After a ton of cleanup and documentation, this is now fully open source:

The BLE radio: github.com/newhouseb/oneb…
The building blocks: github.com/newhouseb/alld…

Since I basically learned RF from scratch to build this, I spent a lot of time documenting, which I'll preview below... Moving top down...

This notebook describes the conversion from bits to radio waves in GMSK encoding and evaluates how good/bad my radio is: github.com/newhouseb/oneb…
Mar 10, 2021 • 8 tweets • 2 min read
I just built a Bluetooth receiver using nothing but a Xilinx FPGA and an antenna.

No ADC, no filters, no mixers, no AGC, not even an amplifier. Just straight RF into a SERDES port at 5Ghz.

Here it is discovering nearby devices ("I LOVE MINDY" is the nearby Particle Argon). ImageImageImage It's not a great receiver, but I'm stunned it works well enough to discover my neighbor's TV!

Written in nmigen, uses ~1.5k LUTs w/ no DSPs (and no CORDIC, fwiw). Packet checking and printing is done in pure hardware because I didn't want to fit a CPU in my dev feedback loop.
Jun 10, 2019 • 14 tweets • 3 min read
Think you understand computers? Which of the following is true:

1. ‘:' < ‘;’
2. ‘:’ > ‘;'

Correct answer: it depends! In a fresh install of MySQL, ‘:' < ‘;’. In Postgres, ‘:’ > ‘;'.

Follow my descent into madness (and Unicode collation) to find out why...

1/n First some context. I had a list of strings:

'foo:a'
'bar:b'
'bar:c'
'baz:d'

I wanted to select everything with a prefix of 'bar:' using only '>' and '<' (the reason is another story), so I looked up the ASCII table and found that ':' + 1 = ';'

2/n