Share this page!

Brendan Dolan-Gavitt Profile picture
Twitter logo
Feb 10 • 4 tweets • 2 min read
I asked GPT-NeoX-20B a hundred arithmetic questions. It didn't get very many of them right (10/100), but it's almost spookier to me that it gets most of them *approximately* correct??
Full result table here; settings were temperature 0.1, top_p = 0, top_k = 0 gist.github.com/moyix/267d122f…
I also just added results and easier-to-parse data files for both the original and expanded (1000 questions) results. Don't have time to do lots of analysis right now sadly gist.github.com/moyix/ca4091f1…
But maybe one quick plot. Real vs NeoX answer for 1000 questions. Curious what those outliers down there are...

• • •

Missing some Tweet in this thread? You can try to force a refresh
　

Keep Current with Brendan Dolan-Gavitt

Brendan Dolan-Gavitt Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @moyix

Brendan Dolan-Gavitt Profile picture
Feb 9
Definitely was a mistake not to strongly isolate the build directories from one another. I wonder which package (or part of my own infra) wiped out all the package metadata :P
One interesting thing might be to put some canary files one level above the build dir and then yell loudly if those files are gone after the build finishes. Might shake out some fun "oops, building this package deleted my home directory" bugs
Right now all the official build infra (as well as the reproducible build projects) run builds in an isolated chroot, so I can easily imagine issues like this going undetected for a long while.
Read 4 tweets
Brendan Dolan-Gavitt Profile picture
Dec 29, 2021
It's under-appreciated how simple and elegant the OS X UI experience is. In just a single glance here I can learn absolutely nothing about where all my disk space went
It calls to mind the classic design principles of ed(1): "generous enough to flag errors, yet prudent enough not to overwhelm the novice with verbosity" Let's look at a typical novice's session with the mighty ed:
Another elegant touch is the way the UI gently keeps users on the path of righteousness. Open source weirdos may complain about their "freedoms", but I sleep more soundly knowing that Big Mac is watching out for me–there will be no rogue disk inventorying in this house! UI screenshot: "Disk Inventory X" cannot be opened
Read 4 tweets
Brendan Dolan-Gavitt Profile picture
Dec 22, 2021
The camera-ready version of our @IEEESSP 2022 paper, "IRQDebloat: Reducing Driver Attack Surface in Embedded Devices" (w/@highw4y2h3ll) is now available! messlab.moyix.net/papers/irqdebl… Screenshot of title and authors (Zhenghao Hu and Brendan Dol
The basic problem we're looking at in this paper is: if you buy some embedded/IoT device, it may come with a bunch of features that you don't use (say, Bluetooth) that nonetheless require driver support and expose unnecessary attack surface.
Maybe you're a company deploying a fleet of Meraki routers, you don't need the Bluetooth Low Energy localization stuff, and you're worried about vulnerabilities like this one arstechnica.com/information-te…
Read 19 tweets
Brendan Dolan-Gavitt Profile picture
Dec 20, 2021
Hmm, this is actually much less impressive than I expected as far as inverting PhotoDNA (based only on reading @hackerfactor's blog post) reddit.com/r/MachineLearn…
@hackerfactor @matthew_d_green perhaps of interest if you haven't seen it yet and want to take a break from fighting with half of CS twitter about NFTs ;)
Ah, I see, it's taking a pure black box ML approach to try and learn the inverse straight from the hashes. OK, that is pretty impressive!
Read 5 tweets
Brendan Dolan-Gavitt Profile picture
Dec 20, 2021
So, with Broadcom's acquisition of Symantec, it seems like the source code for PGP Desktop (aka Symantec Encryption Desktop) is nowhere on the internet? I have a copy but I'm pretty sure I can't host it anywhere: (d) give (meaning sell, loa...
Seems like a loss for archival and data recovery work! :(
FWIW, the version I have is:
MD5 (PGPDesktop10.0.1_Source.zip) = c9193850f923cda995e6d4b9f45fcbdf
Read 4 tweets
Brendan Dolan-Gavitt Profile picture
Dec 18, 2021
Probably getting old, I opted to just pay for a janky conversion utility rather than try to RE the Microsoft Outlook 15 message format :(

(I may still RE it)
The format is a pain in the ass, it stores messages in 3 undocumented binary parts: metadata, message body, and attachments. It has an sqlite database but that just points you to the metadata file.
Also, everything is referenced by GUIDs, which are in a mix of
- Raw binary GUID data
- ASCII GUIDs
- UTF-16-LE GUIDs
- Base64-encoded blobs that contain GUIDs
Read 4 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Like this author's thread?

Get emails when @moyix has new unrolls!

Check out other threads from @moyix!

Read all threads by @moyix

:(