Trail of Bits is heading to the AIxCC finals! 🏆 Our Cyber Reasoning System, Buttercup, ranked in the top 7 out of 39 teams in the @DARPA AI Cyber Challenge semifinals @defcon 2024. buff.ly/4dGrdRz
@DARPA @defcon The AIxCC semifinals put CRSs to the test with challenges on real-world software like nginx and the Linux kernel. Buttercup took an early lead on the achievement board by being the first to patch vulnerabilities in nginx. buff.ly/4djtJ09
@DARPA @defcon Buttercup’s success wasn’t just about finding bugs; it was about vulnerability remediation, a scoring element worth three times more than discovery. buff.ly/4cedjFa
@DARPA @defcon Congrats to all finalists and a huge thanks to our engineers who poured their expertise and passion into creating Buttercup. See you next year!
@DARPA @defcon More about AIxCC:
DARPA's AI Cyber Challenge: We're In!
DARPA awards $1 million to Trail of Bits for AI Cyber Challenge
Today, we are disclosing LeftoverLocals, a vulnerability that allows listening to LLM responses through leaked GPU local memory created by another process on Apple, Qualcomm, AMD, and Imagination GPUs (CVE-2023-4969) buff.ly/48RDP68
Our PoC can listen to another user's llama.cpp session across process or container boundaries. LeftoverLocals can leak ~5.5 MB per GPU invocation on an AMD Radeon RX 7900 XT running a 7B model on llama.cpp, adding up to ~181 MB for each LLM query. buff.ly/41WKaLf
LeftoverLocals impacts the security posture of GPU applications, especially LLMs and ML models that run on impacted GPU platforms. It highlights that many parts of the ML development stack, specifically GPUs, have unknown security risks. Read more: buff.ly/3SjHYu0
The @raft_fi issue is complex and we're actively investigating it. We've offered to help their team however we can. Here's some of what we know so far:
Raft published the exploit details here:
tl;dr get the pool into an unbalanced state, repeatedly mint 1 share because `divUp` mints more than expected, then redeem elsewhere.mirror.xyz/0xa486d3a7679D…
We were the first team to help Raft fortify their codebase in a brief 2 person-week project. We found 8 issues, including several arithmetic issues, and reported the long-term improvements we believed were necessary. github.com/trailofbits/pu…