Thanksgiving in the US is upon us.

With that in mind, _easily_ my favorite new open-source tool of this year is semgrep. (… and

Thread: A few thoughts on why ...

Background: I'm in a fairly small group of people who have actually built a static analysis product. I believe in the technology. I also know the problems.

I think Semgrep made 2 fundamentally clever decisions:

1. They didn't try to solve the whole problem.

How do you want to solve the classically difficult data flow "hard problems"? Do you want to solve XSS by finding all the vulns and fixing each individually? Or ... do you want to use a modern framework and CSP?

Do you want to solve sqli by fixing each, or enforcing input validation and using bind params? The former needs big static analysis. The latter can be done with a much simpler/faster tool.

The folks at r2c (and facebook previously) certainly have the ability to ability to solve the deep problems, but they took the right modern customer-focused pragmatic approach IMO.

2. They flipped the traditional model of "find all bugs" to "define and enforce invariants"

It's difficult to overstate the importance of this intuition practically. Static analysis products are difficult in large part due to rule management.

The core tech is ~straightforward, but supporting every language and framework is a never-ending uphill battle. It's also assymetric. The product company has to support _every_ lang/fwk for every product to provide an out-of-the-box experience.

By moving to invariant enforcement, you as the customer or even application get to choose what langs/fwks you want to use, and then ensure that no-one uses anything but those.

My typical example here is if there are 10 sql ORM frameworks in your language that are popular, you're only using 1. The static analysis vendor needs to support and model safe/unsafe apis in all of these libraries.

You need to support 1, and then just ban all the others. This is focused on the "paved path" idea I talk about often.

note: @mmadou was prescient in this space, and had this idea early on.

Further, Semgrep (IMO) has finally solved the developer UX. Working in just _feels_ right. Copy-paste your code and start writing your rule (with some ~auto-complete capabilities). This is objectively better than anything else I've used.

Also, this is a tool for many uses. I use it for security, but given it's invariant enforcement, the paved path is not just for security - it's for all of dev.

This helps us have a centralized tool to speak the same language as devs. It also means devs don't have to understand arcane custom rule formats.

Finally, I love/build/support open-source, and they are sharing this tool with the community. The team is also really friendly and responsive. I have high hopes.

I remember having a conversation almost ten years ago now with @DinisCruz where we (passionately) discussed having an open-source static analyzer. I feel like we finally have one, and I like it.

TL;DR Like all tools, semgrep's not perfect, but it's genuinely really great. This is as close as I get to fanboy.

16/16 (/fin)

/cc @clintgibler @dlukeomalley @0xine
/cc @yoann_padioleau thanks for your hard work!

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with John Melton

John Melton Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!