The FUZZING'22 Workshop is organized by
* Baishakhi Ray (@Columbia)
* Cristian Cadar (@ImperialCollege)
* László Szekeres (@Google)
* Marcel Böhme (#MPI_SP)
Baishakhi (@baishakhir) is well-known for her investigation of AI4Fuzzing & Fuzzing4AI. Generally, she wants to improve software reliability & developers' productivity. Her research excellence has been recognized by an NSF CAREER, several Best Papers, and industry awards.
Cristian Cadar (@c_cadar) is the world leading researcher in symbolic execution and super well-known for @KLEEsymex. Cristian is an ERC Consolidator, an ACM Distinguished Member, and received many, many awards, most recently the IEEE CS TCSE New Directions Award.
László Szekeres (@lszekeres) is passionate about software security where he wages Eternal War in Memory (SoK). He develops tools & infrastructure for protecting against security bugs, like AFL's LLVM mode or the Fuzzbench fuzzer evaluation platform.
Yannic Noller (@yannicnoller) works on fuzzing and automated program repair and is interested in software reliability, trustworthiness, and security. Yannic was also named as Distinguished Artifact Reviewer at ISSTA'21 and will organize artifact evaluation for our Stage 2.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
There are tremendous opportunities to improve the way we disseminate research in Computer Science. Our current approach is to ask three experts to decide: Accept or Reject.
Here is what's wrong with this publication model 🧵
1/n
1. Providing feedback when the work has already been completed is utterly ineffective. What do we do if reviewers point out flaws in the eval or experiment design? Cycle it through our confs & journals until we are lucky. There is no consistency among reviewer expectations.
2/n
2. Strong focus on results. Papers with ε-novelty are accepted if the results are super-strong even if there is no convincing evidence that the results are due to the paper's contribution. This focus leads to duplicated efforts & overclaims, and ultimately impedes progress.
3/n
I asked #AcademicChatter about incentives & processes behind paper machines (i.e., researchers publishing top-venue papers at unusually high rates).
This is what I learned 🧵
TL;DR: Any incentive emerges from our community values. It is not "them" who needs to change. It is us.
It was tremendously exciting to get so many perspectives from so many junior and senior researchers across different disciplines. This was only a random curiosity of mine but it seemed to hit a nerve. I loved the positive, constructive tone in the thread.
Let's get started.
2/12
Some of you raised serious concerns about academic misconduct. However, to keep the discussion constructive, let's assume researcher integrity. We'll explore alternative explanations and processes below.
3/12
YES! We need to present our plots on a log-x-scale. Why? mboehme.github.io/paper/FSE20.Em…
Two fuzzers. Both achieve the same coverage eventually. Yet, one performs really well at the beginning while the other performs really well in the long run. (What is a reasonable time budget? 🤔)
Nice! I agree, comparing *time-to-same-coverage* provides more information about fuzzer efficiency than comparing coverage-at-a-given-time.
For my new followers, my research group is interested in techniques that make machines attack other machines with maximal efficiency. All our tools are open-source, so people can use them to identify security bugs before they are exploited.
This is how it all started.
My first technical paper introduced a technique that could, in principle, *prove* that no bug was introduced by a new code commit [ICSE'13]. This was also the first of several symbolic execution-based whitebox fuzzers [FSE'13, ASE'16, ICSE'20].
Yet, something was amiss. Even a simple random input generator could outperform my most effective whitebox fuzzer if it generated inputs fast enough. To understand why, we modelled fuzzing as a sampling process and proved some bounds [FSE'14, TSE'15].
Kostya's keynote: LibFuzzer hasn't found new bugs in <big software companie>'s library. We didn't know why. Later we got a note that they are now using LibFuzzer during regression testing in CI and that it prevented 3 vulns from reaching to production.
In Chrome, libFuzzer found 4k bugs and 800 vulns. In OSS-Fuzz, libFuzzer found 2.4k bugs (AFL found 500 bugs) over the last three years.