Let's talk about the differences between novices and experts. But, instead of cyber security, we'll use airport baggage screeners as an example. These are the folks who use the scanner screens to find forbidden items in luggage 1/
We all expect that experts are faster than novices. That's often correct, but WHY? 2/
Experts go through a few steps when looking at a bag image. First, they perceive the whole image quickly, looking for something to draw their attention. Maybe a dark spot or an unknown pattern. This holistic analysis is nearly automatic. 3/
If the expert sees something interesting, they focus on it. They examine specific features and compare those to things they already know about to see if they can figure out what it is. Basic pattern matching. 4/
Experts have large libraries of examples (heuristics), and they go through them quickly due to frequent recall. They have more patterns and process the key features of them well. For example, recognizing hair dryers by the heating coils -- a consistent and unique pattern. 5/
Ultimately, if the expert exhausts their library of examples they can then pull the bag off the belt and see what the item is. It's a great learning system because there's always quick feedback. Mental example libraries build quickly after thousands of bags a day. 6/
Now, consider the novice. The biggest difference is in their first step. They don't have the ability to take in the image as a whole yet. Since they don't yet understand normal they can't spot broad classes of anomalies so easily. Sound familiar? 7/
So, the novice must systematically break down the contents of the bag. In this serial analysis they visually inspect each item one at a time. It's a result of both having fewer total patterns, and not knowing the most common and easily recognizable features of those patterns. 8/
The serial analysis is important, but its the ability to perform a holistic evaluation that's a crucial difference here. It's altitude. Spotting things at 10K feet and knowing when to swoop down to ground level. 9/
Not for nothing, it's also how experts can determine when dangerous items are in bags, but broken into multiple pieces. Holistic analysis allows for assimilation of multiple data points across space better. 10/
An issue often arises here when novices try to perform holistic analysis but aren't capable. It's paralyzing, and slower than approaching it serially. Eventually, novices shift their techniques, often without even recognizing it. 11/
The same shift happens in cyber security, but usually on a per-evidence source basis. Eventually, you build enough of a heuristics library on PCAP that you can perform holistic analysis instead of going line by line. Same for disk artifacts, memory, etc. 12/
It's important to recognize that serial analysis provides three valuable things.
1) a mechanism to get the job done when someone lacks expertise.
2) a mechanism that helps build expertise.
3) A fall back for experts in times of confusion. 13/
1) a mechanism to get the job done when someone lacks expertise.
2) a mechanism that helps build expertise.
3) A fall back for experts in times of confusion. 13/
Security analysts are also going to use serial analysis at first. So the questions become.... How do we optimize data presentation to support it? How do we recognize when folks move beyond it? How do we transition them into holistic analysis? How do tools support these ideas? 14/
This isn't just baggage handlers. This pattern is also repeated in cognitive task analysis of other fields relying on pattern recognition. Radiologists are another great example. Same brush strokes, different canvas. 15/
Remember, while the canvas of infosec is still relatively new, practitioners in many other fields paint in similar ways on their own canvases. A lot to be learned. 16/16
