Thread by @chrissanders88

Folks often ask me about the most important data sources for network defense. That question usually requires some unpacking and winds up as one of these:

1. Most important for detection

2. Most important for analysis

3. Most important for career growth

1/

I wrote a whole chapter about choosing the best data sources for collection in my book Applied Network Security Monitoring. I distinguished between detection and investigative value, but I think I would approach that chapter a bit differently if I were writing it today. 2/

In Applied NSM I introduced something called the Applied Collection Framework. The gist was that you should assess your fears and risks to the network that you're defending and work backwards from that to identify important data sources. 3/

I'd still include that, but position it along with some other things.

First, I would provide a mental model for categorizing evidence sources. I do that now by defining evidence realms. There are six: application, disk, network, memory, threat intel, and friendly intel. 4/

Next, I would define detection vs. investigation value in more detail. There's obviously some overlap here, but they are two distinct functions. Finding anomalies vs. assessing the disposition of anomalies and using known events to discover unknown related events. 5/

For detection, the mechanism often dictates the data sources: packets for Suricata, various logs for Sigma, etc. Your risk profile dictates where you deploy the mechanism and how you configure it (rulesets, verbosity). 6/

Generally, I think this detection related collection is easier to figure out -- detection tools have limited inputs. 7/

For investigations, common attacker and user behaviors usually dictate sources: executions, authentications, downloads, etc. Your risk profile dictates where those actions are most impactful and where you prioritize collection efforts. 8/

Investigation related collection is harder to figure out because of more numerous and diverse inputs to human analysts. It's also just not studied as broadly. 9/

With investigations in mind, folks should start with a baseline of important data sources that are (nearly) universal. These are things that I believe every organization should collect to aid investigations, regardless of risk profile. 10/

For example, everybody needs to be able to examine evidence of process execution. I don't know of any specific risk profile that removes that need. Execution is pretty fundamental to malicious and benign behavior (and analysts often need to prove both dispositions). 11/

How you do that could come from a few different sources... For example, OS logs (Win EID 4688/Sysmon 1), EDR agent, or something else. 12/

When I refer to collection I'm really talking about the analysts capability to access data. That doesn't just mean proactively (like logs sent to a SIEM). It could also mean collection from a live system. You can use the Windows registry, prefetch, etc to prove execution too. 13/

It would be at that point I would suggest considering the Applied Collection Framework to assess risk and apply what you know about that to determining collection engineering priorities. 14/

I didn't have enough data and wasn't quite brave enough to identify a list of must-have investigative data sources when I wrote Applied NSM, but I am now. I maintain that list in my Investigation Theory course. 15/

For the last part of this question, it's about what data sources people should learn if they want to get a security analyst job or level up in the one they have. 16/

The simplest answer is to look at job postings for the types of jobs you want and see what they reference most. If you're already working somewhere, then try to understand what the more senior analysts use the most to answer investigative questions. 17/

If you want to focus on detection engineering, look at the most common detection tools (Suricata, Sigma, Yara) and focus on evidence sources that are inputs to those. 18/

If you want to focus on investigations, I recommend starting with at least one data source from each evidence realm I have listed. Some of those will be easier than others, but you need cross realm expertise. A lot of folks specialize too early. 19/

For example, using this one from every realm strategy you could focus on OS security logs, the disk file system, zeek data, a basic memory image, passive DNS, and listing installed applications. If you can navigate those (even minimally), you'll be in pretty good shape! 20/

Evidence is important because it's where analysts answer investigative questions and connect events to form timelines. And of course, a conclusion without evidence is only just an opinion. 21/21