What does it mean to be a threat detection engineer?

The idea of this post came after a Slack chat with Ryan Long, a Sr. Security Analyst who had asked this very question highlighted in the blog title.

Ryan as many others is realizing there’s sometimes a very thin line between analysis (operations) and engineering when it comes to threat detection.

Update: a fresher view on this topic was highlighted below:

The dotted lines between Threat Hunting and Detection Engineering

Why is Threat Detection so trendy now?

Because the demand is higher? Because Cyber is becoming specialized? I wrote a blog post touching on this topic a few years ago. Let's start with some context first.

Machine Data & Modern Tooling

Logs, logs everywhere! Today we have too much data at our disposal, so much that we need to design data pipelines with routing, filtering and other pre-processing cycles before consuming it in the other end.

I remember working as an architect for a bank in the early 2000s when an item of my product assessments was "Does it provide logs?" Today, we ask "Does it ship JSON logs?", "Does it provide an API to extract logs?"

Any enterprise-grade system today must generate logs.

Besides AVs, what was actually detecting threats — as they happen?

Some years ago, data for detection was basically coming from the wire (NIDS, PCAP analysis) and Syslog — that was it! Going beyond that meant you were on your own, building a sort of log aggregator (shivers!) or custom agents.

The main reason (use case) for log collection used to be Forensics or Compliance. Who was proactively checking Windows Eventlogs 15 years ago? You can probably name a few of your heroes here.

Besides the lack of abundant log data, the ability to query on it was also limited to general programming languages or SQL.

Even some big names from SIEM industry have failed to provide easy and fast ways to question the data with very rudimentary interfaces.

I believe that's another big shift. IMO, platforms such as Elastic, Splunk and Microsoft's Log Analytics (Sentinel) leveraging script-like languages to ask questions to data makes a whole difference.

You've probably noticed I'm focusing on SIEM and Big Data.

Yara or Zeek (Bro) rule writing skills do matter — a lot, but the ability to aggregate and extract signals from these and other data sources combined is what brings threat detection to the next level (Security Analytics).

When these are in the hands of skillful teams generating the expected outcome, it's easier to assert "Detection starts where Prevention ends."

Or as Florian Roth beautifully put "Assume Compromise"! His tweet summarizes the shift in paradigm quite well!

Besides the obvious threat evolution, those are among the main drivers for the recent developments in the threat detection landscape.

Is a Threat Detection Engineer a Security Analyst/Engineer with focus on Threat Detection?

I used to write Snort rules, but that has accounted for a tiny fraction of my full-time job as a senior analyst, security hardening accounted for most. Still, it does qualify as a practice of Threat Detection Engineering.

Today, it can be very different, one can easily land a full-time job writing detection content. It also depends on how the team is structured or how the organization strategy is defined.

Some orgs designate a team responsible for deploying and maintaining (fine-tuning) a multitude of rather specialized security products or appliances, from WAFs, to EDRs, to SPAM (anti-phishing) filters.

What would disqualify those from being considered detection engineers?

Anyways, here's my attempt to define it:

A Threat Detection Engineer is someone who applies domain knowledge on designing, building or maintaining detection content in the form of detections generating indicators and alerts; or interfaces in the form of dashboards or reports supporting the security monitoring practice within an organization.

As a former security analyst, I keep saying that the security analyst's work is a continuation of the detection engineer's one, since many detections would never be feasible without exception handling granted in the end.

What's your take on that? Feel free to reach out or leave comments!