DIY: In-house Threat Detection Engineering

As organizations evolve in terms of detection & response capabilities, more than a decade old SIEM remains an enterprise security must, acting as one of the main platforms within a cyber defense program.

Despite being overshadowed by easy to justify initiatives like vulnerability management and other prevention controls, investments in SIEM will be among the fastest-growing, with Security Analytics playing an important part.

As F-Secure's Mikko Hypponen says:

Every company is a software company.

In addition to in-house built, custom applications, for pretty much every new box or technology an enterprise brings in, it is likely producing new data (DB records, logs) which is the 'raw material' for widening the detection surface.

Given the amount of non-standard data types and distinct threat models or different priorities seen in enterprise environment, it comes as no surprise that organizations should not rely on vendors to come up with ways to leverage that data for detection.

In other words, it's rather naive to assume or expect that a set of out of the box rules shipped with a product (aka canned content) will be able to fulfill even the basic needs when it comes to detection coverage.

SIEM as a development platform

If you follow closely what's going on within #blueteams and #threathunting related forums (Twitter, Slack, etc), it's easy to notice a clear practice being built on top of regular security engineering (device tuning, etc).

With Splunk and ELK being the most prominent platforms, users are building new detection rules as new threats are just uncovered — drastically reducing the time it would otherwise take when waiting on a vendor update.

And that's not only because those platforms can easily ingest different data types, but because the output of closed research or intel (honeypots, threat modeling, etc) in combination with basic stats/coding skills extracts a value that was never seen before.

What is Threat Detection Engineering?

What we are seeing now is a shift from simply deploying and tuning boxes, which isn't commonly done in a continuous basis, but after professional services engagements; to continuously design, develop and maintain new detection mechanisms, a practice that suits eager, mature internal teams.

Don't get me wrong here, device tuning is still needed but it may be just one small step in the process. There's a whole SIEM use case lifecycle to be carried out until a consolidated security alert gets generated.

In a nutshell, Threat Detection Engineering is related but not limited to the following high level activities:

  • Drive proactive identification of threats to the environment with rapid deployment of detection controls.
  • Liaise with internal teams and especially data owners, establishing relationships focused on spotting new detection opportunities.
  • Make threat intelligence consumable by integrating it with SIEM and other tools part of the security arsenal (SOAR, EDR, etc).
  • Create custom code to aid the detection of malicious activity via security alerts (rules), reports (dashboards) or simply automation (scripting).
  • Establish a continuous quality assurance process, with special attention to internal users/customers feedback (SOC, monitoring teams — in special).

I plan to write about an ideal team structure (roles/responsibilities) for supporting orgs willing to establish their Threat Detection Engineering practice. Some teams are already taking this step, a few examples below:

How much does it take?

Engineering is about the best change given the resources available. As it applies to any other practice that involves some sort of craftwork, you need skilled people — or people eager to learn, curious minds.

In times when it's quite hard to find talents in 'Cyber', the strategy should be based on building team awareness and mentoring new comers so that enablement is at the core.

The main advantages, besides the ones already mentioned are the following:

  • Full ownership and responsibility over the development process.
  • Prioritization of new detection ideas is tackled the best way agreed within the team, without relying on a vendor update.
  • Detection based on custom data (ex.: that critical business application logs) is not a blocker anymore.
  • The intel/experience gathered during this process helps mature the team and enable easier content migration (ex.: SIEM replacement).

Blueteamer. Threat Detection Engineering & Research.