Threat Hunting: The Thrill of the Hunt

June 20, 2017 / Ryan Murphy

(Editor’s Note: The text appears in the free eBook: “Threat Hunting for Dummies.”)

When the threat hunting team and tools have been acquired and trained, it’s time to go hunting. This blog explores the thought processes that prepare a threat hunter for a successful hunt, as well as a proven methodology for threat hunting called the Hunt Chain. Created by Carbon Black, the Hunt Chain methodology depicts the entire threat hunting process.

The Mentality of the Hunt

Maybe you’ve been familiar with operating system (OS) internals for a long time, and you’ve been inside the proverbial machine. Maybe you’ve even written some of your own tools and exploits. You’re reading about cybercrime and what hackers do these days, and you’re mad as heck, and you’re not going to take it anymore! It’s time for the chase and to put cybercriminals on the defense. Make them run.

This description is the mental attitude of a threat hunter: He / she knows how systems work, how attackers think and act, and how to use tools to go after them, find them, and kick them out. Your organization has its weak spots — sure. Every company does. That may give cybercriminals an easy way in, but it doesn’t give them the right. You know where they’ll strike, and you’ll be waiting for them.

The primary objective of threat hunting is asset and information protection through the following:

✓ Knowledge of systems, networks, exploits

✓Knowledge of the enterprise applications, how they work, where the treasures are, and how the data flows

✓ Knowledge of endpoints, how they work, and how they’re used

Week in and week out, a threat hunter adds to his / her knowledge, skills, and tools. With the right tools, such as Cb Response, each new query becomes another automatic threat detector, so the hunter slowly gains ground and denies attackers access to more and more attack surface. That way, a threat hunter needs to never hunt for the same thing twice. But at the same time, attackers’ tools improve, and more exploits are discovered, so it’s a tug of war between threat hunters and their adversaries.

Constantly reading and learning about new exploits, threat hunters test out new hunches and see whether attackers are trying these new techniques and, if so, what they look like.

Planning for the Hunt

For the first few weeks of threat hunting, a threat hunter becomes oriented to the environment and masters the tools
used and how they’re configured. Soon it will be time for the threat hunter to venture out on individual campaigns — probing deeper and further than before.

The overall practice of threat hunting is indeed continuous, but it’s broken up into individual missions called hunts. A hunt can last a few hours to several days — it depends on the objectives of the particular hunt. A hunt should have one or more objectives — narrowly focused at times, but not too broad either (or it might not ever really get completed). Some example hunt objectives include the following:

✓Hunting for specific exploits: A threat hunter may have read about some specific new exploits, such as Locky, and will look broadly in the environment for signs of it.

✓ Hunting for attacks against specific vulnerabilities: A threat hunter dives into high‐value systems with one or more known unpatched vulnerabilities to see whether attackers are attempting to exploit them.

✓ Hunting for attacks against specific high‐value targets (HVTs): Here, the threat hunter dives deeply into the operation of a specific asset (or a small number of them), learning more about how it operates and looking for signs of reconnaissance or intrusion.

Threat hunters generally concentrate their attention on endpoints with tools such as Cb Response, which provides detailed forensic data on endpoints. Depending on the hunt’s objective, the threat hunter may be triangulating attack evidence by using additional tools, such as an intrusion prevention system (IPS), web proxy filter, or next‐gen firewall to identify signs of compromise.

Threat hunting is not only about detecting malware but also the abnormal usage of legitimate tools (such as PowerShell and EMET) and accounts.

Keep notes on your threat hunting experiences. Over a long period of time, hunts may all become a blur, but with good records, you can go back and familiarize yourself with past hunts. These records might be highly structured and include hunt objectives, logs, traffics, activities searched for, and analytics. Or they might be more like a narrative describing a hunt.

In the future, if you embark on a similar hunt, you could peruse your records and use them as a springboard.

The Carbon Black Hunt Chain

Carbon Black developed a methodology called the Hunt Chain, which is a series of activities that constitutes a formal threat hunt. The overall chain is depicted below. This section explains the different aspects of the Hunt Chain.

Where and how to start

A threat hunt starts with the collection of data that’s directly or indirectly related to its objective. When developing an objective, the threat hunter needs to know what data will be mined in order to achieve the hunt’s objective.

Define objectives and the scope for a hunt before the hunt begins to quantify success and know when the hunt is
completed. Without clear objectives, a hunt is more of a fishing trip that could go on and on.

Filtering out legitimate activity

As threat hunters begin observing the target environment, they begin observing activities. By using their knowledge about the OS and application(s) in the target environment, they begin to filter out legitimate activity, leaving only anomalous activity to investigate. One by one, as those activities are explained, all that remains, if anything, are attackers and their actions.

Hunt for suspicious activity

During the hunt, the threat hunter observes data and filters out known legitimate activity. Anything that remains could be suspicious. For example, an organization might utilize PowerShell as a part of its endpoint management tools. PowerShell is a command line shell and scripting language; you could liken it to the new and improved version of command line and batch files. A threat hunter can use this knowledge to filter out all of the organization’s legitimate use cases for PowerShell. If any uses of PowerShell remain, they either belong to additional legitimate use cases or attacks. Remember that threat hunts don’t always turn up activity indicating intrusion.

Deeper investigation

Activities that remain unexplained are investigated further. The threat hunter may need to solicit help from experts on the OS, applications, data flows, use cases, or other aspects of the anomalous activity. Oftentimes, the threat hunter discovers aspects of legitimate activities that were previously unknown.

Sometimes the threat hunter discovers aspects of an environment that represent improper implementation of a system. For example, a threat hunter may find persistent temp files containing credit card numbers, where the files were supposed to be encrypted but weren’t. This may have been considered an artifact of an attacker scraping credit card numbers out of an application.

This portion of the Hunt Chain is iterative; as threat hunters investigate anomalies, they filter out legitimate activities and then resume hunting for illegitimate activities.

Scope the impact

When anomalous activity is observed and confirmed to be an attack, the threat hunter continues to investigate to see where and how the attack originated and proceeded. This is essentially a root cause analysis, which — depending on the attack — may narrow into an initial intrusion, but it may also branch out into an investigation into what could be a broader attack on more systems.


After the total extent of an attack is known, the threat hunter — often together with appropriate colleagues (systemengineers, network engineers, security engineers, software developers, and maybe others) — contributes to the remediation effort. The specific activities vary, depending on the nature of the attack, but the general principles are:

✓ Remove malware and restore all altered and removed files to their original state

✓ Update configurations, permissions, and software versions to prevent a similar attack in the future

✓ Apply security patches to prevent similar attacks

✓ Update defenses

The organization needs to update its defenses so similar attacks require greater effort on the part of attackers. Updating includes automating systems to look for what you found. The range of activities may include:

✓New or updated firewall and IPS rules

✓New or updated alerts in a security incident and event management (SIEM) system

✓Improved incident response procedures

✓Updates to infrastructure, application, or security architecture

✓Changes in application development, testing, quality assurance (QA) or quality control (QC) tools, and processes

✓ New alerting rules in Cb Response or similar endpoint detection and response tools

The investment in threat hunting tools and personnel is mostly wasted if there isn’t a feedback loop incorporated that illuminates lessons learned and updates defenses. A threat hunt doesn’t just find outside attackers; insider threats can also be discovered in a threat hunt. A traitor is every bit an enemy as an outside adversary.

The results of a threat hunt will also give the threat hunter a lot of ideas for future hunts. If you’re fishing in a pond and find a hot spot where fish are biting, you’re going to go back to that spot next time.

TAGS: Carbon Black / Cb Response / Hunt Chain / threat hunting

Related Posts