Carbon Black & VMware Announce Expanded Partnership to Secure the Software-Defined Data Center (SDDC) Learn more

Defining Effective “Patterns of Attack” with Machine Learning

June 14, 2016 / Alex Baker

Ben Johnson’s been writing some important posts about moving beyond “Indicators of Compromise” (IOC), and instead focusing on “Patterns of Attack (POA). In this post, I’m going to discuss some research that we’re doing at Carbon Black to make it easier to find patterns and share them with the larger community.

This means getting down into the details of the pattern concept asking such questions as:

“What does a good Pattern of Attack look like?”
“How do we find them?”
“What makes them effective?

As Ben Johnson’s latest post explains, one of the most central qualities of a pattern is its level of abstraction – a pattern is neither completely specific nor completely universal. The best patterns will be general enough to verifiably identify related attacks, regardless of difference in their easily-changed details, without incorrectly matching legitimate activity.


For example, we know that simply tracking the hashes of known bad files (A, above) is not sufficient to provide a durable defensive advantage. We can use patterns to widen our scope and consider context. For example, Notepad itself isn’t scary, but when it executes PowerShell (B), that’s cause for concern. In fact, we might be able to generalize this insight further, and recognize when any humble document-oriented application has initiated a process that carries capabilities beyond what should be necessary (C). Of course, you can alert when PowerShell is run by any application at all (D), but that’s likely to create a lot of harmful noise.


Our patterns need to be general in the right way, too. For example, bad domains come and go, and blocking them isn’t very useful in the long term (A, above). Pattern B, on the other hand, is both more general and more specific than this approach: it doesn’t consider what address notepad is connecting to. The fact that Notepad is trying to access the internet at all is cause for concern. Once again, we may be able to generalize this insight to apply to entire classes of applications (C).

Note that it’s also easy to run afoul by being too general. For example, some malware creates a backdoor binary that calls home, and alerting on binaries that follow this pattern (D) might seem like a good idea. But Adobe Acrobat and its link-enabled documents could match this pattern and generate a fountain of false positives.

Carbon Black Enterprise Response provides a wealth of data, and a skilled analyst will have little trouble tracing and describing the kill chain of a discovered attack. Turning this knowledge into accurate, precise and generally-applicable patterns at scale is the opportunity in front of us.

Machine-Driven Pattern Finding

Fortunately, there are countless machine-driven methods for deriving patterns, and many are specifically designed to find the best divisions between classes of behavior. The Carbon Black R&D team is currently doing research in this vein; some promising general approaches include:

  • Sequence learning methods, which consider a set of events and find sequences that are highly correlated with particular outcomes (in our case, attacks).
  • Taint propagation algorithms, which can be used to learn about processes, binaries, and domains based on their association with known bad entities. When multiple sources of taint combine to make a given event worthy of suspicion, this represents a highly generalizable pattern that can guide future investigations.
  • Graph-based methods, which can abstract behaviors, allowing them to be compared in terms of their overall nature, rather than their specifics.
  • Clustering, which can be used to find common classes of applications, broadening the applicability of known patterns.

These approaches serve to help us find patterns, improve the patterns we’ve found on our own, and apply our patterns more broadly than the original pattern’s creator might have realized was possible. This is crucial if we are going to work together to share patterns. It’s not enough that I describe what happened to my organization. That description must be intelligently generalized into knowledge that can be applied elsewhere. These methods can help. After all, it’s really about effective intelligence.

Finally, note that machine learning can provide a crucial advantage in the game of cat-and-mouse between attackers and defenders. Adversaries are good at knowing how their opponents think. The patterns that are obvious to us are likely to be obvious to them. But machine learning gives us an inside track, finding patterns that are not intuitive to humans. If we can keep attackers guessing about how we’re detecting them, we can make them waste resources trying to evade us, until attacking is no longer an attractive proposition.

The Human Connection

Of course, algorithms alone won’t solve our security concerns. Rather, collaboration between human analysis and computational power is crucial. This takes several forms, including:

  • Extrapolation: patterns discovered by analysts represent valuable input for machine learning algorithms, which can serve to find higher-order patterns in data.
  • Visualization: raw data and derived observations can be presented to the user for further review, leveraging humans’ powerful visual pattern-finding capabilities.
  • Validation: analysts can provide domain knowledge to validate machine-derived patterns. Conversely, machines can provide statistical support for human-derived intuition.

One crucial approach to this human-machine collaboration will be to move beyond the raw events in our data, and instead extrapolate higher-order behaviors.

For example, suppose Process A creates a binary file. Later, that binary is executed and instantiates Process B. This represents an important relationship between Process A and Process B, one that might not be evident by glancing at the raw data stream.

Or maybe two processes communicate across a network. Or a binary could, via a sequence of intervening events, delete itself. These are interesting behaviors! By describing our event data in terms of these behaviors, we help to make the data more understandable and useful (for people and machines alike!). Our work aims to define these behaviors and express kill chains in these higher-order terms.

Understandability is crucial because the success of a pattern-based approach depends on human-to-human connection. It is not enough to build an algorithm that says: “Yes, it’s good” or “No, it’s bad.” We need knowledge that is understandable, usable and verifiable by other members of the community. By sharing this knowledge we can drastically increase the effort necessary to breach our collective defenses.

A Connected Perspective

Carbon Black Enterprise Response provides invaluable data that supports this vision. However, while our current document-based model is efficient and effective, it doesn’t always lend itself to the kinds of machine-driven pattern-finding that I’ve outlined. We’re working on new graph-based approaches to represent process data, which provide several benefits:

  • Finding related processes is extremely fast, and sets of relationships are readily visualized.
  • Higher-level behaviors are easy to find, and can be put back into databases as new relationships, ready for further use in machine learning algorithms
  • Emergent properties can be naturally propagated through graph representations, taking full advantage of the knowledge we have to derive additional insights


A graph-based visualization of Carbon Black Enterprise Response data

By representing our data in terms of its connections, we think we can make it easier for machines and humans alike to find, generalize, and validate patterns of attack.

This generalization is crucial. Indicators of compromise and other spot fixes will never keep up with a world of highly-motivated, dynamic attackers. Our “Collective Defense” depends on us sharing and applying these patterns at scale. We’re working to make that process faster, easier, and more effective.

To that end, the Carbon Black R&D team has been presenting details of our work at the regional User Xchange conferences, gathering feedback and looking at ways that we can bring our learning and technology to the broader Carbon Black community.


Related Posts