Second AV Study Reveals Small Window For Catching New Malware

By Ryan Murphy – 

When antivirus doesn’t detect a piece of malware on day 1, it’s sure to detect it later on, say…day 30, right?

Right?!

In March 2012, we conducted a study to see if there was any benefit to using the signatures of multiple AVs simultaneously. We downloaded 84 random malware samples from malc0de.com and submitted each to VirusTotal to see what each antivirus product had to say about them.

The answer, we found, was a resounding, “Yes, leveraging the signatures of multiple antivirus products would detect far more malware faster.”

(The answer was so resounding that we decided to build a VirusTotal plugin into Carbon Black allowing its users to do exactly what we did only with EVERY binary on their machine.)

However, there was a piece of data that came out of that study made us do a double take.

In addition to finding that using the signatures of 43 AVs simultaneously provided far greater detection power than any single AV’s signatures alone, we found that the average new detections per day dropped to nearly zero after day 6.  What this means is that, on average, if AV doesn’t detect a piece of malware almost immediately, it likely never would.

Some questioned our results. We skeptically questioned them ourselves. Why was it that if AV missed a malware sample on day 1, the sample was more often than not missed again on day 2 and then again on day 3 and then again…on day 30?

Was it just our sampling? We were using publically known malware samples, freely available on malc0de.com. The results confused us. They concerned us. And we weren’t the only ones. So we set out to test this finding a second time to see if it was all just a big fluke.

We ran the same test for 30 days again, using 90 recent samples from malc0de.com. We submitted each malicious sample to VirusTotal to scan with 43 different antivirus products to see if their detection power increased over time as originally expected. Like the first test, we didn’t really care which singular AV was the best among the group, we just wanted to know two things:

1) if the signatures of all AVs collectively were considerably better than using any one signature set individually
2) if over time it was reasonable to expect each piece of malware to be detected by all antivirus products.

Again we found that scanning all malware samples with 43 AVs resulted in, arguably, a 100% detection rate on day 1. (Arguably because on day 1, at least one of the 43 AVs detected 87 of the 90 samples as malicious, leaving three undetected. On day 30, the same 3 samples were undetected by any antivirus product.  This is not to say that these 3 samples were not malicious, just that all 43 antivirus products agreed that they weren’t.)

Number of Malicious Samples Detected by Each AV’s Signatures

As was pointed out when we conducted this study the first time, individual AV results vary based on configuration.  Also, we did not include any of VirusTotal’s new sandboxing results in the most recent study so the results, just like the previous study, are limited to static signatures.  Therefore, you cannot use the graphic above to conclude “AV product A is better than AV product B.” The purpose of our studies has been to answer the question: “If I could leverage the signatures of the entire AV industry, would I be any better off in detecting malicious software?”  And the answer is undoubtedly, again: “Yes. Much better off.”

Remember, we’re not doing AV testing. We’ve found that there is no way to predict which AV will have the best signatures for a certain malicious sample.  So we say, “why not just use all AVs at once?”

Undetected Immediately May Mean Undetected Forever

In the graphic below, you can see that the data confirms our original finding that if the sample was not detected immediately within the first week, it, more often than not, was not detected on day 30 either. The delta between day 1 and day 30 is nearly zero, again not what any of us expected.

But maybe there’s a good reason for this. In 2010, Symantec encountered more than 286 million unique variants of malware. That’s 783,561 per day on average – an insurmountable number of samples to analyze.  This is analogous to trying to drink from a fire hose running on full blast – you’re able to catch a good number of the malicious software samples but many pass by without the time to fully inspect.

These samples can’t be queued up for the next day either because another 783,000-plus will be coming in. With such inundation there’s little you can do besides hope that these samples reappear in the next day’s feed.

Perhaps our data is confirming other reports that after one week, malware authors have moved to a new variant and therefore, their old samples are no longer included in the daily feed.  Unfortunately for you though, this means that if your AV doesn’t detect a particular sample within the first week, it might never detect it – no matter how dated the sample becomes.

Less Detection on Day 30?

Another anomaly present in both studies is that some AVs actually detected LESS on day 30 than on day 1 – something we, and others, did not expect either. Perhaps there’s a good reason for this also.

Let’s assume that a single signature can detect 100 malware variants. If so, one would have to write 7,835 signatures per day just to handle the 783,561 malicious samples being reported. These signatures will accumulate over time, requiring an AV to check each newly created file against an ever-growing list of signatures, which dramatically slows a user’s machine down to a crawl.

As a result, AVs must keep their signatures small and relevant, perhaps needing to remove an old signature for each new one added. Although we can’t guarantee this to be the case, it’s certainly a valid hypothesis as to why certain AVs detected fewer of our samples on day 30 than on day 1.

So What?

According to the most recent Verizon Data Breach Report not only was malware responsible for 95% of stolen data/records, the attackers began exfilling this data within hours of the initial compromise. Although reports like this refer to “records” and “data” as inanimate objects, they are the lifeblood of your company. Lost intellectual property in the hands of the right competitor could mean bankruptcy; lost personal data almost certainly means fines and other penalties. In a malware attack time is of the essence, and no single AV gives you adequate protection.

Help is Here

If the moral of this story is that you cannot rely on a single AV to protect you, and running multiple AVs can be problematic at best, why not use Carbon Black with our VirusTotal plugin to leverage the signatures of the entire antivirus industry?

Using Carbon Black and our methodology, you would have detected all malicious samples on day one! With the majority of records being stolen by malware, and data exfiltration beginning within hours of the initial compromise, you owe it to yourself to inquire about Carbon Black or try it for yourself.