Currently, buying decisions for security solutions are heavily influenced by the reviews and certifications they receive that measure product quality and effectiveness. These ratings, published by independent third parties, are oftentimes used as a barometer for how CIOs make buying decisions and whether they decide to go with one product over another. What CIOs don’t realize however, is that the sources they have been depending on for these “valuable” second opinions, are using outdated and inaccurate testing methodologies, and therefore, providing a false sense of security.

The current testing methodologies utilized by reviewers and independent third parties to verify that a product meets certain requirements mainly takes into perspective a small portion of the vendor's protection model related to prevention capabilities for malicious code. Security products are being rated against their ability to catch known viruses via signature based defenses. In addition, these tests do not take into consideration the vendor's proactive capabilities, either through heuristics or behavioral-based technologies. Thus, current malware testing does not reflect the vendor's actual capabilities to protect their customers from the most relevant security threats.

The main problem with this approach is that signature-based defenses are failing to protect companies against the onslaught of new threats that are classified by malware technicians each day by the thousands. In fact, the average infection rate in systems with up-to-date protection is 72 percent according to a study conducted by PandaLabs (http://research.pandasecurity.com/archive/Think-you_2700_re-protected_3F00_-Think-again.aspx).

Using these inaccurate testing methodologies, product reviewers are not looking at the entire picture and are only basing their ratings on a portion of the entire product’s detection capability. If the reviews are not all-encompassing, conducted inconclusively and/or neglect to factor in all aspects of malware detection and prevention, the ratings will be skewed.

So what is the anti-malware industry doing about this issue?

The industry is addressing this problem through the formation of a standards group known as the Anti-Malware Testing Standards Organization—or AMTSO—in which Panda Security is a founding member. The objective of the AMTSO is to promote standards and best practices for correctly testing and evaluating the effectiveness of anti-malware solutions on the market. A vast number of other vendors including Microsoft, IBM, McAfee and Symantec are also a part of this group because they all recognize that significant improvements need to be made in the review process.

With the formation of the AMTSO, we hope that reviewers and independent third parties adopt the best practices developed for testing and evaluating anti-malware solutions—taking into consideration all parts of a vendor's protection model and not just focusing on signature-based detection as the sole driver for product quality.

By adopting these standards, reviews will become more encompassing of the entire product’s security capability and will offer a more authentic performance rating. This will benefit CIOs in the long term as they will be purchasing products on the basis of actual protection capabilities and not a pre-conceived notion that users are protected by the signature module.

For more information on anti-malware testing standards and how you can stay abreast of emerging issues in information security, please visit: (http://www.takethepandachallenge.com/whitepapers.aspx)

Ryan Sherstobitoff,
Chief Corporate Evangelist, Panda Security

You do not have flash or javascript support.

Average (3 votes)

Reply