In God We Trust, but Security Vendors Need to Sign the Papers

[#1]

In February, it was HBGary Federal.

In March, first, RSA Security was attacked. That same month, Comodo fell.

To add insult to injury, Veracode in April released its State of Software Security Report. The security industry, already reeling from repeated blows to its claims to be able to protect us—obviously, many security companies can’t protect themselves—were found to be one of the two worst performers when it comes to code quality. Veracode testing found that the code of 72% of security applications submitted over the previous 18 months was of unacceptable security quality, second only to the subcategory of customer support applications.

“Security should not be assumed on the part of any industry segment even when it is those producing software for a living—including security software,” the Veracode report warned.

In fact, some industries are slowly beginning to hold security (and other) software vendors accountable. They’re requiring proof of independent security verification and independent testing before they sign off on purchases, with the Finance and Software industries leading the way. Aerospace and Defense are catching up, as well. (It is interesting, Veracode noted, that Aerospace and Defense are only now starting to bring the same rigor to the security supply chain that they long brought to the manufacturing supply chain. But then, security industry watchers will tell you that spending on protection has been frozen at the half-billion mark for years).

It’s a good plan of action. In this two-part look at how to vet security vendors, this first article focuses on assessing a vendor. This is done at arm’s length by simple online research as well as by holding security vendors accountable for not living up to various agreed-upon levels of service, similar to what’s being done increasingly by the industries Veracode notes. The second article, In God We Trust, but It’s Nice to Do a Physical Walk-Through on Security Vendors, features input from Infosec professionals on what to watch for if you can conduct an on-site visit to a security vendor.

Bigger is Not Better

Josh Corman has a long curriculum vitae in the security industry. He recently took over as Director of Security Intelligence at Akamai Technologies.

Akamai are the people whose web acceleration technology kept much of the Internet alive during the horror that took the life of cofounder Daniel Lewin on 9/11; if you haven’t read Hiawatha Bray’s moving account of that time, do. You can find it here.

When HBGary Federal fell, many security professionals were gathered at the RSA conference in San Francisco. Corman was on the show floor. As he looked around, he became angry.

Like all conference show floors, there were different sized booths. Some were very large: the display of commercial success from the mainstream security vendors. Some were small: tucked in the back, staffed by firms that don’t speak up much and that don’t market aggressively—maybe because they lack the money to market aggressively, or maybe because they’re smart enough not to.

What angered Corman was the rich display of success from the large security companies who do very little to effect actual security. The big, splashy names were not the ones who were being threatened. Instead, it was the small security firms that were being attacked: the ones who actually make a difference.

“I saw a schism in the marketplace between grandfathered compliance-mandated budgets that didn’t do anything to have an actual impact on actual adversaries,” Corman says. “Then there was a smaller pocket, with less advertising, who spent less on their booths, who didn’t brag a lot, who didn’t do press releases on how great the new security botnet [they discovered] was. Their size was in adverse proportion to their adversaries.”

On display at the mainstream RSA conference was what Corman calls the partial overlap of two spheres: the highly impactful but not highly visible solution sets vs. the non-impactful but highly visible security solutions from the major vendors.

The two breeds of security vendors are extremely distinct. If he had to write down the 10 most valuable technologies that he’d deploy if his name were on the line, and compare it to the 20-30 most commercially successful security products, “There wouldn’t be overlap,” Corman says.

“It was very troubling to look back over a decade of fighting custom malware, with those big booths making fabricated press releases about some pretty unimportant worm or bot, and none [of their products] would actually help,” he says.

The security axiom is: Those who know, don’t talk. What brought down HBGary Federal, Corman says, was that CEO Aaron Barr knew, and he talked.

HBGary Federal, Corman says, was both relevant and open to talking about its methods and what Barr claimed to be his success in tracking Anonymous members’ true identities. After Anonymous took them down, and after HBGary Federal pulled out of the RSA show, the ripple effect on other vendors was immediate. As Corman took briefings from those small, impactful security players at the show, he noticed their attitudes flip, from their previously forthcoming attitudes to a new, wary quiet, Corman says. The predominant feeling: If it happened to HBGary Federal, it could happen to us.

“The reaction when I took briefings from vendors who were impactful but not visible was them saying, ‛We were talking more, but now we’re not going to,’” Corman says. “Better to stick to a more elite, exotic, boutique clientele.”

Corman’s not sure that strategy will protect a security firm from being targeted. If an adversary is continually thwarted by successful, impactful defense, “They’ll still go after [such security firms] as a target,” he says.

It’s a sorry state of affairs. If the big names make products that don’t actually help, and the small names are only known by the elite, how do the non-elite find a security company that can offer innovative, effective protection?

Step 1: Don’t put your faith in standards or QSAs.

For years, Corman has been trying to talk the private sector out of its reliance on PCI DSS compliance as the basis for a security strategy. He maintains that laws and industry standards such as Sarbanes-Oxley and PCI DSS drive companies to spend more on security than they might otherwise. That’s an incentive for security companies to make easy money off of products that offer, for example, out-of-the-box PCI compliance or painless treatment of HIPAA requirements. It all leads to companies buying products that don’t address specific threats, he told CSO in 2009 and reiterated to me recently.

"There are really bad people out there doing bad things and few pay attention to things like state-sponsored attacks and cyber warfare. This is because everyone's focusing on compliance," Corman told CSO.

The PCI’s Quality Security Assessors (QSAs) have also served as a false security blanket. One notable instance of this going public was when Heartland Payment Systems CEO Robert Carr blamed his company's data security breach on compliance auditors who failed to flag key attack vectors.

“The easiest [security compliance marker] to make go away [if you’re an attacker] is QSA,” Corman says. That’s because such auditing entails a finite, fixed checklist. It doesn’t change often, if at all, he says, and thus has no hope of keeping up with the latest Metasploits. 

Defenders like to say that such compliance weeds out the most basic of casual attackers, but Corman disagrees. “An unskilled, casual attacker, they can do whatever is the strength of the new Metasploit, which is added to on a daily basis. The minimal attack strength of someone using Metasploit is growing. They can easily make the auditor go away.” 

Then there’s SAS 70, one way to get independent auditing of a security program’s claims.

But again, it doesn’t mean anything, Corman says. If a service provider says they’ll do x-y-z, an independent assessor can evaluate for you, the buyer, that the provider did those things; but it only validates the provider’s own claims. It’s not a standard; it’s only an industry taxonomy. SAS 70 wouldn’t have stopped an RSA breach.

Neither would standards compliancy. “Being compliant isn’t going to make sure that RSA didn’t happen,” Corman says. “Someone selling to the defense industry, especially RSA [which, for example, sold its SecurID authentication to Lockheed Martin, whose security systems were initially believed to have been breached in the March attack], went through preapprovals. That still won’t necessarily mean that individual mistakes and lapses won’t happen.”

In lieu of good ways to assess if a security company’s products are secure, we’ve used totems and economic indicators such as PCI compliance, OWASP’s Top 10 Web application security vulnerabilities, government regulations, and SAS 70 audits. There are pockets of security you can get from these assurance levels, Corman says, but none are comprehensive.

Step 2: Demand that security vendors evaluate offerings.

Corman confirms what Veracode is tracking: Smart buyers are starting to get vendors to test or evaluate how secure their code and products are. A smarter approach than relying on compliancy or QSAs, he says, and one he’s seen some buyers and states starting to use, is to begin to demand of vendors some level of guarantee against failures that relate to what he calls “these arbitrary indicators.” Some states have contract language stipulating that if a vendor software falls victim to one of OWASP’s Top 10, for example, there will be penalties.

Others require in the RFP process that bidders be assessed by a third party, such as WhiteHat or Veracode, on their own dime. These steps aren’t necessarily aimed at getting the perfect price or the perfect product, Corman says, but they at least give a buyer some type of ranking to go by, and the contract language to stipulate that either the product will be fixed or the bid will be lost. Losing bids by ignoring the findings of such results will provide the motivation security vendors need to get things fixed, he says, and hopefully create a rising tide of higher security product quality.

Step 3: Assess the company’s history of dealing with breaches.

A researcher at a security vendor who requests anonymity suggests that a track record is one of the few things you can rely on when vetting a security company. It’s easy enough to search for embarrassing security incidents in a company’s history, such as shipping malware or having its site hacked. Buyers can also find feedback on how reliable a company is and how good they are at delivering updates on time.

What’s most particularly useful, however, is to see how a company responds after an attack. “When shit hits the fan, how does the company react?” the researcher asks. “That tells you more than the actual problem.”

For example, Lastpass.com, a cloud-based password management company, suffered a security incident in May. Nobody tried to shove it under the carpet. The company was forthcoming. It wrote a lengthy blog and updated users by the hour as its staff analyzed the issue and dealt with a mob of customers changing passwords. They wrote a blow-by-blow analysis of what they did wrong, what they did right, specific actions they took, additional changes planned, and how those changes would help to prevent future problems.

There are many examples of that not happening. RSA is one of them. When the company was attacked in March, they simply called it a “sophisticated cyber attack.” “We still don’t know what happened,” the researcher notes. “Was it authentication keys? SecureID cards? Only recently did we learn what sort of malware they got hit by. But only that it was something to allow remote access, but not that important. It only allowed attackers in. But what information did they take?”

It’s not that we should look for RSA to say what attackers got, exactly. That would not be a good thing. But it would be helpful to have some indication of the implications of the attack, the researcher says.

Caveat emptor: If the security firm you’re considering making a purchase with has dealt with problems in this type of less-than-transparent way, consider well whether you want to enter into a relationship with them.

These are all preliminary research steps that are easy enough to employ when vetting a security company from afar, and contracts stipulating service levels can help hold a security vendor accountable for screwing up.

But that’s cold comfort for an organization that doesn’t want the screwups to happen in the first place. For those organizations, an on-site visit can add a deeper level of insight into the culture of the security company that’s promising to protect you. If that’s an option, please continue reading the second article in this series, In God We Trust, but It’s Nice to Do a Physical Walk-Through on Security Vendors.