PCI Penetration Testing: Core Impact Lies.

I found myself in a weird position today, listening to an old friend tell me that Bob Russo from the SSC had apparently gone madly insane and started asserting very specific claims on the behalf of vendors - in this case Core Impact. I like Bob, and i've never heard him directly say anything that was contradictory to the reality of PCI, nor do I know him to be a crazy person.

So I decided to listen to this interview, and compare it to Core's claims. Here is Core's press release from November:

http://www.coresecurity.com/content/cipf-enables-organizations-to-prioritize-security-risks-and-maintain-pci-compliance

where Core claims, and I quote:

"In a recent webcast hosted by Core Security, Bob Russo, general manager of the PCI Security Standards Council, reaffirmed that internal use of a penetration testing software solution such as IMPACT meets the specific testing guidelines of DSS and confirmed that reports produced by such technologies will be accepted by certified auditors as proof of compliance with that portion of the mandate. The statement refutes some existing market misconceptions that DSS requires third-party penetration testing."

In fact,what Bob says is almost the complete opposite of this. Here is a transcription from the audio, starting at roughly 45:00:

Mike @ Core Security: "I dont know exactly what you think on this one but we've had a few people who've said ... they were told by their auditors that reports from our pentesting product, core impact, might or might not be accepted, but in reading the standard it says that pci has no reporting requirements for pentesting. so is it basically, go back to .. if you can document that you've conducted a pentest, with or without our product.. if you've documented that you have conducted an effective test, and as long as the auditor can make sense of it, is that acceptable to you or to the PCI standard?"

Bob Russo @ PCI SSC: "That is acceptable to us. We don't look for specifics on the pentest. Again, that would be up to the assessor and you would have to convince the assessor that the pentest is in fact valid."

Mike @ Core Security: "So you have to just basically say 'I did it and heres the results' so you're judging the quality of the pentest and the document, not necessarily the audit trail if you will..."

Bob Russo @ PCI SSC: "That is correct."

This has got to be one of the worst examples of vendor BS I have seen to date. Bob is obviously responding to the part about "if the auditor can make sense of it," and this statement is used in Core's press release to make Bob look like he's inferring that:

"CORE IMPACT = PENETRATION TESTING."

The standard does not "say that there are no specific requirements."

It says that you need to conduct a penetration test, and the person conducting needs to have a clue about what a pentest is and how to execute one. Running a scan with impact and executing a pentest are vastly seperate excercises, and I would love to hear someone from Core try to refute this. Be forewarned though, I have an army of people with various colored hats, myself included, prepared to clue you in, deeply.


During the Q&A Bob never indicates that "reports produced by such technologies will be accepted"... by anybody. Instead, he reasserts that its not the SSCs job or interest to review any of these reports, and that the PCI assessor of record will be the one to review the reports and make a determination as to their legitimacy.

Frankly, if I were still a QSA and someone handed me a core impact report, I would suggest that the results of that report would make a very nice addendum to a real pentest report.

Core goes on further to suggest that Bob directly indicated that Core Impact reports will directly suffice PCI DSS requirement 11.3.

"confirmed that reports produced by such technologies will be accepted by certified auditors as proof of compliance with that portion of the mandate. The statement refutes some existing market misconceptions that DSS requires third-party penetration testing."

Mike,  this never happened and you know it. See the transcript of your own interview for an example.

I can tell you that I listened to this entire 1 hour webcast in hopes of finding out what sort of disorienting malaria Bob had acquired so that I could recommend an appropriate physician and find out where to send the get well card. As it turns out, he has in fact not gone crazy, and he hasn't said anything that even comes close to supporting the claims that Core is making.

Core Technologies on the other hand, is asserting that the SSC has somehow endorsed their product as being capable of directly sufficing a PCI requirement, and that my friends... is nonsense.

QSAs: SecurityMetrics in Remediation

Noone at SecurityMetrics had any comment on why the company was sent into remediation. As you'll notice, Chris Konrad from Fortrex kindly stepped up and posted the yesterday about the plan to remediate, and the company's committment to resolve the  internal issues that caused the status change. Anyone from SecurityMetrics? PSC?

How to Choose a PCI Assessor

Choosing an assessor is a lot like choosing a car. Some of them get you there quickly, some of them are reliable, and your mileage may vary. There are a number of factors to consider, including:

Ask to See Your QSA’s Resume

There are A LOT of people in the information security space, and their backgrounds vary pretty wildly. Make sure that you are comfortable with the skillset of your assessor, his or her work experience, and their knowledge of PCI. Also, make sure that the person who shows up is in fact the person who’s resume you reviewed. You may want to interview them as well to insure that they have a knowledge level that you're comfortable with.

Talk to other Merchants

There are people out there who have gone through this multiple times, and have probably used more than one assessor. Ask them what they thought about the assessor meeting their expectations in terms of technical abilities, and delivery timeframes.

Timeframe to Completion / QSA Availability

What is sales willing to commit to on paper in terms of having an assessor at your facility? How long will the assessment take? The answer may surprise you. If you’re a level 1 service provider, and they’re only going to spend 3 days reviewing your security controls, you may want to keep shopping. On the other hand, if its going to take 6 months to complete the assessment, that's probably less than optimal as well.

Your assessor should be able to start within 30 days, and given that there are 200+ requirements that need to be answered for with interviews, observation, and direct evidence gathering, the process will take some time. For an average level 1 merchant , 60-90 days is a reasonable expectation if you've been through the process before and are prepared.

Perform a “Pre-Assessment” Yourself

Understand what the scope of your environment is, and try to anticipate the results of your assessment. This will allow you to speak to the issues with confidence. In the event that you disagree with your assessor on a particular technology or architecture, you’ll be able to defend your position on its deployment without having to use your bank as an intermediary.

(Shameless Plug: We help a lot of people with this.)

Talk to your bank

Do they have a solid reputation with your acquiring bank and/or the card brands that you work with? Banks have to review a lot of scan reports, ROCs, and self assessment questionnaires. They see the cumulative results of many different assessors, and can provide you with valuable input as to who has delivered consistent, quality assessments. On the other hand, some assessors have a practice of "partnering" with banks, and those banks use that assessor as their "preferred assessor," making the recommendation somewhat worthless, because the answer is always the same, irrespective of the situation.

QSAs: PSC and Fortrex First to Go Down in New SSC Quality Assurance Program

Neither the principals of PSC (Payment Software Company) or Fortrex, Inc. (both Qualified Security Assessors) had any comment when asked about why their companies were put on probation by the PCI Security Standards Council in late January.

Both companies appear to be lacking the appropriate work papers that are required as supporting documentation to PCI assessments. All QSACs were told about the SSC reviews of work materials in July of last year, so unless they just weren't collecting or keeping their evidence, i'm unsure how you would screw up this badly. Maybe they didn't believe the QA program was real?

What will be interesting to see is whether or not they are able to recover from this. I suppose that if Trustwave can be the assessor of record for 2 of the 3 most major breaches in world history and noone wants to talk about it that these guys will be fine by next quarter.

PCI-DSS Compliance – Requirement 6.6: Web App Firewall, or Code Review?

In short – Yes.

Organizations that are in the process of determining whether or not they should install an application firewall in front of their web facing applications, OR do a code review should make a best effort to do both.

Why? Because, web application firewalls, while very useful, and generally very thorough are designed to perform a limited number of protection tasks, that are typically based on existing, deployed web technologies.

Web technologies are evolving, and more and more custom applications exist. For instance, CGI based vulnerabilities popped up as soon as CGI came into existence. Developers simply did not understand the threats associated with query string vulnerabilities, safe passage of variables, or safe backend system call execution. Then of course, php and asp represented new threats such as SQL injection, file path inclusion, XSS, etc.

Every time there has been a shift in web technology, a new threat landscape has emerged. More and more, open-source server-side web-apps are showing up in payment environments. Mambo CMS is a perfect example. It provides a great platform for easy deployment of nearly any site architecture. It’s also been compromised many times, and its been deployed as a front end for applications which process payments.

When you weigh the option that DSS provides of either source code review or web application firewall, you should understand that you’re going to obtain vastly different results from those two methods.

Installing a web application firewall will thwart the 99.9% of attacks that happen as a result of publicly disclosed vulnerabilities that are either being executed by a bot or an attacker with a luke warm degree of motivation.

A source code review will afford you visibility into the security of the application from otherwise unknown attack vectors and hopefully give you insight into not only vulnerabilities that exist as a result of insecure coding techniques, but along with a solid educational experience, the peace of mind that your developers are learning how to avoid common mistakes after you’ve received the results of the review.

Also, less vulnerabilities in the protected zone means less possibility of internal compromise. Don't forget that the majority of compromises happen from within.

So again, the answer to the question is yes. Install a web application firewall if your organization can afford the technology. Perform a source code review of both 3^rd party apps, and –especially- in-house developed code to provide a second layer of protection and valuable feedback to your development team.

This will allow you to use PCI as a valuable tool, instead of just one more exercise in compliance.

PCI Segmentation - Seriously?

A friend who recently went through the new annual QSA re-certification training a while back shared the most recent training manual with me. I have to say, that some of the clarifications are astonishingly... bad.

Specifically what I am taking issue with is the clarification around network segmentation. The title of the slide is:

"What are acceptable forms of Network Segementation?"

Then the slide describes Cisco access control lists.

Right.

Think I'm nuts? The slide goes on further to give the following example:

access-list 100 deny ip 10.0.0.0 0.255.255.255 any log

access-list 100 deny ip 127.0.0.0 0.255.255.255 any log

etc...

Another interesting sub bulletpoint is "Confirm audit logging is in place for all access to segmented network," suggesting that there are cases wherein you would not only be exclusively using ACLs, but that you may create specific ACLs which allow access to the "protected" network from the unprotected interface.

Translated:

"Cisco ACLs are effective protection for the cardholder environment, and if you need to make exceptions to your already bad security practice by allowing administrative traffic in from the outside, then go ahead."

Its any wonder that compliance numbers grow every year, and so do the number of compromises.

PCI QSA Training - Assess, Remediate, Reassess ?

I was speaking with a friend and colleague today about his experience with the most recent PCI training and what I found out was, well, interesting to say the least.

On the topic of independence and thoroughness, the SSC is apparently suggesting that it is acceptable to have an individual QSA:

• Perform the assessment.


• Remediate the issues in the environment.


• Re-assess the environment in subsequent years.

In my opinion, you may as well have the merchant or service provider simply self-assess at this point, because the QSA's objectivity is all but gone.

Anyone who has ever gone through QSA training and has spent any time doing work in the infosec space will tell you that the training materials are fairly simple, and the test is a walk in the park.

In fact most QSAs will tell you that they haven't learned anything substantially new by virtue of becoming a QSA. For the most part, the certification only enables them be the examiner of record on paper.

In other words, no special knowledge required, zero objectivity, and lots of multi-year managed service contracts between QSAs and merchants.

Its not hard to see why things like Heartland and RBS are happening. The ROC has become less honest than the tax return.