PCI Penetration Testing: Core Impact Lies.

I found myself in a weird position today, listening to an old friend tell me that Bob Russo from the SSC had apparently gone madly insane and started asserting very specific claims on the behalf of vendors - in this case Core Impact. I like Bob, and i've never heard him directly say anything that was contradictory to the reality of PCI, nor do I know him to be a crazy person.

So I decided to listen to this interview, and compare it to Core's claims. Here is Core's press release from November:

http://www.coresecurity.com/content/cipf-enables-organizations-to-prioritize-security-risks-and-maintain-pci-compliance

where Core claims, and I quote:

"In a recent webcast hosted by Core Security, Bob Russo, general manager of the PCI Security Standards Council, reaffirmed that internal use of a penetration testing software solution such as IMPACT meets the specific testing guidelines of DSS and confirmed that reports produced by such technologies will be accepted by certified auditors as proof of compliance with that portion of the mandate. The statement refutes some existing market misconceptions that DSS requires third-party penetration testing."

In fact,what Bob says is almost the complete opposite of this. Here is a transcription from the audio, starting at roughly 45:00:

Mike @ Core Security: "I dont know exactly what you think on this one but we've had a few people who've said ... they were told by their auditors that reports from our pentesting product, core impact, might or might not be accepted, but in reading the standard it says that pci has no reporting requirements for pentesting. so is it basically, go back to .. if you can document that you've conducted a pentest, with or without our product.. if you've documented that you have conducted an effective test, and as long as the auditor can make sense of it, is that acceptable to you or to the PCI standard?"

Bob Russo @ PCI SSC: "That is acceptable to us. We don't look for specifics on the pentest. Again, that would be up to the assessor and you would have to convince the assessor that the pentest is in fact valid."

Mike @ Core Security: "So you have to just basically say 'I did it and heres the results' so you're judging the quality of the pentest and the document, not necessarily the audit trail if you will..."

Bob Russo @ PCI SSC: "That is correct."

This has got to be one of the worst examples of vendor BS I have seen to date. Bob is obviously responding to the part about "if the auditor can make sense of it," and this statement is used in Core's press release to make Bob look like he's inferring that:

"CORE IMPACT = PENETRATION TESTING."

The standard does not "say that there are no specific requirements."

It says that you need to conduct a penetration test, and the person conducting needs to have a clue about what a pentest is and how to execute one. Running a scan with impact and executing a pentest are vastly seperate excercises, and I would love to hear someone from Core try to refute this. Be forewarned though, I have an army of people with various colored hats, myself included, prepared to clue you in, deeply.


During the Q&A Bob never indicates that "reports produced by such technologies will be accepted"... by anybody. Instead, he reasserts that its not the SSCs job or interest to review any of these reports, and that the PCI assessor of record will be the one to review the reports and make a determination as to their legitimacy.

Frankly, if I were still a QSA and someone handed me a core impact report, I would suggest that the results of that report would make a very nice addendum to a real pentest report.

Core goes on further to suggest that Bob directly indicated that Core Impact reports will directly suffice PCI DSS requirement 11.3.

"confirmed that reports produced by such technologies will be accepted by certified auditors as proof of compliance with that portion of the mandate. The statement refutes some existing market misconceptions that DSS requires third-party penetration testing."

Mike,  this never happened and you know it. See the transcript of your own interview for an example.

I can tell you that I listened to this entire 1 hour webcast in hopes of finding out what sort of disorienting malaria Bob had acquired so that I could recommend an appropriate physician and find out where to send the get well card. As it turns out, he has in fact not gone crazy, and he hasn't said anything that even comes close to supporting the claims that Core is making.

Core Technologies on the other hand, is asserting that the SSC has somehow endorsed their product as being capable of directly sufficing a PCI requirement, and that my friends... is nonsense.