Table of contents
18
By Salman Khan, OSCP+, Founder of XHack, SRT (Synack Red Team member)
Read this in 30 seconds: An auditor does not read a pentest report to admire the exploits. They read it to answer who tested, what was covered, when, and whether the problems got fixed. Only one standards body has published the checklist for those questions.
- PCI SSC published a 22-question checklist for judging a pentest report. It sits in the Council’s September 2017 Penetration Testing Guidance, which calls it a suggested minimum and says a bare list of vulnerabilities “does not meet the intent of the penetration test.”
- Almost no framework names a report format. Of the frameworks we checked, only FedRAMP and DORA TLPT prescribe report sections, and both also restrict who may run the test.
- The checklist is mostly about the test, not the bugs. Only 4 of its 22 items concern the findings themselves. The rest cover the tester, the scope, the method and what happened during testing.
- We graded our own published sample report against it. It scores 16 items clear, 3 partly and 3 absent: no tester-independence statement, no problems-encountered narrative and no inventory of discovered ports and services.
- Retention has a clock. PCI DSS 4.0.1 requires keeping pentest and remediation results for at least 12 months, and HIPAA’s documentation rule runs 6 years.
A pentest report is not a bug list. To an auditor it is an evidence file, and half the evidence is about the test itself.
You paid for the test, the findings are fixed, and the auditor still sends the report back with questions. That happens when the report proves the bugs but not the process. This guide covers what each framework says about the pentest report, the one published checklist you can hold a report against, and how our own sample report measures up.
The auditor works for a different reader than your engineers do. A CPA firm doing a SOC 2, a PCI assessor and an ISO certification auditor each need the same four answers:
PCI SSC says it plainly in its Penetration Testing Guidance. The report should “clearly communicate what was tested, how it was tested, and the results of the testing”, and “Merely reporting lists of vulnerabilities is not helpful in this endeavor and does not meet the intent of the penetration test.”
That one paragraph explains most rejected reports. A scanner export is a list of vulnerabilities. A pentest report is an account of an attempt to break in.
Very few, and the difference matters. We checked each framework’s own text and guidance:
| Source | What it says about the pentest report | Binding? |
|---|---|---|
| PCI DSS 4.0.1, requirement 11.4 | Methodology must be defined and documented; keep penetration testing results and remediation results for at least 12 months; repeat testing to verify fixes | Yes |
| PCI SSC Penetration Testing Guidance (2017) | A report outline and a 22-question evaluation checklist, described as “only suggested outlines” | No, a supplement |
| FedRAMP Penetration Test Guidance v3 | Section 6 reporting: scope, attack vectors assessed, timeline, tests performed and results, findings and evidence, access paths | Yes, for FedRAMP |
| DORA TLPT, RTS 2025/1190 | Red team test report and blue team test report, each with a deadline, then a summary report and an authority attestation | Yes, for TLPT only |
| NIST SP 800-115 (2008) | Section 8.2: one report may need several formats for several audiences; internal reports include methodology, results, analysis and a remediation plan | Guidance |
| HIPAA, 45 CFR 164.316 | Retain required documentation for 6 years | Yes, retention only |
| SOC 2, ISO 27001, GDPR, NIS2 | No report format named | Auditor’s judgment |
Two of these also decide who may run the test. FedRAMP needs an accredited assessor and DORA TLPT needs testers who meet Article 27, so a report from any other firm will not satisfy them, ours included. Our posts on FedRAMP penetration testing and DORA TLPT cover that, and our compliance pillar maps all of the frameworks together.
For SOC 2 and ISO 27001, what auditors look for is practitioner consensus, not a published rule. Our SOC 2 guide and ISO 27001 guide say so directly.
Section 5.4 of the PCI guidance is titled “Penetration Test Report Evaluation Tool”. It is aimed at “merchants, service providers, and assessors” who receive a report and need to judge its completeness. It has 22 questions in 7 groups:
| Group | Items | What the reader checks |
|---|---|---|
| Tester | 5 | Name, organization and contact details; credentials of the analysts; evidence the testers are organizationally independent from the management of the tested environment; dates of the engagement; date the report was issued |
| Executive summary | 3 | Testing performed, results, and steps for remediation |
| Scope | 5 | Clearly documented; how it was determined; attack perspective (internal, external or both); type of testing (application layer, network layer or both); constraints such as time and bandwidth |
| Methodology | 2 | Clearly stated; reflects industry practice such as OWASP and NIST |
| Narrative | 2 | Automated and manual testing discussed; problems encountered documented, such as protection systems blocking traffic |
| Discovery | 1 | All identified open ports and services, with the originating perspective |
| Results | 4 | Whether retesting is needed and where; a summary list and a detailed list of items to remediate and retest; exploit attempts with the potential risk stated |
Read the checklist’s own caveats before you use it as a pass mark. PCI SSC calls these items “a suggested minimum set”, says the checklist is “not intended to take the place of thorough report inspection”, and says it is “not the intent to generate any type of ‘score’.”
Two more cautions. The guidance is version 1.1 from September 2017, and it still refers to Requirement 11.3, which is 11.4 in PCI DSS 4.0.1. And it is written for PCI, so using it for SOC 2 or ISO 27001 is our judgment. It is the only published yardstick we found, which is why we use it.
The same section gives a report outline: executive summary, statement of scope, statement of methodology, statement of limitations, testing narrative, segmentation test results, findings, tools used, and cleaning up the environment after testing. PCI says these are “only suggested outlines and do not define specific reporting requirements for PCI DSS penetration tests.”
Three details are worth copying into any pentest report:
The report is the summary. The evidence is what sits behind it. PCI’s guidance lists screenshots, raw tool output, dumps from exploitation, logs, photos and recordings, and asks that cardholder data be kept to a minimum if the tester ever touches it.
Retention is where the binding rules are:
If your contract does not say how long the tester keeps evidence, ask before you sign.
Customers and auditors want different documents. Some pentest vendors recommend giving customers and vendor questionnaires a short attestation letter and keeping the full pentest report for auditors, regulators and enterprise deals under NDA. That is vendor practice, not a framework rule.
Be careful with the word “attestation”, because it means three different things:
None of these makes you compliant. The auditor, assessor or regulator makes that decision, and a letter from the tester only confirms the test was completed.
A pentest report full of open findings is weaker evidence than a report plus proof the findings were closed. The frameworks lean the same way:
Plan for it in the contract. Ask whether the retest is included in the price, how long after the fixes it happens, and whether it produces a closing report with each finding marked resolved. The retest should land inside your audit window too, which is why we suggest testing early in a SOC 2 Type II period.
XHack publishes a sample report, a redacted 32-page document from a test window of August 4 to 15, 2026. We read it in full and checked it against the 22 PCI items.
| Group | Clear | Partly | Absent |
|---|---|---|---|
| Tester (5) | 3: credentials, dates, report date | 1: organization is named, contact details are not | 1: no statement of independence from the client’s management |
| Executive summary (3) | 3 | 0 | 0 |
| Scope (5) | 4: documented, perspective, type, constraints | 1: “agreed scope and signed rules of engagement”, but no account of how scope was set | 0 |
| Methodology (2) | 2: PTES, OWASP WSTG, NIST SP 800-115 and MITRE ATT&CK | 0 | 0 |
| Narrative (2) | 0 | 1: findings are manually verified, but there is no narrative of the automated versus manual work | 1: no problems-encountered account |
| Discovery (1) | 0 | 0 | 1: no inventory of open ports and services |
| Results (4) | 4 | 0 | 0 |
| Total (22) | 16 | 3 | 3 |
What the sample pentest report does well is the part auditors can verify at a glance: a document-control page with version history (0.1 internal draft, 0.9 peer and QA review, 1.0 final), an OSCP-certified lead tester, a QA reviewer, a signed rules-of-engagement reference, and a stated retention of at least 12 months. Every finding carries a CVSS v4.0 vector and a status, and a finding-to-control table maps each one to PCI DSS, ISO 27001, OWASP, GDPR and NIST CSF controls. Severity ties to fix windows of 24 to 72 hours for Critical, 7 days for High, 30 for Medium and 90 for Low. The retest is included and flips each finding to “Resolved” in a closing report.
Three caveats. This is a published, redacted sample, and a live engagement may carry more. The sample’s “independent” label describes the QA reviewer, not independence from the client. And the sample is not a PCI test, so some PCI items do not apply to it as written. Treat the three gaps as questions to ask any vendor, us included.
These come from the PCI guidance and from the practitioner consensus we found. They are not an AICPA or PCI list.
Choosing a vendor who avoids these is covered in our guide to choosing a pentest provider.
We write these reports for a living, so here is the straight version.
XHack runs three layers: human-led managed testing, an AI agent that works alongside the testers only with your explicit permission, and a company security platform for scanning between tests. Our pricing page describes the report as “methodology, tooling, CVSS scoring and evidence of testing, structured the way auditors ask for it”, with verified findings only: “If it reaches the report, we proved it was exploitable.”
Concretely, fixed-price engagements start at $2,500 for Essential and $5,000 for Assurance, which is the tier built for SOC 2, ISO 27001 and PCI DSS cycles. Every tier includes a retest, and the pricing page says evidence mapped to PCI DSS, SOC 2, ISO 27001, HIPAA and GDPR is part of the Assurance tier. The pricing page lists a letter of attestation covering scope and test dates in the Essential tier and says attestation letters and auditor-facing evidence packs can be added, so confirm what your tier includes.
Our rule on compliance is simple. XHack does not approve, certify or make anyone compliant. Our pentest report is built to satisfy a framework’s pentest requirement, and your auditor, assessor or regulator makes the decision. Where a framework restricts who may test, such as the FedRAMP assessor or DORA TLPT, check the rule before you hire anyone, us included.
Two privacy facts, kept separate. Chats with our AI agent are not stored by XHack and we cannot access them. A managed engagement’s report is different: the sample shows 12 months minimum retention. Our terms also say AI output can be incomplete or wrong, which is why a person verifies the findings in a managed report.
A pentest report for an auditor should name the tester and their credentials, give the engagement and report dates, state the scope and how it was set, describe the methodology, and list findings with severity, evidence and remediation status. PCI SSC’s 22-question checklist is the most detailed published yardstick, and it adds independence evidence and a discovery inventory.
No. Neither names a pentest report format or even requires a pentest. The AICPA names penetration testing once, as an optional example under CC4.1, and ISO 27001 does not name it in its controls. Auditors set their own expectations, so ask yours what they need before the test.
No. A scan report lists potential vulnerabilities, while a pentest report records an attempt to break in, with proof of what was exploited and how. PCI SSC says a bare list of vulnerabilities “does not meet the intent of the penetration test”, and PCI DSS treats scanning and pentesting as separate requirements.
PCI DSS 4.0.1 requires keeping penetration testing results and remediation results for at least 12 months. HIPAA requires required documentation to be kept 6 years, and a pentest used in your required evaluation would fall under that in our reading. Your auditor’s window may need it longer.
Auditors and regulators often get the full pentest report under NDA, while customers and questionnaires often get a short attestation letter. That split is vendor practice, not a framework rule. Confirm with your auditor, and note that the letter only confirms the test was completed.
No framework forbids it and none has approved an AI-only report. PCI DSS requires a qualified tester, and FedRAMP and DORA TLPT name who must test. A report with human-verified findings and a named, credentialed tester answers the checklist best. Our AI penetration testing compliance guide has the details.
A good pentest report answers the auditor’s four questions before they ask: who tested, what was covered, when, and whether the problems were closed. Only PCI SSC has published the checklist for that, and only 4 of its 22 items are about the bugs.
Before you sign with a vendor, hold a sample pentest report against the checklist and ask for what is missing. We did it to our own sample, and the three gaps we found are the ones worth asking everyone about.
Categories
Related articles
