CA-8: Penetration Testing

What this control requires

CA-08 requires your organization to conduct penetration testing on designated systems and system components at a defined frequency. Unlike automated vulnerability scans, penetration testing demands skilled testers who simulate adversary behavior to identify exploitable weaknesses across network, operating system, and application layers.

The distinction matters because automated tools surface known vulnerabilities, but they can’t replicate how an attacker chains together misconfigurations, weak credentials, and logic flaws to achieve a breach objective. Penetration testing fills that gap by validating whether your defenses actually hold under realistic attack conditions and demonstrating the real-world impact of each exploitable weakness.

In practice, meeting CA-08 means establishing a structured penetration testing program that includes pretest analysis, scoping based on full system knowledge, agreed-upon rules of engagement, and post-test reporting that feeds directly into remediation workflows. Organizations transitioning infrastructure, such as migrating from legacy protocols, face heightened risk and should prioritize these assessments during transition windows.

Why it matters

Most organizations treat penetration testing as an annual checkbox rather than a continuous validation mechanism. That gap between assessment cycles leaves exploitable weaknesses undetected for months, exactly the window adversaries need to establish persistence.

Failure to maintain this control introduces audit risk and may result in certification withdrawal or regulatory findings during NIST SP 800-53 assessments. For organizations operating at the HIGH baseline, the absence of a documented penetration testing program signals a fundamental gap in security assurance to auditors and authorization officials.

The consequences compound across your vendor ecosystem. When your third-party partners lack penetration testing programs, their exploitable vulnerabilities become your inherited risk. A vulnerability assessment alone won’t reveal whether those weaknesses are actually exploitable in context.

Without regular penetration testing, several attack vectors go undetected.

• Unchained vulnerability paths where individual medium or low findings combine into a full compromise path that only hands-on testing reveals
• Misconfigured network segmentation that allows lateral movement, often invisible to scanning tools
• Application logic flaws that automated scanners can’t detect because they require contextual understanding of business workflows
• Stale testing scope that hasn’t been updated to reflect infrastructure changes, leaving new attack surfaces untested
• Physical and social engineering vectors that penetration testing can exercise but automated assessments skip entirely

How to implement

Penetration testing programs fail most often not because organizations skip them entirely, but because they scope too narrowly, test too infrequently, or never act on findings.

For your organization

Start by defining the scope, frequency, and methodology for your penetration testing program. Your assessment plan should identify which systems and system components are in scope, the testing cadence that matches your risk profile, and the rules of engagement all parties must agree to before testing begins.

Document the penetration testing methodology your organization follows. This methodology should include pretest analysis based on full knowledge of the target systems, identification of potential vulnerabilities from that analysis, and testing designed to determine exploitability. Specify whether tests will cover network, application, and physical layers.

Before any test begins, all parties need to agree on scope boundaries, authorized techniques, data handling procedures, and communication protocols. Rules of engagement should align with the tools, techniques, and procedures you anticipate real adversaries would use. This agreement also governs how testers handle any protected information they encounter during testing.

Execute tests using qualified personnel with demonstrable skills in network, operating system, and application security. The penetration test report should detail discovered vulnerabilities, the exploitation paths tested, and a risk-ranked remediation plan. Correlate findings with your existing vulnerability monitoring results.

Feed penetration test findings into your remediation workflow with defined service level agreements. After fixes are applied, retest to confirm vulnerabilities are resolved. Update your system security plan to reflect any architectural changes made during remediation.

The most common implementation failures fall into predictable patterns.

• Conducting only external penetration tests while ignoring internal network testing
• Using the same testing firm and methodology every cycle, which creates blind spots
• Failing to update the scope when infrastructure changes occur
• Treating the penetration test report as a compliance artifact rather than an operational input
• Not correlating penetration testing results with vulnerability scan data from RA-05

For your vendors

When evaluating whether a vendor meets CA-08 requirements, self-attestation alone isn’t sufficient. You need evidence that the vendor’s penetration testing program is structured, current, and acted upon.

Your security questionnaire should probe the structure, scope, and follow-through of the vendor’s testing program.

• Does your organization conduct penetration testing? At what frequency?
• What is the scope of your penetration testing program (network, application, physical)?
• Are penetration tests conducted by independent third parties, internal teams, or both?
• How do you establish rules of engagement for each test?
• Can you share a summary or executive report from your most recent penetration test?
• How are penetration test findings tracked and remediated?

Beyond questionnaire responses, request artifacts that demonstrate an active program.

• Executive summary of the most recent penetration test report (with sensitive details redacted)
• Documented rules of engagement template
• Remediation tracking records showing how findings were addressed
• Evidence that testing scope has been updated to reflect infrastructure changes

Several warning signs indicate a vendor’s penetration testing program may exist only on paper.

• Vendors who provide only automated vulnerability scan reports and label them as penetration tests
• Testing scoped to a single application while the vendor manages complex, multi-system environments
• No evidence of remediation activity following identified vulnerabilities
• Rules of engagement that haven’t been updated in more than two years
• Refusal to share even a redacted executive summary

To verify beyond self-attestation, request evidence that testing personnel hold recognized certifications and have documented experience. Review whether the vendor’s assessment report references specific testing methodologies rather than generic descriptions. Cross-reference the vendor’s claimed testing scope against the systems and data they manage on your behalf.

Evidence examples

Cross-framework mapping

No applicable content for this control.

Related controls

• RA-05 — Vulnerability Monitoring and Scanning: provides automated scanning data that feeds into penetration test planning and helps validate whether scan-identified vulnerabilities are exploitable
• RA-10 — Threat Hunting: complements penetration testing by proactively searching for indicators of compromise that may not surface during scheduled tests
• SA-11 — Developer Testing and Evaluation: addresses security testing during the development lifecycle, ensuring application-layer vulnerabilities are caught before systems reach production
• SR-05 — Acquisition Strategies, Tools, and Methods: governs how organizations evaluate the security posture of acquired components, including whether penetration testing is required for procured systems
• SR-06 — Supplier Assessments and Reviews: extends the third-party dimension of CA-08 by establishing structured assessments of supplier security practices, including their testing programs

Frequently asked questions

What is NIST SP 800-53 CA-08?

CA-08 is the NIST SP 800-53 Revision 5 penetration testing control, applicable at the HIGH baseline for both first-party systems and third-party vendor assessments. It requires organizations to move beyond automated vulnerability scanning by employing skilled testers who conduct pretest analysis based on full system knowledge, identify potential vulnerabilities, and attempt real-world exploitation under agreed-upon rules of engagement. The control covers network, operating system, application, and physical security layers.

What happens if CA-08 is not implemented?

Without a penetration testing program, your organization lacks validation that identified vulnerabilities are actually exploitable, and you lose visibility into chained attack paths that automated scanning tools miss. Auditors reviewing your assessment report will flag the absence of penetration test reports and documented rules of engagement as a material gap. This gap can result in denial or withdrawal of system authorization, particularly for systems operating at the HIGH baseline. Regulatory bodies and authorization officials treat missing penetration testing evidence as an indicator that the organization’s security assurance process is incomplete.

How do you audit CA-08?

Auditing CA-08 starts with verifying that a documented assessment plan defines the penetration testing scope, frequency, and methodology for each in-scope system or system component within the Assessment, Authorization, and Monitoring family. Reviewers examine the signed rules of engagement to confirm all parties agreed on scope boundaries and authorized techniques before testing began. The penetration test report itself must demonstrate that testers conducted pretest analysis, identified potential vulnerabilities, and attempted exploitation. Auditors also check remediation records for evidence that findings were tracked, resolved, and retested.

What is the difference between penetration testing and vulnerability scanning?

Vulnerability scanning uses automated tools to identify known weaknesses across systems, producing a catalog of potential issues ranked by severity. Penetration testing goes further by employing skilled testers who actively attempt to exploit those weaknesses, chain them together, and demonstrate real-world impact under defined rules of engagement. Where a vulnerability scan tells you what might be wrong, a penetration test report tells you what an adversary can actually achieve. NIST SP 800-53 treats these two activities as complementary, with CA-08 covering penetration testing and RA-05 addressing vulnerability monitoring and scanning.