Repeat penetration testing after a material change when the change creates or alters important attack paths, trust boundaries, identities, data exposure, privileged operations or security controls beyond what existing regression can credibly cover. The decision should be risk-based and documented; neither “every release” nor “only annually” fits all systems.

Start with the assurance requirement

Define materiality in policy and incorporate framework, contractual and customer obligations. Some changes require testing regardless of a local risk score or the date of the last engagement.

Record the requesting party, framework or contract clause, assessment boundary, evidence period, due date and decision the evidence must support. Confirm the interpretation with the auditor, assessor or customer where possible. A penetration test can contribute strong assurance, but a report cannot guarantee an audit outcome or replace the organization’s wider control evidence.

Create a traceable evidence index

  • Requirement or control identifier and the assertion being supported.
  • In-scope systems, interfaces, environments, identities and important exclusions.
  • Assessment dates, methodology, tester identity or competence evidence and authorization.
  • Finding identifiers, severity rationale, remediation owner, status and exception reference.
  • Retest evidence, closure date, residual risk and the location of protected technical detail.

Maintain change description and architecture diff, threat-model update, affected assets and data, identity and privilege changes, new interfaces, control modifications, supplier dependencies, security regression results, incidents, previous scope, compliance triggers and reassessment decision.

Define the risk question

Identify changes capable of altering attacker reach, authority, state or impact.

  • Review trust-boundary changes.
  • Check new APIs and integrations.
  • Assess identity and role changes.
  • Review sensitive data movement.
  • Inspect security-control replacement.

Connect the conclusion to a named asset, threat, control objective and decision owner. Separate observed evidence from assumptions and management judgment.

Challenge the evidence quality

Evaluate whether existing tests actually exercise the changed attack paths and authoritative outcomes.

  • Map regression to invariants.
  • Review negative authorization cases.
  • Check environment parity.
  • Assess independent coverage.
  • Identify untested interactions.

Record missing, stale, inconsistent or inaccessible evidence as a limitation. Absence of a reported finding is not proof that the risk was tested.

Test the risk conclusion

Choose full, targeted or deferred reassessment from risk and evidence rather than release size alone.

  • Rate change exposure.
  • Assess blast radius.
  • Review exploit prerequisites.
  • Consider prior findings.
  • Document residual uncertainty.

Use independent review and counter-evidence. Document why alternative explanations were accepted or rejected.

Govern exceptions and residual risk

Make the trigger decision reviewable and time-bound, especially when testing is deferred.

  • Name decision owner.
  • Reference policy threshold.
  • Set interim safeguards.
  • Book target date.
  • Define escalation triggers.

Every exception needs an accountable owner, rationale, expiry, monitoring, review trigger and route back to remediation or reassessment.

Create decision-ready reporting

Explain why the selected reassessment scope is proportionate to the changed risk.

  • Summarize architecture delta.
  • List affected threats.
  • Show existing evidence.
  • State new test scope.
  • Record deferral rationale.

Keep technical detail protected while making scope, status, uncertainty and required management action visible to the authorized reader.

Operate the evidence workflow

  1. Confirm the requirement, evidence owner, reviewer and deadline.
  2. Freeze a versioned scope and record every approved change or exclusion.
  3. Collect assessment artifacts through a controlled, access-limited repository.
  4. Map findings and closure evidence without changing the tester’s original conclusion.
  5. Perform a completeness and consistency review before external sharing.

Run a change-risk review before launch for material candidates and again after deployment if actual architecture or exposure differs. Reserve vendor capacity for likely high-risk changes.

Preserve evidence integrity and confidentiality

Keep the original signed or versioned report, evidence manifest and retest artifacts. Redact copies rather than overwriting the source. Restrict exploit steps, credentials, personal data and internal architecture to approved recipients. Record who received which version, under what authorization and for what purpose.

Quality checks before reliance

  • Scope, dates and environment agree across the report, statement of work and evidence index.
  • Every closure claim points to a finding identifier and retest result.
  • Exceptions name an owner, rationale, review date and compensating controls.
  • Control mappings distinguish direct evidence from supporting context.
  • Sanitized summaries do not imply broader coverage or stronger closure than the source report.

Use framework language carefully

Use the applicable official control text and assessor guidance as the authority. The NIST Technical Guide to Information Security Testing and Assessment supports disciplined assessment planning and reporting, while the OWASP Web Security Testing Guide provides application testing context. Neither substitutes for framework-specific interpretation by the responsible auditor or assessor.

Report the assurance conclusion

Create a reassessment decision record linking the change, threat-model delta, evidence, required tests, timing, owner and compliance interpretation.

State observed facts, limitations and management decisions separately. Do not use “compliant,” “certified,” “secure” or “all vulnerabilities fixed” unless the authorized assessor and evidence genuinely support that precise claim.

The GRC decision

Repeat penetration testing when changed risk exceeds the assurance provided by existing evidence. Scope the reassessment around affected attack paths while preserving periodic whole-system coverage.

Ask WIMD to plan a change-driven reassessment focused on the trust boundaries and attack paths your release actually changed.