Compare penetration-test providers by normalizing risk coverage before comparing severity labels. Map each proposal or report to the same assets, attack paths, identities, tenant boundaries, workflows, methods, evidence standard, exclusions and closure support. Then translate provider ratings into your risk model while preserving the original severity and rationale.

Start with the assurance requirement

Define the security outcomes and assurance boundary both providers must address. Decide which rating factors your organization uses and which mandatory techniques, roles or deliverables cannot be traded away.

Record the requesting party, framework or contract clause, assessment boundary, evidence period, due date and decision the evidence must support. Confirm the interpretation with the auditor, assessor or customer where possible. A penetration test can contribute strong assurance, but a report cannot guarantee an audit outcome or replace the organization’s wider control evidence.

Create a traceable evidence index

  • Requirement or control identifier and the assertion being supported.
  • In-scope systems, interfaces, environments, identities and important exclusions.
  • Assessment dates, methodology, tester identity or competence evidence and authorization.
  • Finding identifiers, severity rationale, remediation owner, status and exception reference.
  • Retest evidence, closure date, residual risk and the location of protected technical detail.

Collect both scopes, asset and API inventories, role and tenant matrices, work plans, day allocations, methods, severity definitions, finding examples, achieved coverage, limitations, QA processes, tester profiles, reporting formats, retest terms and commercial assumptions.

Define the risk question

Build a common coverage matrix around risks and attack paths rather than vendor terminology.

  • Normalize targets.
  • Normalize identities and tenants.
  • Compare critical workflows.
  • Compare trust boundaries.
  • List exclusions and assumptions.

Connect the conclusion to a named asset, threat, control objective and decision owner. Separate observed evidence from assumptions and management judgment.

Challenge the evidence quality

Compare how each provider discovers, validates, rates and reviews evidence.

  • Separate manual and automated work.
  • Inspect proof quality.
  • Review severity factors.
  • Check peer review.
  • Assess business-logic depth.

Record missing, stale, inconsistent or inaccessible evidence as a limitation. Absence of a reported finding is not proof that the risk was tested.

Test the risk conclusion

Translate results to an internal model without erasing original provider judgments.

  • Preserve original rating.
  • Map impact and likelihood.
  • Note rating conflicts.
  • Compare root-cause grouping.
  • Assess uncovered residual risk.

Use independent review and counter-evidence. Document why alternative explanations were accepted or rejected.

Govern exceptions and residual risk

Record tradeoffs and selection rationale so price or report style does not silently determine assurance.

  • Weight mandatory gates.
  • Identify capability gaps.
  • Evaluate retest support.
  • Record decision owner.
  • Plan supplemental coverage.

Every exception needs an accountable owner, rationale, expiry, monitoring, review trigger and route back to remediation or reassessment.

Create decision-ready reporting

Present side-by-side evidence that explains differences in both breadth and depth.

  • Show common scope matrix.
  • Show method and access.
  • Show evidence threshold.
  • Show limitations.
  • Show normalized risk view.

Keep technical detail protected while making scope, status, uncertainty and required management action visible to the authorized reader.

Operate the evidence workflow

  1. Confirm the requirement, evidence owner, reviewer and deadline.
  2. Freeze a versioned scope and record every approved change or exclusion.
  3. Collect assessment artifacts through a controlled, access-limited repository.
  4. Map findings and closure evidence without changing the tester’s original conclusion.
  5. Perform a completeness and consistency review before external sharing.

Ask providers to respond to the same scope worksheet and representative scenarios. Clarify ambiguities in writing and challenge unusually low effort or broad claims.

Preserve evidence integrity and confidentiality

Keep the original signed or versioned report, evidence manifest and retest artifacts. Redact copies rather than overwriting the source. Restrict exploit steps, credentials, personal data and internal architecture to approved recipients. Record who received which version, under what authorization and for what purpose.

Quality checks before reliance

  • Scope, dates and environment agree across the report, statement of work and evidence index.
  • Every closure claim points to a finding identifier and retest result.
  • Exceptions name an owner, rationale, review date and compensating controls.
  • Control mappings distinguish direct evidence from supporting context.
  • Sanitized summaries do not imply broader coverage or stronger closure than the source report.

Use framework language carefully

Use the applicable official control text and assessor guidance as the authority. The NIST Technical Guide to Information Security Testing and Assessment supports disciplined assessment planning and reporting, while the OWASP Web Security Testing Guide provides application testing context. Neither substitutes for framework-specific interpretation by the responsible auditor or assessor.

Report the assurance conclusion

Use a comparison pack containing mandatory gates, normalized coverage, methodology, evidence quality, delivery and retest support, commercial assumptions and residual gaps.

State observed facts, limitations and management decisions separately. Do not use “compliant,” “certified,” “secure” or “all vulnerabilities fixed” unless the authorized assessor and evidence genuinely support that precise claim.

The GRC decision

Choose the provider whose demonstrated approach best covers your material risks and produces defensible evidence. Severity scales can be normalized; missing attack paths cannot.

Ask WIMD for a comparable testing scope with explicit identities, attack paths, evidence standards and closure deliverables.