During a live penetration test, QA should expect authorized exploratory traffic, controlled negative cases, temporary test data, tester questions and validated findings that may resemble ordinary defects or incidents. Avoid confusion with a shared runbook: identify the engagement, test window, accounts and source indicators; preserve normal triage; and add a security-aware classification and escalation path.
Define the architecture question before testing
Define how QA, security, engineering, support and operations will distinguish an authorized test action, a normal product defect, a real unrelated incident and an unexpected test impact. Each has a different owner and response path.
Write the expected security invariant and the prohibited outcome in language that engineering, security and operations interpret the same way. Name the identity, trust transition, protected asset and authoritative state. That statement becomes the basis for scope, evidence and retesting.
Build the minimum evidence pack
- A current component and trust-boundary diagram for the environment under test.
- Identity, credential, network and data-flow details relevant to the decision.
- Representative accounts, workloads and synthetic records with known ownership.
- Logs or traces that follow the action through every material control point.
- Explicit safety limits, stop conditions, owners and restoration steps.
Prepare signed authorization and scope, dates and time zone, tester contacts, emergency owner, approved source indicators, test accounts and naming convention, target environment, excluded workflows, rate limits, stop conditions, feature flags, known defects, ticket labels, evidence channel and incident escalation details.
Establish one engagement identity
Everyone needs a reliable way to correlate tickets, logs, chats and tester questions to the same authorized activity.
- Create engagement identifier.
- Tag test accounts and synthetic data.
- Record approved source indicators.
- Name primary and backup contacts.
- Publish start and end notices.
Source IP alone is insufficient because proxies, mobile networks or shared infrastructure can change. Confirm through the engagement channel.
Keep normal bug triage intact
Security activity may trigger errors, unusual state or support reports, but ordinary defects still need their existing process.
- Retain severity and product impact fields.
- Add security-test correlation field.
- Link duplicate symptoms to one event.
- Separate finding validation from defect ownership.
- Protect restricted security details.
Use a restricted child record for exploit evidence when the general QA tracker has broad visibility.
Define four triage classifications
A small, explicit classification prevents every anomaly becoming either ignored test noise or an emergency.
- Authorized expected test activity.
- Authorized activity with unexpected impact.
- Normal defect discovered during testing.
- Potential unrelated security incident.
- Unconfirmed activity awaiting correlation.
Record who confirmed classification, when, using which engagement evidence and what response followed.
Coordinate tester questions and findings
Testers often need state resets, workflow explanation or evidence confirmation while exploring.
- Route questions through one channel.
- Assign QA workflow owner.
- Provide synthetic fixtures.
- Confirm known defects without biasing testing.
- Track validated findings separately.
Do not close a security observation merely because QA cannot reproduce it with the original steps; preserve preconditions and request clarification.
Use stop conditions consistently
Unexpected data access, service degradation, real transactions or out-of-scope reach require rapid containment.
- Pause on customer data exposure.
- Pause on availability impact.
- Stop real payment or notification.
- Escalate scope boundary ambiguity.
- Document restoration and restart approval.
The engagement lead authorizes resumption after owners confirm stability; individual teams should not silently continue.
Protect evidence and communication
Security proof may contain tokens, personal data or exploit detail inappropriate for ordinary screenshots and chat.
- Use approved restricted repository.
- Redact secrets from tickets.
- Share minimum reproduction detail.
- Apply retention and access rules.
- Preserve timestamps and correlation IDs.
Make evidence usable for remediation and retest while limiting exposure to people who need it.
Close the test window cleanly
End-of-test reconciliation prevents leftover accounts, data and alerts from confusing later operations.
- Send completion notice.
- Disable temporary credentials.
- Remove synthetic records or label retention.
- Resolve unclassified events.
- Review alerts and support tickets.
Keep a closure log of artifacts removed, evidence retained, open findings, pending validation and any operational lessons.
Execute as controlled hypotheses
- Establish the legitimate baseline and capture authoritative state.
- Change one trust variable—identity, route, scope, object, time or environment.
- Observe the decision at each layer rather than relying only on the response.
- Stop at the minimum proof that demonstrates or rejects the hypothesis.
- Restore test state and record residual uncertainty or blocked coverage.
Run a short kickoff with QA, security and operations, then use time-boxed daily checkpoints for active engagements. Do not ask monitoring teams to ignore broad classes of alerts; give them correlation context and explicit contacts.
Report the architectural consequence
Maintain an engagement event log, classified QA tickets, unexpected-impact record, stop/resume decisions, validated finding links and end-of-window reconciliation. Keep exploit detail in restricted evidence storage.
Separate observed evidence from inference. State prerequisites, repeatability, blast radius and the shared component responsible for the decision. When multiple findings have one architectural cause, keep the individual proofs but group the remediation around the common control.
Required closure evidence
- The original proof no longer succeeds.
- A legitimate workflow still succeeds under the intended identity and route.
- Alternate consumers of the same pattern enforce the same invariant.
- Telemetry records both allowed and denied decisions with useful context.
- The architecture record and threat assumptions reflect the new control.
Remediate at the durable control point
Fix confirmed product defects through normal ownership while security findings retain risk, evidence and retest workflow. Improve testability, fixture reset and observability where operational ambiguity delayed classification.
Retest the invariant, not just the payload
For each finding, reproduce with controlled preconditions, confirm authoritative impact, verify the fix and positive behavior, then update QA regression. Do not confuse operational correlation with technical closure.
Use the NIST Technical Guide to Information Security Testing and Assessment for assessment planning and evidence discipline, and the OWASP Web Security Testing Guide for relevant application techniques. Adapt both to the system-specific trust decision described here.
The architecture decision
QA should expect unusual but authorized activity, not disorder. A shared identifier, four-way classification, protected evidence and explicit stop path keep bug triage trustworthy throughout live testing.
Ask WIMD to plan a controlled live engagement with clear QA coordination, evidence handling and incident boundaries.