Testing in Practice
Severity Versus Priority
Scenario
- A.A defect caused a typo in the footer text on every page of a public website, visible to every visitor, but affecting nothing functional.
- B.Another defect caused a rarely used internal report to fail silently once a month for one small team.
- C.Judged only by how many were affected, the footer typo looks more important.
- D.Judged by severity, the silent report failure is worse·it corrupts a decision-relevant output without anyone noticing, while the typo is merely embarrassing.
OFA Interpretation Questions
- E.Which of these two defects would you classify as higher severity, and why?
- F.Which would you classify as higher priority to fix first, and does that match your severity ranking?
Stewardship Application
- G.Severity and priority answer different questions, and conflating them leads to fixing visible-but-harmless issues before quiet-but-serious ones.
- H.A foundational OFA who can articulate this distinction, even informally, brings real value to a triage conversation.
I.
THE DISCREPANCY THAT SAT TOO LONG
Scenario
- A.During validation of a new giving statement feature, an OFA noticed one ward's numbers looked slightly off, but only by a few dollars, and assumed it was a rounding issue.
- B.The OFA made a private note to "keep an eye on it" but did not escalate, since the discrepancy seemed too minor to bother a busy senior OFA with.
Two weeks later, the same pattern turned up in a much larger ward, and the discrepancy turned out to be a real defect affecting how partial payments were totaled, one that had been quietly present the entire time.
OFA Interpretation Questions
- C.What made the discrepancy feel "too small to escalate" the first time it was noticed?
- D.What would a good escalation have looked like at the moment the discrepancy was first noticed, even without knowing yet whether it was a real defect?
Stewardship Application
- E.A discrepancy does not have to be fully understood before it is worth flagging, "I noticed something small and want a second opinion" is itself a legitimate, low-cost escalation.
- F.Waiting to escalate until a pattern becomes undeniable trades a cheap, early fix for a more expensive, later one.
II.
Applied Practice and Evaluation
A.
Applying What Was Learned
- 1.Scenario Context
- a.The learner is given a short description of a reported issue (for example, a form that fails to submit for some users but not others).
- 2.Required OFA Reasoning
- a.Identify what facts are still missing and what initial checks would narrow down the likely cause before escalating.
- 3.Required Output
- a.A note listing known facts, initial observations, and a specific escalation ask.
B.
Performance Indicators
- 1.Strong Performance Indicators
- a.The note distinguishes confirmed facts from assumptions.
- b.At least one reasonable initial check is proposed or performed before escalating.
- c.The escalation ask, if made, is specific and actionable.
- 2.Weak Performance Indicators
- a.The issue is escalated immediately with no initial assessment.
- b.Assumptions are presented as confirmed facts.
III.
Business Case Applications
A.
Case 1 — Data Migration
- 1.Scenario
- a.A batch of migrated ward records is ready for validation before the legacy spreadsheets are decommissioned.
- 2.Task
- a.Execute the assigned validation checks against a sample of records, recording actual results honestly, including any that do not match.
- 3.Deliverable
- a.A completed test record with any discrepancies flagged for the migration lead.
- 4.Evaluation Notes
- a.Discrepancies are reported rather than adjusted or explained away by the learner.
B.
Case 2 — System Integration
- 1.Scenario
- a.During a room reservation and calendar integration pilot, two units report a scheduling conflict on the same day.
- 2.Task
- a.Gather the basic facts of the conflict, check timestamps and logs if available.
- b.Form an initial observation before deciding whether to escalate.
- 3.Deliverable
- a.A note with known facts, initial observations, and an escalation ask if needed.
- 4.Evaluation Notes
- a.The note is based on verified facts, not on the first plausible explanation that rises..
C.
Case 3 — Deep Linking
- 1.Scenario
- a.During testing of the temple appointment deep link feature, some links return 404.
- 2.Task
- a.Identify which appointment types the failing links have in common before escalating, rather than reporting "some links are broken."
- 3.Deliverable
- a.A note identifying the failure pattern and a specific escalation ask.
- 4.Evaluation Notes
- a.The note names the specific pattern (for example, group appointments versus individual appointments) rather than describing the issue generically.