Readiness Questions

5 min read

Ask Oracle whether a release, test run or release execution is ready, and it works out the answer from the latest results and defects.

These readiness figures aren’t stored on the record. Oracle calculates them when you ask, so Releases that are risky is a question, not a report you have to build.

Ask in Plain Terms

You don’t need to name the figures. Just ask:

Ask	Answered from
`Releases that are risky`	the verdict
`Test runs still in progress that are healthy`	the verdict
`Explain why the August release is not healthy`	the reason behind the verdict
`Releases with open blocker or critical defects`	the number of open blocker or critical defects
`Releases where failures have no defect recorded`	failures with no defect
`Test runs with no work left in them`	work remaining
`Test runs that are failing heavily`	fail rate
`Release executions that have stalled`	run condition

Healthy Means the Verdict, Not the Score

There are two different measures, and they can disagree:

The verdict: Ready, At risk, Not ready or Not started. It is decided by the readiness gates, a set of pass/fail checks.
The score: a number out of 100.

A release with one open blocker can score 88 and still be Not ready. The blocker check failed, so the score doesn’t matter.

Not started is not Not ready. If nobody has run anything yet, nothing has failed. Oracle keeps the two apart, so Releases that are risky doesn’t include work that simply hasn’t started.

The Reason, Not Just the Result

Explain why the August release is not healthy returns the check that actually caused the verdict, not a list of everything that is less than perfect.

You can also ask which readiness gates didn’t pass. They are:

Execution coverage
Fail rate under control
No open blocker / critical defects
Every failure has a defect
Requirements verified

If nothing in the release or run is traced to a requirement, the Requirements verified gate is left out. That is different from failing it.

Signed-Off Figures Are Frozen

When a release is moved to Completed or Archived, Hawzu saves its readiness figures and the acceptance criteria it was judged against. From then on, its readiness shows what was true at sign-off, not what is true today. If the release is reopened, the saved figures are dropped and it shows live figures again.

This keeps comparisons fair: every signed-off release shows the figures from its own sign-off.

Counting Failures

Releases where failures have no defect recorded counts results, not test cases. A test case run with four sets of data that failed on three of them is three failures, not one.

A closed defect still counts. The question is whether the failure was ever written up, not whether the defect is still open.

What You Can See Depends on Defects

Needs only execution results	Needs access to defects
Execution coverage, pass rate, fail rate	Readiness verdict and its reason
Results total, executed, remaining	Readiness score
Run conditions: stalled, blocked, failing heavily	Open blocker or critical defects, failures with no defect

A role that can’t read defects can still ask which runs have stalled or how far a run has got. Without defect access you can’t see verdicts or scores, either as a column or as a sort.

See Limits and Permissions for how permissions work across Oracle.

Failures, and What They Affect

Oracle can list the results in a release and what each one is traced to, in one question: which tests failed in the August release and what requirements do they affect.

You can count them per requirement too: failures per requirement in the August release. A test case traced to three requirements counts once for each, so the counts can add up to more than the number of failures.

Counting Verdicts and Gates

You can group by readiness:

Releases by readiness verdict: how many are ready, at risk, not ready or not started
Which gate fails most often across our releases: the check your team keeps missing
Test runs by verdict

A release failing three checks counts once under each, so the gate counts can add up to more than the number of releases.

A grouped answer can’t also show calculated columns. Releases by verdict works, but asking for each release’s readiness score in the same question doesn’t, because a group is a count, not a single release. Ask for the list of releases, or ask for the groups.

What Can’t Be Asked

Readiness can’t be charted. You can chart releases by pass rate, execution coverage and open critical defects, but not by verdict or score. Oracle can still sort and filter by them.
The explanation needs a single release or run. Explain why the August release is not ready gives the reason because the answer is one release. A question that returns several shows the list without the explanation, so ask about one at a time.
More than two hundred at once. If readiness would have to be worked out for more than two hundred releases, runs or executions, Oracle won’t answer rather than answering for some of them. Narrow the question with a date range or a status.

Comparing Scores Across Releases

Every release is scored against the same default acceptance criteria today, and a signed-off release keeps the criteria it was judged against. If the defaults change, an older signed-off score may have been judged against different bars, so compare across time with care. The verdict is a better guide, because it reflects each release’s own criteria.