How Results Map to Test Cases
3 min read
Each row of a report has to become a result on the right test case, and stay right when someone renames a test.
How Hawzu finds the test case
Section titled “How Hawzu finds the test case”Matching on the test’s name alone breaks as soon as someone renames the test. Hawzu tries these in order for each report row:
-
An explicit Hawzu code stated by the report. The most robust option, because it survives renaming the test.
-
A test case code in the test name or classname, for example
CHK-123 logs in successfully. No tooling required — just put the code in the test title. -
An exact match on the test case title. A convenient fallback; an ambiguous title is never guessed at.
Codes match regardless of upper or lower case and extra spaces.
Stating the code explicitly
Section titled “Stating the code explicitly”| Format | How |
|---|---|
| JUnit | <property name="testcase_code" value="CHK-123"/> on the <testcase> (or the suite) |
| TRX | An MSTest trait named testcase_code |
| NUnit | <property name="testcase_code" value="CHK-123"/> |
| Cucumber | A scenario tag: @CHK-123 |
| Allure | A label named testcase_code, or a CHK-123 tag |
One test covering several test cases
Section titled “One test covering several test cases”An end-to-end test often stands in for several manual test cases. Name them all and each one receives the same verdict:
Or state them explicitly — as one delimited value, or as repeated properties:
If the run doesn’t contain one of the named cases, that case is listed individually as unmatched, so a partially covered test is visible rather than quietly leaving cases Not Executed.
Status mapping
Section titled “Status mapping”Results are turned into Hawzu’s statuses:
| Hawzu | Comes from |
|---|---|
Passed | A passing test |
Failed | A failure or error |
Skipped | A skipped/ignored test, or a JUnit <skipped> |
Blocked | Allure broken, NUnit/TRX Inconclusive, TRX Blocked, a Cucumber undefined/pending step — cases where the test never reached a verdict, so the feature was never actually judged |
JUnit has no way to express Blocked. To set it, add <property name="hawzu_status" value="Blocked"/> to the test case.
Automation never sets Needs Rerun. Only a person can.
When several report rows map to one test case — parametrised or retried suites — the worst status wins. A case that failed any variant is not green.
The failure message, duration, and stack trace are kept as the result comment.
Scope: which test cases a report may touch
Section titled “Scope: which test cases a report may touch”This differs by run type.
On a manual test run, the test cases someone picked are the scope. A report row naming a case outside the run is recorded as unmatched and changes nothing.
On an automated test run, the report can extend the scope. A row naming a code that exists in the project’s repository is added to the run and given its result. Only a code match adds a case, never a title match, so a test with a similar title can’t pull an unrelated case into a run.
In both cases, test cases that are in the run but not in the report stay Not Executed, so you can see what you meant to automate but didn’t run.
Rows that match nothing
Section titled “Rows that match nothing”Every row is recorded with an outcome, and the two unmatched outcomes are visible on the build:
- Not in this run — the row named a real test case that is not part of this run.
- No matching test case — nothing in the project matched the row.
Unmatched rows never affect status counts, analytics, test case history, or release readiness. They are kept so you can see what your pipeline runs that doesn’t match any test case, which is what the Automation Health report summarises over time.