Add Qualflare TestNG to Testing > Miscellaneous - #1328
Open
ibrahimsn98 wants to merge 1 commit into
Open
ibrahimsn98 wants to merge 1 commit into
ibrahimsn98 wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds Qualflare TestNG under Testing → Miscellaneous in
README_SOURCE.md. None of the focused subsections fit a result reporter — it is not a framework, matcher, mock or fixture — so Miscellaneous per the guidelines.Why it is noteworthy (fills a useful niche): TestNG hands a retried failure to
onTestSkippedwithwasRetried()set. The consequence is that Surefire's JUnit XML records a test which failed twice and passed on the thirdretryAnalyzerattempt as two<skipped/>entries and one pass — with no<failure>element and no stack trace. Every tool that reads that file is working from a record the failures were already removed from. This listener reads TestNG's callbacks instead, so all three attempts are preserved with their errors, and it separates timeouts (onTestFailedWithTimeout) and configuration failures (onConfigurationFailure) from assertion failures — distinctions the JUnit schema cannot express.Distinct from similar entries:
junit-dataprovideris a JUnit runner, not reporting. ReportNG and ExtentReports render HTML from results after the fact; this runs inside the run and captures what the result files drop. It is a listener only — no runner, no assertions, no HTML.Java-first and developer-friendly:
com.qualflare:qualflare-testng, signed releases.wasRetried()must exist) and Java 11+. CI covers Java 11/17/21 and TestNG 7.4.0 and 7.12.0.ServiceLoadersupport — adding the test dependency is the entire setup.Disclosure: I'm the author. I've followed the one-line format in
README_SOURCE.mdand leftREADME.mduntouched since it is generated; happy to reword or move it.Summary by cubic
Adds Qualflare TestNG to the Testing > Miscellaneous section of
README_SOURCE.md. The tool is a TestNG listener that preserves every retry attempt and reports timeouts and configuration failures as distinct statuses, avoiding Surefire's JUnit XML behavior of recording retried failures as skipped and losing stack traces.Written for commit 377f11a. Summary will update on new commits.