Skip to content

Add Qualflare TestNG to Testing > Miscellaneous - #1328

Open
ibrahimsn98 wants to merge 1 commit into
akullpp:mainfrom
ibrahimsn98:add-qualflare-testng
Open

ibrahimsn98 wants to merge 1 commit into
akullpp:mainfrom
ibrahimsn98:add-qualflare-testng

Conversation

@ibrahimsn98

@ibrahimsn98 ibrahimsn98 commented Sep 15, 2026

Copy link
Copy Markdown

Adds Qualflare TestNG under Testing → Miscellaneous in README_SOURCE.md. None of the focused subsections fit a result reporter — it is not a framework, matcher, mock or fixture — so Miscellaneous per the guidelines.

Why it is noteworthy (fills a useful niche): TestNG hands a retried failure to onTestSkipped with wasRetried() set. The consequence is that Surefire's JUnit XML records a test which failed twice and passed on the third retryAnalyzer attempt as two <skipped/> entries and one pass — with no <failure> element and no stack trace. Every tool that reads that file is working from a record the failures were already removed from. This listener reads TestNG's callbacks instead, so all three attempts are preserved with their errors, and it separates timeouts (onTestFailedWithTimeout) and configuration failures (onConfigurationFailure) from assertion failures — distinctions the JUnit schema cannot express.

Distinct from similar entries: junit-dataprovider is a JUnit runner, not reporting. ReportNG and ExtentReports render HTML from results after the fact; this runs inside the run and captures what the result files drop. It is a listener only — no runner, no assertions, no HTML.

Java-first and developer-friendly:

  • Apache-2.0, on Maven Central as com.qualflare:qualflare-testng, signed releases.
  • Requires TestNG 7.4.0+ (wasRetried() must exist) and Java 11+. CI covers Java 11/17/21 and TestNG 7.4.0 and 7.12.0.
  • Zero runtime dependencies, and registers itself through TestNG's ServiceLoader support — adding the test dependency is the entire setup.
  • English docs in the README; no account needed to use the library, which writes a local report directory and makes no network calls. Uploading to the hosted Qualflare service is optional and has a free tier.

Disclosure: I'm the author. I've followed the one-line format in README_SOURCE.md and left README.md untouched since it is generated; happy to reword or move it.


Summary by cubic

Adds Qualflare TestNG to the Testing > Miscellaneous section of README_SOURCE.md. The tool is a TestNG listener that preserves every retry attempt and reports timeouts and configuration failures as distinct statuses, avoiding Surefire's JUnit XML behavior of recording retried failures as skipped and losing stack traces.

Written for commit 377f11a. Summary will update on new commits.

Review in cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Re-trigger cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant