Skip to content

Add Microsoft.Testing.Extensions.HtmlReport extension#8283

Open
Evangelink wants to merge 5 commits into
mainfrom
evangelink/html-reporter
Open

Add Microsoft.Testing.Extensions.HtmlReport extension#8283
Evangelink wants to merge 5 commits into
mainfrom
evangelink/html-reporter

Conversation

@Evangelink
Copy link
Copy Markdown
Member

Closes #5754.

Summary

Adds a new MTP extension Microsoft.Testing.Extensions.HtmlReport that produces a single, self-contained HTML test report at the end of a test session.

  • CLI flags follow the TrxReport conventions:
    • --report-html — enable the HTML report.
    • --report-html-filename <name>.html — override the file name. Must be a pure file name (no path, no drive letter, no .., no leading/trailing whitespace, no invalid file-name char).
  • The report file is published as a SessionFileArtifact, so other extensions and dotnet test pick it up like a TRX file.

Screenshots

Default view — failures first, dynamic toolbar

Overview

Grouped by Class

Group by Class

Grouped by Duration

Six buckets: < 10 ms, 10 ms – 100 ms, 100 ms – 1 s, 1 s – 10 s, 10 s – 1 min, ≥ 1 min.

Group by Duration

Grouped by Trait — dark theme

The toolbar gets one Trait: <key> option per distinct trait/category found in the report.

Group by Trait (dark)

Multiple results for the same TestNode.Uid

Some test frameworks emit several distinct terminal-state results sharing the same TestNode.Uid (parameterized rows, data-driven theories, in-process retries, framework bugs). The extension never deduplicates — every observation is preserved and the affected rows get a #N of M badge so users can tell them apart.

Duplicate UIDs

UX details

  • Single HTML file, no CDN, all CSS/JS/data inlined → trivial to attach to PR comments, e-mail, archive as CI artifact.
  • Native CSS variables, automatic light + dark theme via prefers-color-scheme.
  • Header with summary cards (Total / Passed / Failed / Errored / Timed out / Skipped / Duration) and metadata block (framework, machine, user, started / finished, exit code).
  • Free-text search, outcome filter pills (Failed selected by default when there are failures), sortable columns (outcome / name / duration).
  • Group by: None / Class / Duration / Trait: <key> (one option per distinct trait key in the report). Group headers are collapsible and show per-outcome counts.
  • Expandable per-row detail panel with error message, exception type, stack trace, standard output, standard error.
  • Pagination (default 200 rows per page) keeps the report usable for very large runs.
  • Filter / sort / search / grouping state is persisted in sessionStorage so refreshing the page keeps your view.

Security: handling test-controlled content safely

The data is embedded as a JSON island inside <script type="application/json">. A hand-rolled HTML-safe JSON encoder escapes <, >, &, ', \u2028, \u2029 and all control characters as \uXXXX so that test-controlled content (display names, stack traces, stdout/stderr) cannot break out of the script block. The renderer always uses textContent, never innerHTML.

A regression test (GenerateReportAsync_EscapesScriptInjection_InDisplayName) verifies that hostile payloads such as </script><img onerror=alert(1)> never appear literally in the produced HTML.

Duplicate UIDs

Some test frameworks misuse TestNode.Uid (or legitimately reuse it across parameterized rows / in-process retries). The consumer therefore:

  1. Appends every terminal-state result — no observation is silently dropped.
  2. The engine does a pre-pass to count results per UID and emits attemptIndex / attemptOf only on rows whose UID is non-unique.
  3. The UI shows a #N of M badge on those rows and uses a synthetic per-row key for expand/collapse state so the rows stay independent.

Standard output / standard error

Already handled — captured from StandardOutputProperty / StandardErrorProperty and rendered in the expandable detail panel (truncated to 32 KiB with a UI-visible marker for very large streams).

Retries

For host-level retries (--retry-failed-tests), each attempt runs in its own results directory, so each gets its own HTML report — same behaviour as TRX. Within a single host run, in-process retries are surfaced via the #N of M mechanism above.

Performance

  • No reflection-based JSON serialization → works on netstandard2.0 without a System.Text.Json dependency, AOT/trim safe.
  • Per-test stdout / stderr (32 KiB) and stack traces (32 KiB) are truncated with a UI-visible suffix so 100k-test runs stay manageable.
  • Browser-side pagination at 200 rows per page by default.

Future hook: live updates (out of scope for this PR)

The page exposes a global window.mtpRender(newReport) function. A future OpenTelemetry / SSE / WebSocket integration can stream deltas and call this function to refresh the page in place without reloading. None of the current code is structured in a way that closes the door on this.

Files

  • src/Platform/Microsoft.Testing.Extensions.HtmlReport/
    • Microsoft.Testing.Extensions.HtmlReport.csproj (multi-targets netstandard2.0;$(SupportedNetFrameworks)).
    • HtmlReportExtensions.AddHtmlReportProvider() + TestingPlatformBuilderHook (auto-registered via MSBuild props).
    • HtmlReportGeneratorCommandLine, HtmlReportGenerator (IDataConsumer, ITestSessionLifetimeHandler, IDataProducer, IOutputDeviceDataProducer), HtmlReportEngine.
    • Templates/report-template.html (embedded resource).
    • Resources/ExtensionResources.resx + 12 .xlf files generated via dotnet msbuild ... /t:UpdateXlf.
    • PACKAGE.md, BannedSymbols.txt, PublicAPI/, build/, buildMultiTargeting/, buildTransitive/.
  • TestFx.slnx: project registered.
  • src/Platform/Microsoft.Testing.Platform/Microsoft.Testing.Platform.csproj: InternalsVisibleTo added.
  • test/IntegrationTests/.../HelpInfoAllExtensionsTests.cs: expectations updated for --report-html / --report-html-filename lines, the new HtmlReportGeneratorCommandLine --info block, and the new PackageReference in the AllExtensions test asset.
  • test/IntegrationTests/.../MSBuild.KnownExtensionRegistration.cs: assertion added for the new TestingPlatformBuilderHook and --report-html, plus a new PackageReference.
  • test/UnitTests/.../HtmlReportGeneratorCommandLineTests.cs and HtmlReportEngineTests.cs: 21 new tests covering CLI validation, JSON shape, traits, duplicate-UID handling, output truncation and HTML/JS injection.
  • docs/images/html-report/ — screenshots used in this PR description.

Public API

Only two public types / two members:

Microsoft.Testing.Extensions.HtmlReport.TestingPlatformBuilderHook
Microsoft.Testing.Extensions.HtmlReportExtensions

static TestingPlatformBuilderHook.AddExtensions(ITestApplicationBuilder, string[]) -> void
static HtmlReportExtensions.AddHtmlReportProvider(this ITestApplicationBuilder) -> void

Both declared in PublicAPI.Unshipped.txt. No init accessors.

Validation

  • Microsoft.Testing.Extensions.HtmlReport builds clean across netstandard2.0;net8.0;net9.0;net462;net472.
  • Microsoft.Testing.Platform and Microsoft.Testing.Platform.Acceptance.IntegrationTests build clean (0 warnings, 0 errors).
  • All 105 unit tests in Microsoft.Testing.Extensions.UnitTests pass on net9.0 (84 pre-existing + 21 new).
  • The acceptance HelpInfo / MSBuild known-extension-registration tests require ./build.cmd -pack to publish the new package before they can run end-to-end; the expectation files are updated so those tests will match on the next pack.

Evangelink and others added 2 commits May 16, 2026 12:21
Closes #5754.

New MTP extension `Microsoft.Testing.Extensions.HtmlReport` that
produces a single, self-contained HTML test report at the end of a
test session.

CLI options (mirroring TrxReport conventions):

  --report-html              enable the HTML report
  --report-html-filename     override the file name (must end with
                             `.html`, no path component)

The report is published as a SessionFileArtifact so other extensions
and dotnet test can pick it up.

UX

- Single file, no CDN: all CSS/JS/data are inlined. Easy to attach to
  PR comments, e-mail, archive as CI artifact.
- Native CSS variables, automatic light/dark theme via
  `prefers-color-scheme`.
- Summary cards, free text search, outcome filter pills (failed first
  by default), sortable columns.
- Group by: None / Class / Duration / Trait <key> (one option per
  distinct trait key in the report).
- Duration buckets: <10 ms, 10-100 ms, 100 ms-1 s, 1-10 s, 10-60 s,
  >=1 min.
- Expandable per-row detail panel with error message, stack trace,
  standard output, standard error.
- Pagination (200 rows / page) for large runs.
- UI state (filter / sort / search / grouping) persisted in
  sessionStorage.

Handling of duplicate UIDs

Some test frameworks emit multiple terminal-state results that share
the same `TestNode.Uid` (parameterized rows, in-process retries,
broken UID generators). The consumer never deduplicates: every
observation is preserved, and the engine annotates each row with
`attemptIndex` / `attemptOf` so the UI can display a `#N of M` badge
on the affected rows.

Security

The embedded JSON island is built with a hand-rolled HTML-safe JSON
encoder that escapes `<`, `>`, `&`, `'`, `\u2028`, `\u2029` as
`\uXXXX`, so test-controlled content (display names, stack traces,
stdout / stderr) cannot break out of the `<script type=
"application/json">` block. The renderer uses `textContent` only.

A regression test verifies that hostile content such as
`</script><img onerror=alert(1)>` never appears literally in the
produced HTML.

Performance

- No reflection-based JSON serialization (works on netstandard2.0
  without depending on System.Text.Json).
- Per-test stdout / stderr (32 KiB) and stack traces (32 KiB) are
  truncated with a UI-visible suffix to keep huge runs usable.
- Browser-side pagination at 200 rows / page by default.

Future hook (live mode)

The page exposes `window.mtpRender(newReport)`. A future OTel /
Server-Sent Events extension can stream deltas and call this function
to refresh the page in place without reloading.

Repo wiring

- TestFx.slnx: project registered under `/src/1 - Platform and Extensions/`.
- Microsoft.Testing.Platform.csproj: `InternalsVisibleTo` added.
- HelpInfoAllExtensionsTests / MSBuild.KnownExtensionRegistration:
  expectations updated for the new `--report-html` options and the
  new auto-registered builder hook.
- 21 new unit tests covering CLI validation, JSON shape, traits,
  duplicate-UID handling, output truncation and HTML/JS injection.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 16, 2026 10:23
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new Microsoft.Testing.Platform HTML report extension that can be auto-registered through MSBuild, exposes --report-html CLI options, generates a self-contained HTML session artifact, and adds unit/acceptance coverage for the new package.

Changes:

  • Introduces Microsoft.Testing.Extensions.HtmlReport with CLI registration, report generation, embedded HTML template, resources, packaging, and public API declarations.
  • Registers the extension in the solution, platform internals, MSBuild hook tests, and all-extension help/info acceptance expectations.
  • Adds unit tests for filename validation, JSON/report generation, duplicate UIDs, traits, truncation, and injection escaping.
Show a summary per file
File Description
TestFx.slnx Registers the new HtmlReport project.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/Microsoft.Testing.Extensions.UnitTests.csproj Adds test project reference to HtmlReport.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/HtmlReportGeneratorCommandLineTests.cs Adds CLI validation tests.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/HtmlReportEngineTests.cs Adds report engine/unit generation tests.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/MSBuild.KnownExtensionRegistration.cs Adds MSBuild auto-registration assertions.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs Updates help/info expectations and asset package references.
src/Platform/Microsoft.Testing.Platform/Microsoft.Testing.Platform.csproj Grants internals access to HtmlReport.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/TestingPlatformBuilderHook.cs Adds MSBuild builder hook.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/Templates/report-template.html Adds embedded HTML/CSS/JS report UI.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/Resources/ExtensionResources.resx Adds localizable extension strings.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/Resources/xlf/ExtensionResources.cs.xlf Adds Czech localization stub.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/Resources/xlf/ExtensionResources.de.xlf Adds German localization stub.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/Resources/xlf/ExtensionResources.es.xlf Adds Spanish localization stub.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/Resources/xlf/ExtensionResources.fr.xlf Adds French localization stub.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/Resources/xlf/ExtensionResources.it.xlf Adds Italian localization stub.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/Resources/xlf/ExtensionResources.ja.xlf Adds Japanese localization stub.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/Resources/xlf/ExtensionResources.ko.xlf Adds Korean localization stub.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/Resources/xlf/ExtensionResources.pl.xlf Adds Polish localization stub.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/Resources/xlf/ExtensionResources.pt-BR.xlf Adds Portuguese Brazil localization stub.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/Resources/xlf/ExtensionResources.ru.xlf Adds Russian localization stub.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/Resources/xlf/ExtensionResources.tr.xlf Adds Turkish localization stub.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/Resources/xlf/ExtensionResources.zh-Hans.xlf Adds Simplified Chinese localization stub.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/Resources/xlf/ExtensionResources.zh-Hant.xlf Adds Traditional Chinese localization stub.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/PublicAPI/PublicAPI.Unshipped.txt Declares new public API surface.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/PublicAPI/PublicAPI.Shipped.txt Initializes shipped API file.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/PACKAGE.md Adds NuGet package documentation.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/Microsoft.Testing.Extensions.HtmlReport.csproj Defines project/package layout and resources.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/HtmlReportGeneratorCommandLine.cs Adds CLI option provider and validation.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/HtmlReportGenerator.cs Adds data consumer/lifetime handler that emits the artifact.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/HtmlReportExtensions.cs Adds builder extension registration method.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/HtmlReportEngine.cs Generates HTML-safe JSON and writes the report file.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/buildTransitive/Microsoft.Testing.Extensions.HtmlReport.props Adds transitive package import.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/buildMultiTargeting/Microsoft.Testing.Extensions.HtmlReport.props Adds MSBuild self-registration metadata.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/build/Microsoft.Testing.Extensions.HtmlReport.props Adds build package import.
src/Platform/Microsoft.Testing.Extensions.HtmlReport/BannedSymbols.txt Adds analyzer banned-symbol rules.

Copilot's findings

  • Files reviewed: 35/40 changed files
  • Comments generated: 12

Comment on lines +513 to +515
// Default filter: if there's any failure, scope to "failed" first.
if (state.filter === "all" && (report.summary.failed || report.summary.errored || report.summary.timedOut)) {
state.filter = "failed";
Comment on lines +96 to +106
foreach (char c in fileName)
{
if (c is '/' or '\\' or ':')
{
return false;
}
}

foreach (char invalid in Path.GetInvalidFileNameChars())
{
if (fileName.IndexOf(invalid) >= 0)
Comment on lines +816 to +826
pager.hidden = true;
var frag = document.createDocumentFragment();
for (var g = 0; g < order.length; g++) {
var bucket = buckets[order[g]];
frag.appendChild(buildGroupHeader(bucket));
if (!state.collapsedGroups[bucket.id]) {
for (var j = 0; j < bucket.tests.length; j++) {
frag.appendChild(buildRow(bucket.tests[j]));
}
}
}
row.className = "test-row";
// For frameworks that emit multiple results for the same UID we need a key that is
// unique per row, not per UID, otherwise expanding one would expand all of them.
var rowKey = (t.attemptOf && t.attemptOf > 1) ? (t.uid + "#" + t.attemptIndex) : t.uid;
Comment on lines +89 to +91
// Note that we need to dispose the IFileStream, not the inner stream.
// IFileStream implementations will be responsible to dispose their inner stream.
using IFileStream stream = _fileSystem.NewFileStream(finalPath, fileNameExplicitlyProvided ? FileMode.Create : FileMode.CreateNew);
// For frameworks that emit multiple results for the same UID we need a key that is
// unique per row, not per UID, otherwise expanding one would expand all of them.
var rowKey = (t.attemptOf && t.attemptOf > 1) ? (t.uid + "#" + t.attemptIndex) : t.uid;
if (state.expanded[rowKey]) row.classList.add("expanded");
Comment on lines +831 to +844
var hdr = document.createElement("div");
hdr.className = "group-header" + (state.collapsedGroups[bucket.id] ? " collapsed" : "");

var chev = document.createElement("div");
chev.className = "chevron";
chev.textContent = "▾";
hdr.appendChild(chev);

var title = document.createElement("div");
title.className = "title";
title.textContent = groupLabel(bucket.id) || bucket.id;
hdr.appendChild(title);

var counts = document.createElement("div");
Comment on lines +20 to +21
/// <inheritdoc />
public string Version => ExtensionVersion.DefaultSemVer;
Comment on lines +91 to +92
if (fileName == "." || fileName == ".." || fileName.Contains(".."))
{
Comment on lines +103 to +107
// Append every terminal-state result. We intentionally do NOT deduplicate on
// TestNode.Uid: some test frameworks emit several distinct results sharing the
// same UID (parameterized rows, theory data, in-process retries, framework
// misbehavior). The engine surfaces all of them so no data is silently dropped.
_tests.Add(update);
Copy link
Copy Markdown
Member Author

@Evangelink Evangelink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary — PR #8283: Microsoft.Testing.Extensions.HtmlReport

# Dimension Status
1 Algorithmic Correctness ✅ LGTM
2 Threading & Concurrency ✅ LGTM — ConsumeAsync is always invoked from a single reader task (SingleReader = true); _tests and _testStartTime are not subject to concurrent mutation
3 Security & IPC Contract Safety ✅ LGTM — AppendString correctly escapes <, >, &, ', \u2028, \u2029; IsValidPureFileName blocks path traversal
4 Public API & Binary Compatibility ✅ LGTM — all public types declared in PublicAPI.Unshipped.txt, XML docs present, no init accessors
5 Performance & Allocations ⚠️ MODERATE — OfType<TestMetadataProperty>() allocates per test node in BuildJson hot path
6 Cross-TFM Compatibility ✅ LGTM — #if NETCOREAPP guards, polyfills included, no missing API usage
7 Resource & IDisposable Management ✅ LGTM — IFileStream wrapped in using
8 Localization & Resources ✅ LGTM — all user-facing strings in .resx, no hardcoded UI strings
9 Test Completeness & Coverage 🚫 MAJOR — no unit tests ship with this new extension

Blocking Concern

The PR adds a non-trivial new extension — custom JSON serializer with HTML escaping, filename validation with six rejection branches, duplicate-UID attempt tracking, field truncation — but ships zero unit tests. The codebase's test infrastructure (TestFramework.ForTestingMSTest) is already available. Both IsValidPureFileName and BuildJson are well-suited to pure unit testing (deterministic inputs/outputs, no I/O).

Generated by Expert Code Review (on open) for issue #8283 · ● 19.4M

Dictionary<string, int> countByUid = new(testNodes.Length);
foreach (TestNodeUpdateMessage n in testNodes)
{
string uid = n.TestNode.Uid.Value;
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR] Test Completeness & Coverage

BuildJson contains several non-trivial code paths that are completely untested:

  • Each outcome classification branch (passed, failed, skipped, timedOut, errored)
  • The duplicate-UID attemptIndex/attemptOf annotation
  • Field truncation at MaxMessageLength, MaxStackTraceLength, MaxStandardStreamLength boundaries
  • HTML-safe escaping of characters like <, >, &, \u2028 in test-controlled content

A broken JSON payload would silently corrupt every generated report without any test catching it.

Recommendation: Add unit tests for BuildJson covering each outcome, duplicate UIDs (verify correct attemptIndex/attemptOf), truncation at the exact boundary, and HTML-unsafe character escaping.

}

string? stdout = node.Properties.SingleOrDefault<StandardOutputProperty>()?.StandardOutput;
string? stderr = node.Properties.SingleOrDefault<StandardErrorProperty>()?.StandardError;
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MODERATE] Performance & Allocations

In a large run (e.g. 50 000 tests), node.Properties.OfType<TestMetadataProperty>() allocates a new LINQ enumerator object on every iteration of the outer foreach loop — one heap object per test node.

Recommendation: Replace the LINQ OfType<T>() call with a plain loop and an is check to avoid the allocation:

foreach (TestProperty prop in node.Properties)
{
    if (prop is not TestMetadataProperty meta) continue;
    // ... existing body ...
}

return ValidationResult.InvalidTask(ExtensionResources.HtmlReportFileNameRequiresHtmlReport);
}

return commandLineOptions.IsOptionSet(HtmlReportOptionName)
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR] Test Completeness & Coverage

IsValidPureFileName has six distinct rejection branches (null/whitespace, leading/trailing whitespace, .. traversal, //\/: separator, Path.GetInvalidFileNameChars() scan, non-.html extension) but the PR ships no unit tests. A regression in any branch would go undetected.

Recommendation: Add unit tests covering each rejection case (e.g. "../evil.html", "sub/dir.html", " spaced.html", ".", "..", "report.txt") and at least one valid case ("report.html" → passes validation).

Evangelink and others added 3 commits May 16, 2026 16:26
Addresses both the Copilot reviewer findings and the Expert reviewer
critique on PR #8283.

Engine & generator
- Project incoming TestNodeUpdateMessage into a capped DTO
  (CapturedTestResult) immediately at consume time, so we no longer
  retain entire test node payloads for the whole session. Truncation
  caps protect the test host process, not only the file.
- Replace OfType<TestMetadataProperty>() with a plain foreach + 'is'
  check to avoid the per-test LINQ enumerator allocation.
- Add a retry loop for default file names: if the second-precision
  default file already exists, append "_1", "_2", ... until a unique
  name is found (5 s overall budget), mirroring TrxReportEngine.
- Emit a stable per-row rowKey so the UI can use it as the expand-
  state key. This removes the synthetic "uid#attempt" key, which
  could collide with a UID that genuinely contains "#N".

Filename validation
- Replace Path.GetInvalidFileNameChars() (OS-dependent) with a hard-
  coded list that is a strict superset of Windows + Unix invalid file
  name characters. The same input is now rejected regardless of host.
- Reject Windows reserved device names (CON, PRN, AUX, NUL, COM0..9,
  LPT0..9) up-front so the option doesn't pass validation only to
  fail later during file creation.

HTML template
- Default filter: when only errored or only timed-out tests are
  present (no real failures), open the report on that filter instead
  of "failed" (which would have shown an empty list).
- Convert filter pills and sortable column headers from <span>/<div>
  to <button>, with aria-pressed / aria-sort state.
- Make expandable test rows and group headers focusable (tabindex=0,
  role=button, aria-expanded) with Enter/Space keyboard activation.
- Cap rendered rows per group (default 200) with "Show more" / "Show
  all" controls so switching to grouped view doesn't materialize tens
  of thousands of DOM nodes on a huge run.

Tests
- Add the HtmlReportGeneratorCommandLine to ExtensionVersionTests.
- Add unit tests for Windows-invalid characters (*, ?, ", <, >, |),
  reserved Windows device names (CON, PRN, ...) on any OS, and a
  positive case for names like CONfig.html that just start with a
  reserved prefix.
- Add unit tests for TestResultCapture: outcome classification for
  every well-known state, truncation at + 7 over the boundary,
  no-truncation when exactly at the boundary, null result for
  discovered/in-progress states.
- Add HTML/JS escape tests for each of <, >, &, ', U+2028, U+2029.
- Add a duplicate-UID test asserting attemptIndex / attemptOf
  emission and that unique UIDs do NOT get those fields.
- Add a stable per-row rowKey test.
- Add a default file-name retry test that asserts WriteAsync is
  called twice when the first file already exists and the resulting
  path has the "_1" disambiguation suffix.

Build is clean: 0 warnings, 0 errors across the extension, the
platform, the integration test project, and the extension unit test
project on net462, net472, net8.0 and net9.0.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 16, 2026 15:09
@Evangelink
Copy link
Copy Markdown
Member Author

Thanks for the thorough review — addressed every finding. Force-pushed evangelink/html-reporter (f97c4ccf6).

build.cmd is clean: 0 Warning(s), 0 Error(s) across the entire repo (debug build, net462, net472, net8.0, net9.0, netstandard2.0).

Code changes

# Comment Fix
1 Default filter doesn't surface errored/timedOut-only runs When summary has no failed but has errored or timedOut, open on that specific filter rather than failed. template.html state.filter init.
2 Path.GetInvalidFileNameChars() is OS-dependent (Linux accepts *?<>"|) Replaced with a hard-coded InvalidFileNameChars array that is a strict superset of Windows + Unix invalid characters, so the same input is rejected on every host.
3 Grouped views bypass pagination, hangs browser on huge runs Per-group rendering is now capped (default 200 rows/group) with Show more / Show all buttons. shownPerGroup state is tracked separately and persisted.
4 Synthetic key uid#N can collide with a real UID containing #N Engine now emits a stable per-row rowKey (the result index). UI uses that key for expand-state — no derivation, no possible collision.
5 Default filename has second precision + FileMode.CreateNewIOException on rapid reruns Added a 5 s retry loop that appends _1, _2, … to the default name until a unique one is found. Explicit --report-html-filename still uses FileMode.Create (overwrite) with the existing warning. Mirrors TrxReportEngine.RetryWhenIOExceptionAsync.
6 Filter pills are <span>, not keyboard-accessible Converted to <button> with aria-pressed state and :focus-visible styling.
7 Sortable column headers are <div> with click only Converted to <button> with aria-sort (ascending/descending/none) and keyboard focus.
8 Test rows aren't focusable, no expanded/collapsed announcement Rows with details now get role="button", tabindex="0", aria-expanded, and an Enter/Space keydown handler.
9 Group headers aren't keyboard accessible Group headers got the same role="button", tabindex="0", aria-expanded and keyboard handler.
10 New provider not covered by ExtensionVersionTests Added HtmlReportGeneratorCommandLine_UsesItsOwnAssemblyVersion.
11 Reserved Windows device names (CON.html, NUL.html, …) accepted New WindowsReservedNames table; the bare name (without extension) is compared case-insensitively against CON, PRN, AUX, NUL, COM0..9, LPT0..9. New regression tests + a positive test that CONfig.html is still accepted.
12 Generator retains entire TestNodeUpdateMessage (with huge stdout/stderr) in memory New CapturedTestResult DTO + TestResultCapture.TryCapture() projects each message at consume time. Truncation now caps at consume, so memory is bounded by the per-field caps × number of results — the engine no longer needs to truncate at all.
13 Missing engine tests Added: outcome classification for every well-known state, truncation at +7 over the boundary, exact-boundary non-truncation, null-return for non-terminal states, escape tests for each of <, >, &, ', U+2028, U+2029, stable rowKey emission, default-filename retry, duplicate-UID attemptIndex/attemptOf annotation (with the unique-UID negative case).
14 OfType<TestMetadataProperty>() allocates per node Replaced with foreach + is pattern in TestResultCapture so no enumerator object is allocated per test.
15 Missing CLI validator tests Added cases for each rejection branch including the new cross-platform invalid chars and reserved Windows names; the positive case report.html is also covered.

Test counts

  • Microsoft.Testing.Extensions.UnitTests filter ~HtmlReport: 50 tests, 0 failures on net9.0 (DataRow expansions included).

Screenshots refreshed

The screenshots in docs/images/html-report/ have been regenerated against the updated UI so the focus styles and ARIA-converted controls render correctly.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 38/43 changed files
  • Comments generated: 3

Comment on lines +524 to +532
// Restore persisted UI state per report file (best-effort).
try {
var saved = JSON.parse(sessionStorage.getItem("mtp-state") || "{}");
if (saved.filter) state.filter = saved.filter;
if (saved.sort) state.sort = saved.sort;
if (saved.sortDir) state.sortDir = saved.sortDir;
if (saved.search) state.search = saved.search;
if (saved.groupBy) state.groupBy = saved.groupBy;
if (typeof saved.pageSize === "number") state.pageSize = saved.pageSize;
Comment on lines +480 to +483
<div class="results-header" role="row">
<button type="button" class="sortable" data-sort="outcome" aria-sort="none">Outcome</button>
<button type="button" class="sortable" data-sort="name" aria-sort="none">Test</button>
<button type="button" class="sortable duration" data-sort="duration" aria-sort="none" style="text-align:right">Duration</button>
Comment on lines +149 to +155
await _messageBus.PublishAsync(
this,
new SessionFileArtifact(
testSessionContext.SessionUid,
new FileInfo(reportFileName),
ExtensionResources.HtmlReportArtifactDisplayName,
ExtensionResources.HtmlReportArtifactDescription)).ConfigureAwait(false);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HTML logger porting

3 participants