Skip to content

fix(test): make the native-windows test suite build and run - #11482

Merged
thebentern merged 2 commits into
developfrom
fix-native-windows-tests
Aug 14, 2026
Merged

fix(test): make the native-windows test suite build and run#11482
thebentern merged 2 commits into
developfrom
fix-native-windows-tests

Conversation

@caveman99

@caveman99 caveman99 commented Aug 13, 2026

Copy link
Copy Markdown
Member

pio test -e native-windows failed all 53 suites at the build stage. Five independent causes, all Windows-only, plus one Unity configuration gap that turned out not to be Windows-specific at all.

Causes

lstat() in test/TestUtil.cpp. MinGW-w64 does not provide it. The state-checkpoint walk added in #11322 is fenced with ARCH_PORTDUINO, which native-windows also satisfies, so the file failed to compile and took every suite with it. This is the regression that broke the Windows runner; the others were masked behind it. Routed through a stat() shim on _WIN32.

Missing setUp/tearDown in four suites. test_default, test_http_content_handler, test_meshpacket_serializer and test_serial define neither and relied on the weak defaults PlatformIO emits in its generated unity_config.c. Those defaults are enabled on MinGW, but GCC lowers a weak definition on PE-COFF to a weak external:

0000000000000000 T .weak.setUp.unityOutputStart
                 w setUp

setUp stays undefined (w) with a renamed alternate, rather than ELF's weak definition (W), so it does not satisfy unity.c's strong reference and the link fails. Defined explicitly, as the other 49 suites already do.

<arpa/inet.h> in test_mqtt. Absent on MinGW; included only for htonl(). Uses winsock2.h there.

UNITY_INCLUDE_DOUBLE was never defined anywhere. Unity omits double support unless asked, so TEST_ASSERT_DOUBLE_* compiles into an unconditional "Unity Double Precision Disabled" failure. This is not Windows-specific. Verified on Debian with gcc, against the Linux env's own Unity 2.6.1 and PlatformIO's generated native unity_config:

UNITY_INCLUDE_DOUBLE : NOT defined
UNITY_EXCLUDE_DOUBLE : defined
test_double_within:FAIL: Unity Double Precision Disabled

The macro appears nowhere in the repo, the ini files, test_native.yml, or PlatformIO's Unity runner, which adds only UNITY_INCLUDE_CONFIG_H. It is therefore defined in portduino_base.build_flags_common, which every native env resolves: native, native-tft, native-fb, native-tft-debug, coverage, coverage-event-policy, native-macos, native-windows, native-wasm.

test_getfiles_rejects_overlong_path. Excluded on _WIN32. Overrunning the 228-byte file_name needs at least 229 bytes below the portduino root, and that root is already about 34 bytes, so every qualifying path passes the 260-byte MAX_PATH. The nested mkdir() fails, the file is never created, and getFiles() has nothing to drop. No component layout satisfies both limits.

Verification

Full pio test -e native-windows on Windows 11 / UCRT64 gcc 16.1.0, before and after:

Before After
Suites built and run 46 of 53 53 of 53
Test cases 839 938
Failed 2, plus 7 suites that never built 1

Scope

Every guard is _WIN32-only except UNITY_INCLUDE_DOUBLE, which now applies to all native envs and does change Linux and macOS: TEST_ASSERT_DOUBLE_* becomes a real comparison instead of a stub that always fails. test_gps_update_scheduling is the only suite in the repo using those macros, and its arithmetic is integer-based and bit-identical across platforms, so it should pass wherever it runs.

Worth flagging for review: that suite currently reports PASSED on CI in 0.03 seconds while emitting no Unity output at all, so its assertions appear never to execute there. That is a separate issue and is not addressed here.

No CI job currently compiles the test tree for native-windows: test_native.yml runs on Ubuntu against -e coverage, and build_windows_bin.yml runs pio run, never pio test. That is why the lstat regression stood for a week.

Known limitation

test_fscommon_getfiles still fails in a full run, for a cause outside this change: VFSImpl::rmdir() in framework-portduino calls unlink(), which cannot remove a directory, so the tree its depth-limit case creates survives setUp()'s rmDir() and makes getFiles() report a spurious depth truncation. Fixed by meshtastic/framework-portduino#77; once that lands and the platform pin picks it up, the suite goes green on Windows with no further change here.

Summary by CodeRabbit

  • Bug Fixes

    • Improved cross-platform directory traversal and network byte-order handling for Windows builds.
    • Excluded unsupported long-path tests on Windows.
  • Tests

    • Improved test compatibility with MinGW by adding required test lifecycle hooks.
    • Enabled double-precision assertions for GPS scheduling tests.

pio test -e native-windows failed every suite at the build stage. Five
independent causes, all Windows-only:

- TestUtil.cpp called lstat(), which MinGW-w64 does not provide. The
  state-checkpoint walk added in #11322 is fenced with ARCH_PORTDUINO,
  which native-windows also satisfies, so all 53 suites failed to
  compile. Route it through a stat() shim on _WIN32.

- test_default, test_http_content_handler, test_meshpacket_serializer
  and test_serial define no setUp/tearDown and relied on the weak
  defaults PlatformIO emits in unity_config.c. GCC lowers a weak
  definition on PE-COFF to a weak external, leaving the symbol
  undefined, so it does not satisfy unity.c's reference and the link
  fails. Define them explicitly, as the other 49 suites already do.

- test_mqtt included <arpa/inet.h>, absent on MinGW, for htonl(). Use
  winsock2.h there.

- test_gps_update_scheduling uses TEST_ASSERT_DOUBLE_WITHIN. Unity
  omits double support unless UNITY_INCLUDE_DOUBLE is defined, so the
  assertion compiled to an unconditional failure. Define it for the
  env.

- test_getfiles_rejects_overlong_path is excluded on _WIN32. Overrunning
  the 228-byte file_name needs at least 229 bytes below the portduino
  root, and that root is already ~34 bytes, so every qualifying path
  passes the 260-byte MAX_PATH: the nested mkdir() fails, the file is
  never created, and getFiles() has nothing to drop. No component
  layout satisfies both limits.

Each of the seven suites that failed on Windows was verified
individually after the change. test_fscommon_getfiles still fails in a
full run, for a cause outside this change: rmDir() does not remove
directories on Windows, so empty dirs left by an earlier run survive
setUp() and make getFiles() report a depth truncation. That is a
pre-existing FSCommon bug, reported separately.

No Linux or macOS behaviour changes: every guard is _WIN32-only except
UNITY_INCLUDE_DOUBLE, which is scoped to env:native-windows.
@coderabbitai

coderabbitai Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 2d7c6fc8-efbc-45b3-98b5-f34d0d44af79

📥 Commits

Reviewing files that changed from the base of the PR and between c9b3cc9 and 1191f87.

📒 Files selected for processing (1)
  • variants/native/portduino.ini

📝 Walkthrough

Walkthrough

The changes improve native test compatibility across Windows, non-Windows platforms, and MinGW. They add platform-specific filesystem and networking handling, Unity lifecycle hooks, Windows-specific test guards, and Unity double-precision support.

Changes

Native test compatibility

Layer / File(s) Summary
Cross-platform filesystem traversal
test/TestUtil.cpp
Directory traversal uses stat on Windows and lstat on other platforms.
MinGW Unity lifecycle hooks
test/test_default/test_main.cpp, test/test_http_content_handler/test_main.cpp, test/test_meshpacket_serializer/test_serializer.cpp, test/test_serial/SerialModule.cpp
Test targets define empty setUp and tearDown callbacks for MinGW linking.
Windows test conditions and headers
test/test_mqtt/MQTT.cpp, test/test_fscommon_getfiles/test_main.cpp, variants/native/portduino.ini
Windows uses winsock2.h, skips the unsupported overlong-path test, and enables Unity double-precision support.

Estimated code review effort: 2 (Simple) | ~10 minutes

Mergeability Score: ⚪ Minimal · up to 1191f

This change is limited to making the native-Windows test suite build and run, with no actionable merge-blocking risk remaining after normal checks and review.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes the primary change: making the native Windows test suite build and run.
Description check ✅ Passed The description is detailed and relevant, covering causes, fixes, verification results, scope, and the known remaining limitation.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix-native-windows-tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

⚡ Try this PR in the Web Flasher

Note

Building this pull request… the flash button, badges and supported-board
list will appear here automatically once CI finishes.

@caveman99 caveman99 added the bugfix Pull request that fixes bugs label Aug 13, 2026
@caveman99

Copy link
Copy Markdown
Member Author

The test_fscommon_getfiles limitation described above is fixed by meshtastic/framework-portduino#77: VFSImpl::rmdir() called unlink(), which cannot remove a directory (EISDIR on Linux, EACCES on Windows), so rmDir() never cleared the directory tree its depth-limit case creates and the leftovers made a later run report a spurious depth truncation.

Once that lands and the platform pin picks it up, the suite goes green on Windows with no further change here.

@caveman99
caveman99 requested review from NomDeTom and thebentern and removed request for NomDeTom August 13, 2026 12:52
The flag was scoped to env:native-windows, but the gap is not
Windows-specific. Verified on Debian with gcc against the Linux env's
own Unity 2.6.1 and PlatformIO's generated native unity_config:

  UNITY_INCLUDE_DOUBLE : NOT defined
  UNITY_EXCLUDE_DOUBLE : defined
  test_double_within:FAIL: Unity Double Precision Disabled

UNITY_INCLUDE_DOUBLE appears nowhere in the repo, the ini files, the
workflow, or PlatformIO's unity runner, which adds only
UNITY_INCLUDE_CONFIG_H. So TEST_ASSERT_DOUBLE_* is an always-failing
stub on Linux and macOS too, not only on Windows.

Moved to portduino_base.build_flags_common, which every native env
resolves: native, native-tft, native-fb, native-tft-debug, coverage,
coverage-event-policy, native-macos, native-windows and native-wasm.

This does change Linux and macOS: TEST_ASSERT_DOUBLE_* becomes a real
comparison instead of a stub. test_gps_update_scheduling is the only
suite using those macros and its arithmetic is integer-based and
bit-identical across platforms, so it should pass wherever it runs.
Note it currently reports PASSED on CI in 0.03s while emitting no Unity
output at all, so those assertions appear never to execute there; that
is tracked separately and is not addressed here.
@caveman99
caveman99 enabled auto-merge August 13, 2026 16:54
@caveman99
caveman99 added this pull request to the merge queue Aug 13, 2026
@vidplace7
vidplace7 removed this pull request from the merge queue due to a manual request Aug 13, 2026
@thebentern
thebentern added this pull request to the merge queue Aug 14, 2026
Merged via the queue into develop with commit 3468083 Aug 14, 2026
66 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Pull request that fixes bugs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants