Full Rebuild September 2026 + Sync cross-distribution Vinca package coverage - #271
Tobias-Fischer wants to merge 113 commits into
Conversation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Port pixi.toml structural changes: update vinca rev to 6aacb6f, inline platform-based glibc, pixi-build preview, PYTHONIOENCODING activation env (structural changes already in main, ensuring rev sync) - Sync .github/workflows/testpr.yml: add permissions block, upgrade setup-pixi to v0.10.0 (pixi v0.75.0), add 3-attempt recipe-generation retry loop, fix delete-outdated-cache-entries exit bug, move PYTHONIOENCODING to pixi.toml activation - Sync check_patches_clean_apply.py: narrow except to AttributeError, add git-cache retry on fetch failure, fix sys.exit always returning 2 - Normalize AGENTS.md: use $DISTRO placeholder instead of ros-jazzy- hardcoded prefix in all examples Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
pixi.lock was stale after removing the corrupted .pixi/envs/default (pyparsing namespace-package corruption) and running pixi install fresh. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Update build_gap_report.py (cross-distro sync) - Update libg2o, python-qt-binding, qt-gui-cpp patches - Refresh pixi.lock Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The "Save build cache" step gates on steps.build-recipes.outcome, but the "Build recipes" step had no id: build-recipes set, so that context reference was always empty and the always() && (...) condition was always false. The cache was never being saved regardless of outcome. Mirrors the same fix applied to ros-humble. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…zy macOS build Ports the OpenCV5, CMake4, and dependency-config unification work already landed on humble, and adds fresh fixes surfaced while getting jazzy's macOS build to complete cleanly: - OpenCV5 API migration across ~15 packages: legacy CV_* color/type macros -> cv::COLOR_*/cv::* equivalents, calib3d -> calib/geometry component split, removed C-API headers (types_c.h etc.), and the free cv::aruco::detectMarkers()/estimatePoseSingleMarkers() functions replaced with the ArucoDetector class (image-proc, image-rotate, cv-bridge, image-geometry, compressed-image-transport, rqt-image-view, theora-image-transport, grid-map-cv, apriltag-mit, moveit-ros-perception, rtabmap). - Legacy EIGEN3_INCLUDE_DIR (unset by modern Eigen3Config.cmake) replaced with the Eigen3::Eigen target across the autoware osqp/qp/kalman-filter chain, plus an fmt::fmt link fix for autoware-ekf-localizer. - Boost modernization: dropped the no-longer-resolvable "system" component from find_package(Boost COMPONENTS ...) and ported boost::asio::io_service -> io_context / boost::filesystem::complete|extension -> absolute()/ path::extension() (libpointmatcher, sick-safetyscanners-base). - libnabo: C++14 bump for modern Eigen, numpy>=2.0 header path, missing <cassert>, and OpenMP linked explicitly for clang (plus find_dependency in its exported Config.cmake so consumers pick it up too). - libg2o: switched host Qt dependency from qt-main (Qt5) to qt6-main to match the rest of the Qt6 stack and unblock rtabmap's PCL/VTK requirement. - vinca.yaml: closed the largest gaps in jazzy's package coverage versus humble's meta-package tree, bumped vtk to 9.7.0 to match humble. - Added check_dependency_compat.py (pre-build pin-conflict solver) and vinca_pinning.yaml, matching humble's tooling. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Resolved conflicts by hand: - pkg_additional_info.yaml/vinca.yaml: several hunks were additive on both sides at the same insertion point (new package entries); kept the union, de-duplicated, and re-sorted with `pixi run sort`. Kept our newer hpp_fcl/pinocchio/visp version bumps (with their libboost/conda-forge compat comments) over main's older overrides. - pixi.toml: kept the pinned vinca rev plus curl/go-yq/colordiff (used by check_dependency_compat.py); took main's setup-pixi bump. - pixi.lock: regenerated with `pixi lock` against the merged pixi.toml instead of hand-merging the generated lockfile. - testpr.yml: combined both branches' cache-cleanup steps. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…onstraints Already-built packages pin ros2-distro-mutex 0.16.* jazzy_*; bumping to 0.17.0 (matching the pattern already used on kilted) makes sure a stale mutex build isn't silently resolved once vtk 9.7.0 lands in the run_constraints, instead of the constraint mismatch surfacing later as a confusing solver failure. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Avoid multiple pushes racing multiple full CI matrices in parallel; same concurrency group already added to ros-humble's testpr.yml this session. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…o-mutex 904749d bumped ros2-distro-mutex 0.16.0 -> 0.17.0 for the new vtk 9.7.0 run_constraint, but ros2-ros2cli and ros2-rosidl-cli are already published on robostack-jazzy with build_number 22, hard-pinned (via run_exports) to ros2-distro-mutex 0.16.*. vinca's already-built check only compares each package's remote build_number against this per-package override (or the global default), so it kept treating both as current and never regenerated recipes for them -- even though their published build's mutex pin is now unsatisfiable, breaking the whole solve for anything that depends on them (e.g. ros-jazzy-rosidl-generator-type-description -> ros2-rosidl-cli). This isn't the local-cache class of staleness (a testpr.yml cache-bust can't touch an already-published remote artifact); bumping the per-package build_number is the mechanism vinca provides for forcing a specific package's republish without touching the global counter. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Same class of issue as 3dc6b32 (ros2cli/rosidl_cli), but a local-cache staleness rather than a published-remote one: ros2-ros-workspace's cached artifact is already at build_number 22 (matching current), so vinca's already-built check treats it as current, but that cached build predates 904749d's ros2-distro-mutex 0.16.0 -> 0.17.0 bump and is hard-pinned to the now-unsatisfiable old version. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… C:\bld\...
ament_python_install_package()/ament_python_install_module() embed a raw
Windows path (with backslashes, from get_executable_path() and
CMAKE_INSTALL_PREFIX) directly into an install(CODE "...") string. That
string gets written verbatim into cmake_install.cmake and re-parsed as CMake
source at install time, where CMake's own string-escape rules choke on
whatever backslash-letter sequence the path happens to contain -- in this
case "\b" from this workflow's C:/bld/win-64 build root ("Invalid character
escape '\b'", first hit by ros2-ament-cmake-test). Convert both paths via
file(TO_CMAKE_PATH ...) before embedding, the standard fix for this class of
CMake footgun.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Same pattern as ddbda94 (ros2-ros-workspace). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…tex bump Individually cache-busting each affected package (ddbda94, 71ff3ad, and now ros2-ament-cmake-core) as it surfaced was turning into unbounded whack-a-mole -- this is a systemic issue (any package cached/published at build_number 22 with the pre-904749dd mutex pin baked in), not isolated bugs in each one. The comment already left at this line anticipated exactly this ("next build number should be 23"). This is a full-rebuild-triggering change: every package in the distro will rebuild fresh once its cached build_number no longer matches, so this CI run will take considerably longer than the incremental --skip-existing runs tonight. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
crop_foremost.cpp used the legacy C-API CV_THRESH_TOZERO_INV macro, which OpenCV5 dropped in favor of the cv::THRESH_TOZERO_INV enum value. Same class of legacy CV_* macro removal already hit elsewhere this cycle (CV_GRAY2RGB in rqt_image_view). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
disparity_node.cpp included the old-style <opencv2/calib3d/calib3d.hpp> path; OpenCV5 only ships the flat <opencv2/calib3d.hpp> header. Same class of header-layout change already hit elsewhere in image_pipeline this cycle. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
mcap_vendor FetchContent's mcap v1.3.1 (foxglove/mcap); its types.hpp uses uint16_t/uint64_t/etc without including <cstdint>, which GCC 15's leaner libstdc++ headers no longer pull in transitively. It's a downloaded tarball, not a git checkout, so force-include the header via CMAKE_CXX_FLAGS instead of source-patching, same approach used for as2_platform_multirotor_simulator. Separately, ros2-control-msgs failed on win-64 with sensor_msgs's cmake config missing, despite ros2-sensor-msgs being present locally -- its cached .conda was only ~25KB (implausibly small for a message package), suggesting a prior interrupted/cancelled build (many superseding pushes via the concurrency group this cycle) got cached in a corrupt state. Cache-bust it to force a fresh rebuild. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Same depthai-core-release tag (2.31.1-1) as humble, hitting the identical two-layer CMake4 floor violation: its own top-level cmake_minimum_required(VERSION 3.4), and Hunter's subprocess-spawned cmake invocation. Port both fixes over: bump the version, set CMAKE_POLICY_VERSION_MINIMUM as a real env var before HunterGate (env vars propagate to the child process; a -D cache arg would not), and proactively add the same additional_cmake_args cache arg for cmake/CMakeRC.cmake's separate same-process include() later in the file (confirmed necessary for this exact source on humble). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Same depthai-ros-release tag (2.12.2-1) and identical CMakeLists.txt as humble's depthai_bridge (confirmed via diff) -- hits the same OpenCV major-version-pin, calib3d->calib+geometry component split, and hardcoded opencv_calib3d link library issues. Port the same patch over verbatim. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…al source) Same depthai-ros-release tag (2.12.2-1) and identical CMakeLists.txt as humble's depthai_examples (confirmed via diff). Port the same fix over. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Same pattern as ros2-sensor-msgs: ros2-rcl failed to find rcl_interfaces's cmake config despite it being present in the local channel, and its cached .conda was also only ~25KB. Force a fresh rebuild. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…om source Building from ROS source fails to find Simde (its vectorization backend) via find_package. Same fix already applied on humble: skip the source build and generate a dummy package depending on the conda-forge proxsuite release directly (also more up to date than the ROS-packaged version). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Third instance of the same ~25KB corrupt-cache pattern: ros2-hardware-interface failed to find control_msgs's cmake config despite it being present in the local channel. Force a fresh rebuild. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Same fixes as humble (identical upstream source, ros2-gbp/libg2o-release): qt isn't declared in package.xml and CMakeLists.txt's find_package(QGLViewer) is optional, so the qt host dep is dead weight that conflicts with vtk's now-Qt6-only build (vtk 9.7.0 pinned here too). Also add -DCMAKE_WINDOWS_EXPORT_ALL_SYMBOLS=ON for csparse_extension's win-64 LNK1181 (no import .lib generated without it). Root-caused and verified by a peer session fixing the identical issue on rolling. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The ros2-gz-gui-vendor/ros2-ros-gz eviction alone didn't fix irobot_create_gz_plugins: ros2-gz-common-vendor (which pulls in a bare, unconstrained ffmpeg host dep from its own package.xml) was still being skipped/reused from a build that predates this repo's conda_build_config.yaml bump to libopencv 5.0.0 / libprotobuf 7.35.1 / ffmpeg 9. A fully protobuf-7-compatible chain already exists on conda-forge today (libopencv 5.0.0 -> ffmpeg 9.0.x -> libopenvino 2026.3.1 -> libprotobuf 7.35.1, confirmed via direct channel queries) -- gz-common-vendor just needed to actually rebuild against the current variant config instead of reusing its stale artifact. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…traints The actual, distro-wide root cause of irobot_create_gz_plugins (and likely other libopencv/ffmpeg/gz-vendor consumers) failing to solve: libopencv/ffmpeg builds needing libprotobuf 6.x still coexist in the channel alongside protobuf-7-compatible ones, and resolvo doesn't reliably pick the compatible combination on its own even though one exists (confirmed directly: libopencv 5.0.0 -> ffmpeg 9.0.x -> libopenvino 2026.3.1 -> libprotobuf 7.35.1 is already published). Pinning libopenvino-tensorflow-frontend >=2026.3.1 (the actual carrier of protobuf-7 support in this chain) as a mutex run_constraint forces this centrally for every consumer, the same mechanism already used for libboost/pcl/gazebo/libprotobuf -- rather than pinning each affected downstream package's exact build individually as new ones are discovered (reverted approach from earlier today). Version bump 0.17.0 -> 0.18.0 forces a fresh mutex publish so this constraint actually takes effect, per the established precedent for run_constraints changes in this repo's history. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…, bump build_number for full rebuild Same corrections as ros-rolling: 1. The >=2026.3.1 floor was too strict: >=2026.3.0 already carries libprotobuf 7.35 support (verified directly), and on osx-arm64/win-64 today's only published protobuf-7-compatible libopencv build pins ffmpeg to a version that requires exactly 2026.3.0.x, which the >=2026.3.1 floor excluded entirely -- confirmed via direct dry-run solves per-platform. 2. Also pin libopencv/py-opencv >=5.0.0 directly: resolvo has repeatedly failed to propagate the openvino constraint transitively through ffmpeg -> libopencv on its own in the actual CI runs, even though a compatible combination exists. 3. Bumped build_number 23->24: mutex version alone doesn't force a rebuild of unrelated packages like ros2-ros-workspace that still reference the old ros2-distro-mutex 0.17.* and don't otherwise change. A full build_number bump forces every package to republish against the current mutex version, matching this repo's own historical precedent for run_constraints changes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
My first attempt at this fix (aa16de5, mutex version 0.18.0, run_constraints pinning only libopenvino-tensorflow-frontend >=2026.3.1) got far enough in its CI run (35490729546) to actually publish a real ros2-distro-mutex-0.18.0 build before I corrected the constraint in the very next commit (5132499) -- which reused the SAME version number "0.18.0" for the corrected content. Since packages match the mutex by "0.18.* jazzy_*" (not by build string), resolvo could pick EITHER the old wrong build or the new correct one, and the current full-rebuild run (35492632746) hit exactly this ambiguity: the conflict tree showed two different "would constrain" outputs for the same "ros2-distro-mutex 0.18.0" label, one with the old too-strict >=2026.3.1-only constraint and one with the corrected >=2026.3.0 + libopencv/py-opencv constraints. Bumping the version to a genuinely new number (0.19.0) that was never published under any prior content eliminates the ambiguity. Also bumping build_number again (24->25): packages already built in the cancelled run under build_number 24 still embed a dependency on the old "0.18.*" mutex and would otherwise be wrongly skip-cached as already-built once vinca regenerates them referencing 0.19.* instead. Lesson for next time: bump the mutex version on every actual change to run_constraints content, including quick follow-up corrections to a same-day fix -- never reuse a version number across two different sets of constraint content, even briefly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
ffmpeg 8.1.2 builds needing a protobuf-6-only libopenvino still coexist in the channel alongside the protobuf-7-compatible ffmpeg 9.x chain, and resolvo intermittently picks the incompatible one when solving irobot_create_gz_plugins's environment (which needs the gz-vendor chain and the image-transport/opencv/ffmpeg chain simultaneously). Pin ffmpeg directly in the mutex's run_constraints, same mechanism already used for libopenvino-tensorflow-frontend/libopencv/py-opencv. No mutex version/build-number bump: this workflow's build cache is PR-scoped and never touches the real published channel, so the mutex's identity can stay the same as long as the stale cached copy in this PR's cache is evicted, matching the technique from e6988a4. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The ffmpeg 9.* pin alone wasn't sufficient: libopencv 5.0.0 depends on a second, separate openvino frontend package (tensorflow-LITE-frontend, distinct from tensorflow-frontend already pinned above), and every 5.0.0 build hardcodes it to the 2026.3.0.x line. Some ffmpeg 9.0.1/ 9.0.2 builds instead need lite-frontend 2026.3.1, which conflicts and made irobot_create_gz_plugins unsolvable on osx-arm64/linux-aarch64. Constrain lite-frontend to the same range libopencv needs so resolvo is forced to pick a compatible ffmpeg 9.x build. Verified via direct dry-run solves on all 5 platforms. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
ros2-nav2-util's CMake configure failed with "Could not find a package configuration file provided by nav2_msgs" on win-64, even though skip-existing found a matching ros2-nav2-msgs package -- its cached .conda archive exists but is missing its installed CMake config files, most likely truncated by one of this PR's several win-64 job cancellations mid-build. Evict it so it rebuilds cleanly, matching the same corrupt-cache-entry pattern already used for moveit_ros_visualization and warehouse_ros in this file. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
All the packages these lines were forcing rebuilds for (roboplan, warehouse_ros, gz-gui-vendor/ros-gz, gz-common-vendor, distro-mutex, nav2_msgs) have since rebuilt successfully and are no longer corrupted or stale in the PR-scoped cache -- this run went fully green with all of them in place. Restore this step to the clean, no-active-eviction shape it has on main.
…pher dep removal Windows Defender exclusions/real-time-scanning-disable are already handled by GitHub's hosted runner image setup (Configure-WindowsDefender.ps1 in actions/runner-images), so this step was redundant. cartographer_ros can depend on both ros2-cartographer and cartographer 2.* without conflict; the remove_host/remove_run wasn't needed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
| # conda-forge's sip 6.16.x (6.16.1 uploaded 2026-09-08) regressed ABI | ||
| # targeting for PyQt5-based bindings: sip-build now fails with "ABI v12 | ||
| # is being targeted but the <module> module doesn't support it" for | ||
| # packages like qt_gui_cpp_sip, which build against pyqt5-sip's fixed | ||
| # ABI v12. Pin back to the last known-good line until upstream fixes it. | ||
| sip: | ||
| - 6.15 |
traversaro
left a comment
There was a problem hiding this comment.
I left some comments that it would be good to get fixed, but if that takes time I think it is probably more beneficial to just merge and handle cleanup later, perhaps in later rebuild, as the size of this PR is already quite big.
However, I am not sure if we want to wait for ROSCon to be passed before merging, so that it easier to deal with any potential leftover/post-PR problem. |
At least until after today! In about 30 minutes, they are doing a workshop for beginners that uses robostack jazzy haha |
I hope you have good lockfiles. : ) But yes, that was my idea, avoiding starting a major rebuild during ROSCon. |
…dback All the packages the cache-eviction lines were forcing rebuilds for (moveit_ros_visualization, roboplan, camera-calibration/image-pipeline, distro-mutex, gz-vendor family, mavros) have since rebuilt successfully and are no longer corrupted or stale in the PR-scoped cache -- this branch went fully green with all of them in place. Restore this step to the clean, no-active-eviction shape it has on main. Also address PR review feedback (see RoboStack/ros-jazzy#271): Windows Defender exclusions/real-time-scanning-disable are already handled by GitHub's hosted runner image setup, so that step was redundant. cartographer_ros can depend on both ros2-cartographer and cartographer 2.* without conflict; the remove_host/remove_run wasn't needed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…odules Port of RoboStack/ros-rolling#51 (without its pkg_additional_info.yaml build-number bumps): stop exporting the lib<pkg>__rosidl_generator_py library, which has unresolved CPython symbols, to C++ consumers of interface packages. The generated conversion code is compiled as an OBJECT library into each Python extension module, and conversion functions of nested types from other packages are looked up from those packages' Python message classes (_CONVERT_FROM_PY/_CONVERT_TO_PY). Adds the std_msgs C++-consumer regression test and a CI cache eviction for std_msgs so it is rebuilt and tested. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s rebuilt rosidl_generator_py's patch changed, but its build string didn't, so --skip-existing would reuse the cached, unpatched build (and with it rosidl_core_generators/rosidl_default_generators), and std_msgs would be rebuilt with the old generator. Evict them alongside std_msgs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Address review feedback (#271): conda-forge's cartographer was built with lua 5.4.8, so pin lua globally in pinning_overrides (as done in ros-rolling) rather than adding `lua 5.4.*` to ros2-cartographer's host dependencies. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Heads-up on the current CI failures after mirroring RoboStack/ros-rolling#51 (rosidl_generator_py no longer exports The PR-scoped build cache is now inconsistent: std_msgs and the rosidl generators were rebuilt with the new generator, but interface packages cached from earlier runs (e.g. This is expected and will resolve on |
Summary
Validation