Allow vendoring build inputs for offline builds - #95
Conversation
azenla
left a comment
azenla
left a comment
There was a problem hiding this comment.
Thank you, this is great, and very timely for us too!
We are in the process of making our standard dev process across Edera much faster (hence #93), one small gap I see here is that we don't verify the libscap source hash like we do bpftool. This is fine for this PR but this might be a good follow-up we can do internally. We can file an issue after it is merged.
There is just one minor thing to note that I saw, should be an easy change. Approving otherwise.
Allow hermetic build environments (Nix, Bazel, air-gapped CI) to supply pre-fetched inputs for the two fetches build.rs performs itself: the libscap source tree (VENDOR_LIBSCAP_SRC_DIR replaces the git clone) and the bpftool release archive (VENDOR_BPFTOOL_ARCHIVE replaces the download, verified against the same pinned sha256). Unset variables fall back to the existing network behavior. The downloads that happen inside cmake (ExternalProject tarballs for zlib/libbpf/uthash, FetchContent for libelf) need no hooks: cmake accepts pre-existing archives in its download dirs, so hermetic builders can seed the build folder directly. Vendored source trees need no .git: version strings can be supplied via VENDOR_LIBS_VERSION / VENDOR_DRIVER_VERSION, defaulting to GetVersionFromGit's own no-git fallback (0.0.0), so nothing new needs maintaining alongside the pinned checkout. The defines are always passed for .git-less trees so git can't walk up from OUT_DIR and describe an enclosing repo. A stamp file keyed on the pin SHA and source path drops the copied tree whenever either changes; unsetting the hook drops the tree and the cmake build dir (so cached version defines don't leak into clone-path builds).
6b75212 to
0311544
Compare
…-env-hooks # Conflicts: # build.rs
|
@jades-ant Looks like we have a conflict with main now due to some perf changes |
bleggett
left a comment
bleggett
left a comment
There was a problem hiding this comment.
Yeah I believe @antoineco hit this during his nixification and supporting vendored copies was a TODO anyway, thanks!
|
Glad I can remove those build-time patches from downstream. Thanks! |
|
MC fixed, and yep, removing build-time patches on my side is exactly why I figured I would just PR |
Awesome! Thank you! Merging! |
4 participants
With
full_bindingsenabled,build.rsfetches two things itself: a clone of falcosecurity/libs and a bpftool release archive. In hermetic build environments (Nix, Bazel, air-gapped CI) there's no network at build time, so these make the crate unbuildable.This adds optional env vars to point those at pre-fetched inputs instead:
VENDOR_LIBSCAP_SRC_DIR— an extracted checkout of the pinned commit, copied instead of cloned (no.gitneeded)VENDOR_BPFTOOL_ARCHIVE— a local bpftool archive, verified against the same pinned sha256 as the download pathVENDOR_LIBS_VERSION/VENDOR_DRIVER_VERSION— version strings for.git-less trees, wheregit describecan't run (default0.0.0, libscap's own no-git fallback)Everything is opt-in; unset means the current network behavior, unchanged. The downloads cmake does on its own (zlib/libbpf/uthash, libelf) don't need hooks — cmake already skips a download when the archive is pre-seeded in its build dir, so a hermetic builder can handle those without
build.rsgetting involved.Tested: full build with hooks set (no clone/download in logs, regenerated
src/types unchanged), full build with hooks unset (normal network path), and a wrong bpftool archive fails the checksum check.