Rebase onto Git for Windows v2.55.0-rc1#938
Merged
Merged
Conversation
Add robust-retry mechanism to automatically retry a request after network
errors. This includes retry after:
[] transient network problems reported by CURL.
[] http 429 throttling (with associated Retry-After)
[] http 503 server unavailable (with associated Retry-After)
Add voluntary throttling using Azure X-RateLimit-* hints to avoid being
soft-throttled (tarpitted) or hard-throttled (429) on later requests.
Add global (outside of a single request) azure-throttle data to track the
rate limit hints from the cache-server and main Git server independently.
Add exponential retry backoff. This is used for transient network problems
when we don't have a Retry-After hint.
Move the call to index-pack earlier in the response/error handling sequence
so that if we receive a 200 but yet the packfile is truncated/corrupted, we
can use the regular retry logic to get it again.
Refactor the way we create tempfiles for packfiles to use
<odb>/pack/tempPacks/ rather than working directly in the <odb>/pack/
directory.
Move the code to create a new tempfile to the start of a single request
attempt (initial and retry attempts), rather than at the overall start
of a request. This gives us a fresh tempfile for each network request
attempt. This simplifies the retry mechanism and isolates us from the file
ownership issues hidden within the tempfile class. And avoids the need to
truncate previous incomplete results. This was necessary because index-pack
was pulled into the retry loop.
Minor: Add support for logging X-VSS-E2EID to telemetry on network errors.
Minor: rename variable:
params.b_no_cache_server --> params.b_permit_cache_server_if_defined.
This variable is used to indicate whether we should try to use the
cache-server when it is defined. Got rid of double-negative logic.
Minor: rename variable:
params.label --> params.tr2_label
Clarify that this variable is only used with trace2 logging.
Minor: Move the code to automatically map cache-server 400 responses
to normal 401 response earlier in the response/error handling sequence
to simplify later retry logic.
Minor: Decorate trace2 messages with "(cs)" or "(main)" to identify the
server in log messages. Add params->server_type to simplify this.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Add trace2_thread_start() and trace2_thread_exit() events to the worker threads used to read the index. This gives per-thread perf data. These workers were introduced in: abb4bb8 read-cache: load cache extensions on a worker thread 77ff112 read-cache: load cache entries on worker threads Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Expose the differences in the semantics of GET and POST for
the "gvfs/objects" API:
HTTP GET: fetches a single loose object over the network.
When a commit object is requested, it just returns
the single object.
HTTP POST: fetches a batch of objects over the network.
When the oid-set contains a commit object, all
referenced trees are also included in the response.
gvfs-helper is updated to take "get" and "post" command line options.
the gvfs-helper "server" mode is updated to take "objects.get" and
"objects.post" verbs.
For convenience, the "get" option and the "objects.get" verb
do allow more than one object to be requested. gvfs-helper will
automatically issue a series of (single object) HTTP GET requests
and creating a series of loose objects.
The "post" option and the "objects.post" verb will perform bulk
object fetching using the batch-size chunking. Individual HTTP
POST requests containing more than one object will be created
as a packfile. A HTTP POST for a single object will create a
loose object.
This commit also contains some refactoring to eliminate the
assumption that POST is always associated with packfiles.
In gvfs-helper-client.c, gh_client__get_immediate() now uses the
"objects.get" verb and ignores any currently queued objects.
In gvfs-helper-client.c, the OIDSET built by gh_client__queue_oid()
is only processed when gh_client__drain_queue() is called. The queue
is processed using the "object.post" verb.
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
…ension Add regions around code to read and write the cache-tree extension when the index is read or written. This is an experiment and may be dropped in future releases if we don't need it anymore. This experiment demonstrates that it takes more time to parse and deserialize the cache-tree extension than it does to read the cache-entries. Commits [1] and [2] spreads cache-entry reading across N-1 cores and dedicates a single core to simultaneously read the index extensions. Local testing (on my machine) shows that reading the cache-tree extension takes ~0.28 seconds. The 11 cache-entry threads take ~0.08 seconds. The main thread is blocked for 0.15 to 0.20 seconds waiting for the extension thread to finish. Let's use this commit to gather some telemetry and confirm this. My point is that improvements, such as index V5 which makes the cache entries smaller, may improve performance, but the gains may be limited because of this extension. And that we may need to look inside the cache-tree extension to truly improve do_read_index() performance. [1] abb4bb8 read-cache: load cache extensions on a worker thread [2] 77ff112 read-cache: load cache entries on worker threads Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
During development, it was very helpful to see the gvfs-helper do its work to request a pack-file or download a loose object. When these messages appear during normal use, it leads to a very noisy terminal output. Remove all progress indicators when downloading loose objects. We know that these can be numbered in the thousands in certain kinds of history calls, and would litter the terminal output with noise. This happens during 'git fetch' or 'git pull' as well when the tip commits are checked for the new refs. Remove the "Requesting packfile with %ld objects" message, as this operation is very fast. We quickly follow up with the more valuable "Receiving packfile %ld%ld with %ld objects". When a large "git checkout" causes many pack-file downloads, it is good to know that Git is asking for data from the server. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
If our POST request includes a commit ID, then the the remote will send a pack-file containing the commit and all trees reachable from its root tree. With the current implementation, this causes a failure since we call install_loose() when asking for one object. Modify the condition to check for install_pack() when the response type changes. Also, create a tempfile for the pack-file download or else we will have problems! Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Create t/helper/test-gvfs-protocol.c and t/t5799-gvfs-helper.sh to test gvfs-helper. Create t/helper/test-gvfs-protocol.c as a stand-alone web server that speaks the GVFS Protocol [1] and serves loose objects and packfiles to clients. It is borrows heavily from the code in daemon.c. It includes a "mayhem" mode to cause various network and HTTP errors to test the retry/recovery ability of gvfs-helper. Create t/t5799-gvfs-helper.sh to test gvfs-helper. [1] https://github.com/microsoft/VFSForGit/blob/master/Protocol.md Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Teach gvfs-helper to support "/gvfs/prefetch" REST API. This includes a new `gvfs-helper prefetch --since=<t>` command line option. And a new `objects.prefetch` verb in `gvfs-helper server` mode. If `since` argument is omitted, `gvfs-helper` will search the local shared-cache for the most recent prefetch packfile and start from there. The <t> is usually a seconds-since-epoch, but may also be a "friendly" date -- such as "midnight", "yesterday" and etc. using the existing date selection mechanism. Add `gh_client__prefetch()` API to allow `git.exe` to easily call prefetch (and using the same long-running process as immediate and queued object fetches). Expanded t5799 unit tests to include prefetch tests. Test setup now also builds some commits-and-trees packfiles for testing purposes with well-known timestamps. Expanded t/helper/test-gvfs-protocol.exe to support "/gvfs/prefetch" REST API. Massive refactor of existing packfile handling in gvfs-helper.c to reuse more code between "/gvfs/objects POST" and "/gvfs/prefetch". With this we now properly name packfiles with the checksum SHA1 rather than a date string. Refactor also addresses some of the confusing tempfile setup and install_<result> code processing (introduced to handle the ambiguity of how POST works with commit objects). Update 2023-05-22 (v2.41.0): add '--no-rev-index' to 'index-pack' to avoid writing the extra (unused) file. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
gvfs-helper prints a "loose <oid>" or "packfile <name>" messages after they are received to help invokers update their in-memory caches. Move the code to accumulate these messages in the result_list into the install_* functions rather than waiting until the end. POST requests containing 1 object may return a loose object or a packfile depending on whether the object is a commit or non-commit. Delaying the message generation just complicated the caller. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
…and report_tracking() Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Earlier versions of the test always returned a packfile in response to a POST. Now we look at the number of objects in the POST request. If > 1, always send a packfile. If = 1 and it is a commit, send a packfile. Otherwise, send a loose object. This is to better model the behavior of the GVFS server/protocol which treats commits differently. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Add data for the number of files created/overwritten and deleted during the checkout. Give proper category name to all events in unpack-trees.c and eliminate "exp". This is modified slightly from the original version due to interactions with 26f924d (unpack-trees: exit check_updates() early if updates are not wanted, 2020-01-07). Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
It is possible that a loose object that is written from a GVFS protocol "get object" request does not match the expected hash. Error out in this case. 2021-10-30: The prototype for read_loose_object() changed in 31deb28 (fsck: don't hard die on invalid object types, 2021-10-01) and 96e41f5 (fsck: report invalid object type-path combinations, 2021-10-01). Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
The gvfs-helper allows us to download prefetch packs using a simple subprocess call. The gvfs-helper-client.h method will automatically compute the timestamp if passing 0, and passing NULL for the number of downloaded packs is valid. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Update tracing around report_tracking() to use 'tracking' category rather than 'exp' category. Add ahead/behind results from stat_tracking_info(). Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Teach helper/test-gvfs-protocol to be able to send corrupted loose blobs. Add unit test for gvfs-helper to detect receipt of a corrupted loose blob. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Add a new JavaScript GitHub Action to download secrets from Azure Key Vault using the `az` CLI, mask the secret values, and store them as: * outputs, * environment variables, or * files; Values are all masked for safe consumption by other steps in a workflow. Callers of this action can optionally perform base64 decoding of secret values using the syntax: `INPUT base64> OUTPUT`. It is assumed that the `az login` command has already been run prior to this action being invoked. Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
Add a manual-only GitHub Actions workflow for building the Windows installer (x86_64 plus portable Git), driven via `workflow_dispatch:`. The production release path for the official microsoft/git installers lives in .azure-pipelines/release.yml; this workflow is kept around as a fallback so the Windows installer can still be produced on demand for debugging or comparison. The build steps are pinned to `windows-2019` (rather than `windows-latest`) to ensure the correct Visual Studio version is used (verified in the pipeline via `type -p mspdb140.dll`), and the SDK used is the `full` flavor rather than `build-installers` due to a known (but not-yet-fixed) issue downloading the `build-installers` flavor with the `git-for-windows/setup-git-for-windows-sdk` Action. There is no code-signing certificate available to this workflow, so the artifacts it produces are unsigned and must not be published as releases; they are useful only for build-time debugging. Signed-off-by: Victoria Dye <vdye@github.com> Assisted-by: Claude Opus 4.7 Co-authored-by: Matthew John Cheetham <mjcheetham@outlook.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When building Git as a universal binary on macOS, the binary supports more than one target architecture. This is a bit of a problem for the `HOST_CPU` setting that is woefully unprepared for such a situation, as it wants to show architecture hard-coded at build time. In preparation for releasing universal builds, work around this by special-casing `universal` and replacing it at run-time with the known values `x86_64` or `arm64`. Signed-off-by: Jeff Hostetler <jeffhostetler@github.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
- include `scalar` - build signed .dmg & .pkg for target OS version 10.6 - upload artifacts to workflow Co-authored-by: Lessley Dennington <ldennington@github.com>
Teach gvfs-helper to better support the concurrent fetching of the same packfile by multiple instances. If 2 instances of gvfs-helper did a POST and requested the same set of OIDs, they might receive the exact same packfile (same checksum SHA). Both processes would then race to install their copy of the .pack and .idx files into the ODB/pack directory. This is not a problem on Unix (because of filesystem semantics). On Windows, this can cause an EBUSY/EPERM problem for the loser while the winner is holding a handle to the target files. (The existing packfile code already handled simple the existence and/or replacement case.) The solution presented here is to silently let the loser claim victory IIF the .pack and .idx are already present in the ODB. (We can't check this in advance because we don't know the packfile SHA checksum until after we receive it and run index-pack.) We avoid using a per-packfile lockfile (or a single lockfile for the `vfs-` prefix) to avoid the usual issues with stale lockfiles. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
- include `scalar` - build & upload unsigned .deb package Co-authored-by: Lessley Dennington <ldennington@github.com> Co-authored-by: Sverre Johansen <sverre.johansen@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Add a release-pipeline scaffold for microsoft/git on Azure Pipelines, structured around a prereqs stage, a per-platform build stage with placeholder jobs, and a release stage that downloads the build artifacts and publishes them to a draft GitHub release. The per-platform build, signing, and validation logic lands in subsequent commits. The pipeline targets Microsoft-internal 1ES-hosted images across Windows x64, Windows ARM64, macOS, Ubuntu x64, and Ubuntu ARM64. Windows and Linux matrix entries carry a `poolArch` dimension because the 1ES hosted pools select their image from `hostArchitecture`; an arm64 entry on a default x64 pool would silently grab the wrong image. macOS uses the same matrix-parameter shape as Windows and Linux, so future macOS variants drop in the same way an extra Windows toolchain would. The prereqs stage derives the Git version, tag name, and tag SHA via resolve-version.sh and exposes them as pipeline variables for downstream stages to pick up. For that walk to find tags at all, the prereqs checkout uses fetchDepth: 0 / fetchTags: true. setup-git-bash.cmd is the Windows-side prerequisite that prepends Git Bash to PATH, since the bare hosted image does not provide a bash for Bash@3 tasks to find. ESRP signing to be added later. Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com> Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
## TL;DR Add a new `vfs-functional-tests.yml` workflow that builds Git from this repository and runs the VFS for Git functional tests against it, using VFSForGit's reusable workflow. ## Why? VFS for Git functional tests currently only run in the VFSForGit repository, against a tagged microsoft/git release. This means VFS-related regressions in Git are only caught *after* a release is tagged. By running the FTs here on every push and PR to `vfs-*` branches, we can catch regressions before they ship. This is the counterpart to microsoft/VFSForGit#1932, which extracted the functional tests into a reusable `workflow_call` workflow. ## How it works 1. **Build Git** — checks out this repo, builds with the Git for Windows SDK, and packages the result into a `MicrosoftGit` artifact with an `install.bat` that deploys via robocopy to `C:\Program Files\Git`. Both ARM64 and x64 are built and combined into a single artifact for the FTs to install and use. 2. **Find VFSForGit build** — locates the latest successful VFSForGit CI run on `master` to get the GVFS installer and FT executables. If the build was a 'skipped' build (because an existing run succeeded with that tree) then follow the annotation to the real run. 3. **Call reusable workflow** — invokes `microsoft/VFSForGit/.github/workflows/functional-tests.yaml@master`, which handles the full test matrix (2 configs × 2 architectures × 10 slices)
Add diff.renameThreshold, merge.renameThreshold, and status.renameThreshold configuration options to control the minimum similarity threshold for rename detection without requiring command-line flags. The cascade follows the existing pattern for renameLimit and renames: - merge.renameThreshold overrides diff.renameThreshold for merges - status.renameThreshold overrides diff.renameThreshold for status - CLI flags (-M, --find-renames) override all config values The value accepts the same format as -M: a percentage (e.g. 50%) or a fraction (e.g. 0.5). If unset, the default remains 50%. Assisted-by: Claude Opus 4.6 Signed-off-by: Tyrie Vella <tyrielv@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Add a stub pipeline for releases using Azure Pipelines. The pipeline runs on Microsoft internal images/runners across: * Windows x64 * Windows ARM64 * macOS * Ubuntu x64 * Ubuntu ARM64 At the start of a run there is a prerequisite stage and pre-build validation. Today this does nothing, and should be updated to: * validate the current commit is tagged (annotated), and * capture the Git version, tag name and SHA. Artifacts are uploaded from the build stage, and downloaded into the release stage later for uploading to a draft GitHub release. ESRP signing to be added later.
Bring ESRP code signing into the release pipeline, gated behind an `esrp` boolean parameter that defaults to false until the rest of the signing wiring catches up. Microsoft policy precludes shipping unsigned binaries from this pipeline, so every per-platform build job needs an obvious place to plug signing in. The Windows flow runs through a custom esrpsign.sh script rather than the EsrpCodeSigning ADO task. The custom script gives the later Windows installer commit a CLI-shaped seam it can register as Git for Windows' `git signtool` alias from build-extra's release pipeline, so every binary embedded inside the installer gets signed rather than only the outer .exe wrapper. The canned ADO task does not expose that integration point. The accompanying setup template uses AzureCLI@2 to bind to the WIF service connection by name rather than by GUID, and relies on addSpnToEnvironment to surface the service principal ID, tenant ID, and connection GUID at runtime via ENDPOINT_URL_* env vars. That way esrpsign.sh composes the auth JSON with no hardcoded identifiers leaking into the repository. EsrpClientTool@4 takes care of downloading and caching the ESRP client binary itself. macOS and Linux take the simpler path: the EsrpCodeSigning@6 task via a shared sign.yml template. macOS in particular requires an archive submission (useArchive: true), so centralising the copy/zip/sign/extract cycle in the template keeps each platform job from re-implementing it. The Linux hosted agents do not ship with .NET, which EsrpCodeSigning requires. UseDotNet@2 installs the .NET 8 SDK ahead of the signing template invocation so Linux signing works out of the box without per-platform plumbing. Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Smoke tests and dry runs of the pipeline itself need to build microsoft/git from an untagged tip, where resolve-version.sh's tag walk would either fail or pick up an unrelated tag. Add a queue-time `versionOverride` parameter; when it is set to anything other than the empty string or the sentinel `-`, the prereqs stage emits the override verbatim as the build version, labels the tag name as `untagged`, and bypasses resolve-version.sh entirely. A build from an untagged commit must not race a real release upload, so a non-empty override also forces the GitHub publishing job off regardless of the `github` parameter, and the prereqs step logs a warning to make that consequence visible in the run summary. Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Add the per-platform Linux job that takes microsoft/git from
source to a signed `microsoft-git_<version>_<arch>.deb` staged
under `$(Build.ArtifactStagingDirectory)/_final/`. The flow
ports `create-linux-unsigned-artifacts` and
`create-linux-signed-artifacts` from the GitHub workflow at
.github/workflows/build-git-installers.yml, keeping the Make
recipe and DEBIAN/control body byte-for-byte identical so a
diff against the workflow's output is empty modulo the
deliberate departures called out below.
The GitHub workflow runs everything inside an ubuntu:20.04 /
ubuntu:22.04 container, both to pin the resulting .deb's glibc
ABI floor and to give apt-get a root-owned filesystem. The 1ES
pool images we run on
(GitClientPME-1ESHostedPool-{intel,arm64}-pc) silently ignore a
job-level `container:` directive, so the build executes on the
bare Ubuntu host VM as the unprivileged agent user. apt-get
therefore runs via `sudo`, and the job logs the Ubuntu version,
kernel, and effective UID up front so an audit can read the
.deb's effective glibc floor back from the build output.
Re-introducing a real container later (whether via 1ES's
container option, a custom container job, or docker invoked
from a step) is a separate question. The workflow's
`DEBIAN_FRONTEND=noninteractive` and `TZ=Etc/UTC` env vars
exist only to keep `tzdata` quiet inside a fresh container; the
bare 1ES image already has tzdata configured, so they are
dropped. The Node.js workaround in the workflow similarly
exists only to satisfy GitHub Actions' Node-based shim and is
not needed under Azure Pipelines.
A few intentional content changes: parallelism switches from
the workflow's hard-coded `-j5` (a runner-specific holdover) to
`-j$(nproc)`, which adapts to whatever the 1ES pool gives us;
the shell prologue changes from `set -ex` to `set -euo
pipefail` so an unbound variable or a failing stage in a pipe
aborts the job rather than silently producing a broken .deb;
`$(git_version)` now comes from the prereqs stage, dropping the
workflow's runtime dpkg-architecture round-trip in favour of
the matrix's explicit `amd64` / `arm64` entries via
`$(deb_arch)`. The `s/-rc/.rc/g` substitution carries over
because Git's GIT-VERSION-GEN spells release-candidate tags
with a dot.
The build drops its output under
`$(Build.ArtifactStagingDirectory)/app/` so the existing ESRP
signing template's `**/*.deb` pattern picks it up. A focused
move of just the signed
`microsoft-git_<version>_<arch>.deb` into
`$(Build.ArtifactStagingDirectory)/_final/` then feeds the
existing `templateContext.outputs.pipelineArtifact` for the
`linux_x64` / `linux_arm64` artifact. Naming the file
precisely turns "ESRP signed something else" into a
missing-file error rather than a silent wrong-artifact upload.
Assisted-by: Claude Opus 4.7
Co-authored-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Add the per-platform macOS job that takes microsoft/git from source to a signed-and-notarized `git-<version>-universal.pkg` plus the corresponding `.dmg`, both staged under `$(Build.ArtifactStagingDirectory)/_final/`. The flow ports `create-macos-artifacts` and `create-macos-signed-artifacts` from the GitHub workflow at .github/workflows/build-git-installers.yml and leans on .github/macos-installer/Makefile for the heavy lifting, but swaps the workflow's productsign + xcrun notarytool path for ESRP signing and ESRP MacAppNotarize. The native Homebrew on the macOS-15-arm64 pool image is arm64 and lives at /opt/homebrew. Producing a universal binary additionally requires the x86_64 build of gettext/libintl, so a separate x86_64 Homebrew gets installed under /usr/local via the upstream installer running under Rosetta and pulls gettext from there as well. The two arch-specific libintl.a copies are then combined with lipo into a universal archive at the workspace root, which the upcoming config.mak's `LDFLAGS = -L"$(pwd)"` resolves. libintl depends on iconv, but the system /usr/lib/libiconv.dylib is already universal and exports the `_iconv*` symbols Homebrew's gettext was built against; Homebrew's own libiconv exports `_libiconv*` and would not link, hence the explicit `USE_HOMEBREW_LIBICONV` / `ICONVDIR` overrides in config.mak. Spotlight indexing on the boot volume is disabled (`mdutil -i off /`) at the start of the job because leaving it on caused intermittent file-locking failures in subsequent steps. config.mak collects the Make flags that turn on the dual-arch compile and route around several macOS quirks: HOST_CPU=universal, dual-arch CFLAGS (the actual universal-binary driver), -DNO_OPENSSL for contrib Makefiles that do not see the main Makefile's NO_OPENSSL handling, the USE_HOMEBREW_LIBICONV / ICONVDIR overrides, gettext include dirs from both Homebrew prefixes, and CURL_LDFLAGS / CURL_CONFIG pinned against the OS-supplied libcurl rather than a Homebrew copy. SKIP_DASHED_BUILT_INS disables the dashed built-ins because on macOS the hard-link optimisation does not kick in for the staging tree and the resulting full copies would bloat the eventual .dmg. `make GIT-VERSION-FILE dist dist-doc` runs in the source tree; `git get-tar-commit-id` recovers the original commit OID from the resulting source tarball (this becomes GIT_BUILT_FROM_COMMIT, which the macos-installer Makefile bakes into `git version --build-options`); the source and manpage tarballs extract into payload/ and manpages/; a copy of config.mak is dropped inside the extracted source so the universal-build flags apply during the real compile; finally `make -C .github/macos-installer payload` produces the universal binary tree. `git get-tar-commit-id` reads only the leading pax header and then closes its stdin, which makes `gunzip -c` exit 141 (SIGPIPE) under the outer `set -o pipefail`; the pipeline is wrapped in a `set +o pipefail` subshell so the SIGPIPE does not abort the build. The macos-installer Makefile produces the install tree at stage/git-universal-<ver>/ but its `pkg` target packages from build-artifacts/, so the tree is copied across after `make payload` completes, mirroring the GitHub workflow. GITHUB_WORKSPACE=$(Build.SourcesDirectory) is exported because the Makefile derives BUILD_DIR from $(GITHUB_WORKSPACE), which is unset under Azure Pipelines. XML_CATALOG_FILES points at the catalogs from the Homebrew docbook installed in the dependencies step. (FUTURE: the duplication exists only because .github/macos-installer/Makefile hardcodes both DESTDIR=stage/... and ARTIFACTDIR=build-artifacts; overriding ARTIFACTDIR on the `make pkg` line below to point at stage/ would let us drop the cp entirely. Worth cleaning up alongside moving the macOS installer Makefiles out of .github/, where they live for historical reasons rather than because they are GitHub-specific.) Signing happens against the install tree at .github/macos-installer/build-artifacts/usr/local/git/, not the source tree under payload/git-<version>/, because the macos-installer Makefile's `pkg` target packages from build-artifacts/; signing the source tree would have no effect on the resulting .pkg. Following the pattern in git-credential-manager/.azure-pipelines/release.yml, the install tree is pre-filtered to just the Mach-O files (using `file --mime` matching `mach`, the same heuristic .github/scripts/codesign.sh uses), copied into a staging directory under $(Build.ArtifactStagingDirectory)/macos-tosign/ preserving relative paths, handed to the existing .azure-pipelines/esrp/sign.yml template (which zips, signs via EsrpCodeSigning@6 with KeyCode CP-401337-Apple + OperationCode MacAppDeveloperSign + Hardening enabled, and extracts back into the staging dir), and finally copied back into the install tree. The pre-filter is necessary because the existing template's CopyFiles@2 step uses minimatch globs and the only reliable way to pick out Mach-O files is by file content; signing the entire install tree would either fail on non-binary files or sign things that should not be signed (shell scripts, perl, manpages, templates, the uninstall.sh). UseDotNet@2 (8.x) installs the .NET SDK that EsrpCodeSigning@6 depends on, since the macOS-15-arm64 pool image does not provide it. The macos-installer Makefile's `pkg` target then produces an unsigned .github/macos-installer/disk-image/git-<version>-universal.pkg from the signed payload tree. APPLE_INSTALLER_IDENTITY is deliberately left undefined so pkgbuild does not try to sign; the next step submits the .pkg back through ESRP for signing (KeyCode CP-401337-Apple covers both Developer ID Application and Developer ID Installer certs in this account, so MacAppDeveloperSign on a .pkg is the productsign equivalent), and then through ESRP for Apple notarization (MacAppNotarize, BundleId com.git.pkg, matching the identifier pkgbuild bakes in via `--identifier com.git.pkg` from the Makefile). The ESRP MacAppNotarize operation handles both submission and ticket stapling, returning the notarized .pkg back into disk-image/ via the same zip-extract template path the previous sign step used; this is what replaces the `xcrun notarytool submit ... --wait` plus `xcrun stapler staple` flow from .github/scripts/notarize.sh. Finally the Makefile's `image` target builds .github/macos-installer/git-<version>-universal.dmg from the contents of disk-image/. The .dmg lands at the macos-installer root, while the signed-and-notarized .pkg lives somewhere under disk-image/: ESRP's MacAppNotarize op repacks its output zip to wrap the notarized .pkg in a UUID-named .zip.unzipped/ subdirectory, so depending on whether notarization ran, the .pkg ends up either directly under disk-image/ or at disk-image/<uuid>.zip.unzipped/git-...pkg. `find` locates it and moves it (along with the globbed .dmg) into $(Build.ArtifactStagingDirectory)/_final/, which the job's templateContext.outputs already publishes as the `macos_universal` pipeline artifact. `set -euo pipefail` means an empty `find` result, or a missing .dmg, fails the mv loudly rather than producing a silent half-empty upload, matching the same defensive choice the Linux stage step makes. Assisted-by: Claude Opus 4.7 Co-authored-by: Matthew John Cheetham <mjcheetham@outlook.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Add the per-platform Windows job that takes microsoft/git from source to ESRP-signed `Git-<version>-<arch>.exe` and `PortableGit-<version>-<arch>.exe`, plus the matching sha-256.txt sidecar, all staged under $(Build.ArtifactStagingDirectory)/_final/. The flow ports `create-windows-artifacts` and `create-windows-signed-artifacts` from the GitHub workflow at .github/workflows/build-git-installers.yml, leaning on git-for-windows/build-extra (please.sh + installer/release.sh) for the build itself. GitHub Actions has the git-for-windows/setup-git-for-windows-sdk@v1 action that drops a full SDK onto the runner; Azure Pipelines has no equivalent task, so the Windows job has to bootstrap the SDK by hand before any of the bash-driven build steps can run. Bootstrap is driven by .azure-pipelines/scripts/windows/setup-git-sdk.sh, which a Bash@3 task invokes via `filePath:` so it runs under the agent's MinGit-provided bash. A `sdk_repo` field on each windows_matrix entry (git-for-windows/git-sdk-64 for x64, git-sdk-arm64 for ARM64) lets the script pick the right upstream; the script does a partial+bare clone of the SDK, clones git-for-windows/build-extra into a sibling directory, then runs `please.sh create-sdk-artifact --sdk=<bare> --out=<sdk> build-installers` to materialise the build-installers flavour of the SDK at the requested output path. Routing through `please.sh create-sdk-artifact` keeps the bytes flowing via plain GitHub HTTPS clones (which 1ES allows) rather than the raw and release-asset CDNs that an earlier download-the-snapshot approach hit. Once the SDK is in place, its own usr/bin (which ships cygpath) and the matching MinGW toolchain bin/ are exposed to subsequent tasks via ##vso[task.prependpath]. The arm64 Windows hosted agents do not have Azure CLI pre-installed, which the AzureCLI@2 task in the ESRP setup step needs. Install the x64 MSI (which runs under x86-64 emulation on arm64 Windows) and prepend it to PATH, gated on a poolArch condition. This is a workaround until the bug preventing us from baking Azure CLI into the hosted pool image is fixed, at which point this step can be dropped. A small helper, .azure-pipelines/scripts/windows/utils.sh, provides `to_windows_path` / `to_unix_path` for scripts running on Windows agents. Both prefer `cygpath` when it is on PATH and fall back to a small pure-shell parser otherwise. The fallback matters because some bash steps run before the SDK is bootstrapped and only have MinGit's bash available, which does not ship cygpath. Both setup-git-sdk.sh and the ESRP-sign script source utils.sh rather than duplicating the conversion logic. Git for Windows' build tooling (please.sh, signtool.sh, the installer .iss templates, and the MINGW-packages helpers) lives in git-for-windows/build-extra rather than in the SDK snapshot. The GitHub workflow's Windows job clones it into /usr/src/build-extra of the SDK before invoking please.sh; this job does the same. A partial clone (--filter=blob:none) plus --single-branch -b main is enough for everything please.sh needs and avoids pulling the full blob history, matching the workflow's invocation byte for byte. The mingw-w64-git package is built via please.sh build-mingw-w64-git from a Bash@3 task using the SDK's bash that the bootstrap put on PATH. Outputs land in $(Build.SourcesDirectory)/artifacts/ so the subsequent installer-build step can pass them to please.sh make_installers_from_mingw_w64_git via --pkg= flags. Three small adaptations from the GitHub workflow's source step are worth flagging. First, the /usr/bin/git trampoline that delegates to the matching MinGW-built git.exe is the same one the workflow writes by hand; makepkg-mingw shells out to plain `git`, and the SDK bash's git candidates would otherwise come from MinGit, not the toolchain we are building against. Second, the user.name / user.email / PACKAGER values are hardcoded to a build-bot identity since Azure Pipelines has no GitHub-actor equivalent. Third, please.sh's --only-<arch> flag takes the bare CPU name (x86_64 or aarch64), not the toolchain triple, so a `cpu_arch` matrix dimension surfaces the right value next to each toolchain entry. The build task detaches stdin via `exec </dev/null` before invoking makepkg-mingw, because pacman's git-extra post_install hook runs `for s in $(grep -l PAT $(find /mingw*/bin/ ...))` and falls back to reading stdin when /mingw*/bin/ is absent and find produces empty output. Bash@3 leaves the task's stdin pipe open with no writer (the GitHub Actions runner closes it for the same reason; see actions/runner ProcessInvoker.cs), so without the detach the build would hang indefinitely waiting for input. Two pieces of the workflow's pkg-build step are intentionally not ported and are noted in a comment for follow-up: the per-tarball GPG signing (replaceable with an ESRP PGP operation analogous to the Linux LinuxSign flow if needed downstream), and the MINGW-packages bundle creation, which depends on a PKGBUILD.<tag_name> snapshot that microsoft/git does not currently ship. A single bash task drives please.sh make_installers_from_mingw_w64_git for both installer and portable variants. The GitHub workflow runs these as separate matrix jobs (one per type/arch combination); keeping both builds in the same Azure Pipelines job means the .pkg.tar.* artifacts produced by the previous step are available without an inter-job artifact passing trip. The PDB archive copy into build-extra/cached-source-packages is the same prerequisite that --include-pdbs needs in the GitHub workflow. The --pkg= filter that strips signatures and the optional archimport / cvs / p4 / gitweb / doc-man pieces matches the workflow's sed exactly so the resulting .exe sizes are comparable. The same `exec </dev/null` stdin detach as the build mingw-w64-git step applies. To make a silent hang inside the nested pacman / makepkg / Inno Setup chain debuggable, BASH_ENV is pointed at a tiny `set -x` fragment so every nested non-interactive bash subshell auto-enables xtrace, and please.sh itself is invoked via `sh -x`. The trace volume is the cost we pay for being able to identify which step a future hang is stuck in; cheap compared to debugging a silent hang. The GitHub Actions workflow applies five `sed` transformations during the Windows build to turn upstream Git for Windows into the microsoft/git distribution. They are spread over five run: blocks and largely opaque without following each sed pattern by hand. Capture them as patches under .azure-pipelines/patches/ instead, grouped by the upstream tree they mutate (build-extra/* for installer customisation, git-sdk/* for the SDK's git-update-git-for-windows helper); the patches themselves carry the explanatory commit-style headers each transformation deserves. A small helper, .azure-pipelines/scripts/apply-patches.sh, applies every *.patch in a directory in lexicographic order via `patch -p1`. patch(1) is used rather than `git apply` because the latter is strict about context whitespace; CRLF/LF mismatches between the patch context (as authored) and the working tree (which may be CRLF on Windows checkouts) trip it up. patch is more forgiving by default, and matches the convention used by msys2/MINGW-packages PKGBUILDs and git-for-windows/build-extra's get-sources.sh. The patches apply against /usr/src/build-extra and against the SDK's /$(mingwprefix), between cloning build-extra and building the mingw-w64-git package, mirroring where the GitHub workflow's sed steps slot in. ESRP signing is wired through build-extra's `signtool` alias hook so that ESRP signs every binary that ends up inside the Windows installer and portable Git, not just the outer .exe wrapper. A naive post-build sign of just `Git-*.exe` and `PortableGit-*.exe` would leave every binary embedded inside the installer (DLLs, helper exes, the mingw-w64-git pkg payload) shipping unsigned. The mechanism, set up by build-extra: please.sh's build_mingw_w64_git checks `git config alias.signtool` and, if set, exports `SIGNTOOL="git ... signtool"` into makepkg-mingw so the PKGBUILD can sign individual binaries during the package build; build-extra's installer/release.sh checks the same alias and passes `//Ssigntool="git signtool $f" //DSIGNTOOL` to Inno Setup's ISCC.exe, which then signs every embedded file via the SignTool=signtool directive in install.iss. The portable Git .exe is a 7z self-extractor that bypasses the Inno Setup signtool path, so it is signed explicitly after `make_installers` returns. The ESRP setup template (which sets ESRP_TOOL and ESRP_AUTH) runs before the build steps; a Bash@3 task registers the signtool alias to invoke .azure-pipelines/esrp/windows/esrpsign.sh; the ESRP env vars are added to both build tasks so esrpsign.sh has ESRP_TOOL, ESRP_AUTH, and SYSTEM_ACCESSTOKEN available when invoked via the alias from inside please.sh, makepkg-mingw, or Inno Setup. MSYSTEM is exported in the build installer task's env block because installer/release.sh requires it to select the architecture branch, and Bash@3 does not source /etc/profile. esrpsign.sh's calling convention (`<file> [file ...]`, sign in place) already matches signtool.sh's, so no further script changes are needed. Stage `Git-*.exe` and `PortableGit-*.exe` alongside a SHA-256 sidecar into $(Build.ArtifactStagingDirectory)/_final/, which the job's templateContext.outputs.pipelineArtifact already publishes as the `windows_x64` / `windows_arm64` artifact. The SHA-256 sidecar is computed here, post-build, rather than in the build step because ESRP signing rewrites the .exe contents; a SHA-256 computed before signing would mismatch the bytes that ship. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
With build, signing, notarization, validation, and draft-release publishing all in place, the Azure Pipeline is ready to take over from the GitHub Actions build-git-installers workflow. Switch `trigger: none` to a tag-only trigger matching the same `v[0-9]*vfs*` pattern the GitHub workflow used (and that resolve-version.sh validates against), and explicitly exclude all branches so the pipeline does not fire on every topic-branch push. Flip the `esrp` and `github` parameter defaults from false to true. The GitHub release job still uses `isDraft: true`, so a maintainer inspects and publishes the release manually; manual runs in the Azure DevOps UI can still uncheck either box for a dry run. Assisted-by: Claude Opus 4.7 Co-authored-by: Matthew John Cheetham <mjcheetham@outlook.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Refactor install_prefetch() to process prefetch packs in two distinct phases: Phase 1 (extraction): Read the multipack stream sequentially, copying each packfile to its own temp file and recording its checksum and timestamp in a prefetch_entry array. This must be sequential because the multipack is a single byte stream. Phase 2 (indexing): Run 'git index-pack' on each extracted temp file and finalize it into the ODB. Today this still runs sequentially, but the separation makes it straightforward to parallelize in a subsequent commit. The new extract_packfile_from_multipack() only does I/O against the multipack fd plus temp-file creation. The new index_and_finalize_packfile() only does the index-pack and rename work. Neither depends on the other's state, so they can operate on different entries concurrently once the extraction phase completes. No behavioral change; this is a pure refactor. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Add blame-specific configuration options for rename detection: - blame.renames: disable rename following (defaults to true) - blame.renameThreshold: minimum similarity for rename detection - blame.renameLimit: limit on rename detection candidates These are independent of the diff.renames/renameThreshold/renameLimit settings, which blame does not read because its config callback chains to git_default_config rather than git_diff_basic_config. The threshold and limit options forward to git_diff_basic_config to set the same globals that repo_diff_setup() copies into diff_options, following the existing pattern used for diff.algorithm. Also move diff.renameThreshold from git_diff_ui_config to git_diff_basic_config, alongside the related diff.renameLimit, since it controls rename detection behavior rather than UI presentation. Assisted-by: Claude Opus 4.6 Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
The 1ES PT Windows build job did not run binskim against the binaries
we ship. By default the template would point binskim at the published
artifact (the Inno Setup installer and the 7z self-extracting portable
.exe), neither of which binskim can crack open to find the PE files
inside, so any findings on those wrappers are also unactionable: they
are produced by external tools we do not control.
Opt the job into binskim explicitly and aim it at the actual product
binaries instead. Stage only the first-party pacman packages that
`please.sh build-mingw-w64-git` emits --
mingw-w64-<toolchain>-{git,git-credential-wincred,git-pdb}-*.pkg.tar.xz
-- into _bin/<mingwprefix>/ and scope the analyzer to the .exe/.dll
files in that tree. By construction those packages carry only the
binaries this repo's Makefile builds (git.exe, the dashed subcommands,
scalar.exe, headless-git.exe, git-gvfs-helper.exe,
git-credential-wincred.exe, ...) plus their cv2pdb-generated .pdbs,
so a broad **/*.{exe,dll} glob is safe.
Excluding everything else keeps the full Git for Windows installer
payload out of the scan: MSYS2/MinGW runtime, Perl, Tcl/Tk,
libcurl/libssl/libssh2, Git Credential Manager, Git LFS, tig, and the
build-extra git-wrapper launcher shims are all third-party content we
cannot fix from this repo.
Since the baseline that is added automatically does not conform to Git's
whitespace rules, also add a `.config/.gitattributes` file to suppress
those checks.
Assisted-by: Claude Opus 4.7
Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Replace the sequential index-pack loop in install_prefetch() with run_processes_parallel(), spawning up to four concurrent 'git index-pack' workers. The packfiles are already ordered by timestamp (oldest first) in the multipack response. In the common fresh-clone scenario the oldest pack is by far the largest, so it starts indexing immediately on the first worker while the remaining workers cycle through the smaller daily and hourly packs. Note that this works for the GVFS prefetch endpoint as all prefetch packfiles are non-thin packs. The bundle URI feature uses thin bundles that must be unpacked sequentially. Worker count is min(np, PREFETCH_MAX_WORKERS) where PREFETCH_MAX_WORKERS is 4, so we never create more workers than there are packfiles. When there is only a single packfile the parallel infrastructure is skipped entirely and index-pack runs directly. The default grouped mode of run_processes_parallel() is used so that child-process completion is detected via poll() on stderr pipes rather than the ungroup mode's aggressive mark-all-slots-WAIT_CLEANUP approach, which can misfire on slots that never started a process. The run_processes_parallel() callbacks are always invoked from the main thread, so finalize_prefetch_packfile() (which renames files into the ODB) needs no locking. If any index-pack fails, the error is recorded and remaining tasks still complete so that successfully-indexed packs are not lost. I performed manual performance testing on Linux using an internal monorepo. I deleted a set of recent prefetch packfiles, leading to a download of a couple daily packfiles and several hourly packfiles. This led to an improvement from 85.2 seconds to 40.3 seconds. Signed-off-by: Derrick Stolee <stolee@gmail.com>
blame's config callback did not load diff.renameThreshold because it chained directly to git_default_config, skipping git_diff_basic_config (and git_diff_ui_config) where that setting is handled. As a result, repo_diff_setup() always saw a zero rename_score, and blame fell back to DEFAULT_RENAME_SCORE (50%) regardless of the configured threshold. This PR moves diff.renameThreshold from git_diff_ui_config to git_diff_basic_config, alongside the related diff.renameLimit setting, so that plumbing commands that use git_diff_basic_config can also pick it up - but instead of trying to import that for blame, this PR adds separate configuration for blame.renames, blame.renameThreshold, and blame.renameLimit Assisted-by: Claude Opus 4.6
Add diff.renameThreshold, merge.renameThreshold, and
status.renameThreshold configuration options to control the minimum
similarity threshold for rename detection without requiring command-line
flags.
The cascade follows the existing pattern for renameLimit and renames:
- merge.renameThreshold overrides diff.renameThreshold for merges
- status.renameThreshold overrides diff.renameThreshold for status
- CLI flags (-M, --find-renames) override all config values
The value accepts the same format as -M: a percentage (e.g. 50%) or a
fraction (e.g. 0.5). If unset, the default remains 50%.
This also gives git-blame users control over the rename threshold for
the first time, since blame has no -M threshold flag but inherits
diff.renameThreshold via repo_diff_setup().
Assisted-by: Claude Opus 4.6
__________________
Thanks for taking the time to contribute to Git!
This fork contains changes specific to monorepo scenarios. If you are an
external contributor, then please detail your reason for submitting to
this fork:
* [x] This is an early version of work already under review upstream.
* [ ] This change only applies to interactions with Azure DevOps and the
GVFS Protocol.
* [ ] This change only applies to the virtualization hook and VFS for
Git.
BinSkim flags toolchain-rooted issues on every release build that we cannot fix from this repo: BA2008 (Control Flow Guard) and BA2012 (stack cookie not locatable) on every clangarm64 binary, and BA2025 (CET shadow stack) on every mingw64 binary. CFG and CET shadow stack are gated on linker support that lld's MinGW driver does not expose, and BinSkim's stack-cookie check uses an MSVC PE walker that does not find clang's emitted cookie. None are actionable from microsoft/git. Point the SDL templateContext at a per-arch suppression file at .azure-pipelines/sdl/$dim.id/gdnsuppress so Guardian skips these known-bad findings on each scan. Per-arch paths keep the entries isolated to the matching toolchain and let either arch grow new entries without touching the other. Seed windows_arm64/gdnsuppress with the 44 hydrated entries Guardian auto-published in the drop_build_windows_arm64_sdl_analysis artifact on the previous release run; the signatures are derived from (tool, ruleId, target URI) and remain stable across rebuilds, so the same file applies to future runs. windows_x64/gdnsuppress ships as a stub with no suppression entries. BA2025 is the only BinSkim finding on x64 and it is Warning-severity, so it does not break the build, and Guardian's pipeline-export only hydrates findings at or above Error severity, so no canonical entries were auto-generated to seed from. The stub keeps the per-arch path uniform without requiring a YAML conditional, and gives us a place to drop x64 entries later if we ever want to silence the warning. Assisted-by: Claude Opus 4.7 Signed-off-by: Matthew John Cheetham <mjcheetham@outlook.com>
Introduce a new config option, gvfs.prefetchThreads, that controls the number of parallel index-pack processes used when installing prefetch packfiles. The default value is 1, which processes packfiles sequentially without any parallel infrastructure, preserving the existing behavior as a safety measure. When set to a value greater than 1, the prefetch installation phase uses run_processes_parallel() with up to that many concurrent index-pack workers. This can significantly speed up installation when multiple prefetch packs are downloaded. The sequential path (threads=1) loops through each extracted packfile one at a time using my_run_index_pack() + finalize, while the parallel path (threads>1) dispatches index-pack processes via the existing parallel process machinery. Add trace2 data points (prefetch/install_mode) to log which code path is taken, enabling tests to verify the correct mode is active. Add t5797-gvfs-helper-prefetch-threads.sh with tests that exercise both sequential and parallel modes by looping over threads=1 and threads=4. Each iteration runs four scenarios: fetch-all, fetch-since, up-to-date re-fetch, and corrupt-pack error handling. Trace2 assertions confirm the expected code path is taken in each mode. Signed-off-by: Derrick Stolee <stolee@gmail.com>
This ports the changes from #894 from the `vfs-2.53.0` release train to the `vfs-2.54.0` release train.
Originally added to vfs-2.53.0. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When using Scalar clones with microsoft/git against Azure DevOps and GVFS Cache Servers, `git fetch` will download potentially multiple precomputed prefetch packfiles. The current mechanism indexes these files sequentially. Let's make those `git index-pack` processes run somewhat in parallel. For now, I've chosen to have a maximum of four parallel processes to limit the potential load on the disk. However, this already has some significant gains. When testing an internal monorepo (that uses Codespaces, for easy Linux testing) and deleting a few days of recent prefetch packfiles, the end-to-end `git fetch` time improved as follows: | Command | Mean [s] | Min [s] | Max [s] | Relative | |:---|---:|---:|---:|---:| | `new` | 40.306 ± 1.598 | 37.564 | 42.383 | 1.00 | | `old` | 85.213 ± 2.389 | 82.402 | 89.207 | 2.11 ± 0.10 | When downloading fewer prefetch packfiles, the improvement is still relevant: | Command | Mean [s] | Min [s] | Max [s] | Relative | |:---|---:|---:|---:|---:| | `new` | 6.411 ± 0.800 | 5.559 | 7.553 | 1.00 | | `old` | 13.906 ± 2.848 | 10.941 | 17.697 | 2.17 ± 0.52 | I should mention that I _first_ tried streaming data directly from the curl download into a sequence of `git index-pack` processes, but that did not make any serious difference in the performance. Based on these numbers, we are clearly blocked on the CPU time spent computing deltas and evaluating object hashes and not blocked on the "download to disk, then index from disk" I/O. I think it would be worthwhile to do some performance testing on Windows, at minimum, before merging this change. I'd like to get some feedback on the concept before going through those actions. Another question to ask is whether it is worth making this behavior configurable: should it be possible to disable parallel indexing in favor of a sequential process if a certain config option is set? Should we allow increasing the parallelism via config?
When 'git checkout <tree> -- <pathspec>' updates the index with entries from the source tree, update_some() creates a new cache entry that unconditionally clears skip-worktree. In a virtual filesystem repo (core.virtualfilesystem is set), this causes checkout_entry() to attempt unlink() on files that exist only as virtual projections with no physical NTFS entry. The unlink fails with ENOENT, producing 'error: unable to unlink old' messages and exit code 255. Fix this in three places: 1. update_some(): Propagate CE_SKIP_WORKTREE from the existing index entry to the replacement entry when core_virtualfilesystem is set. 2. mark_ce_for_checkout_overlay(): Allow skip-worktree entries that have CE_UPDATE (set by update_some for tree entries) to still match the pathspec, so report_path_error() does not reject them. 3. checkout_worktree(): Skip checkout_entry() for entries that have both CE_MATCHED and CE_SKIP_WORKTREE, since the virtual filesystem provider will serve the correct content from the updated projection. The index is updated to the new tree entry's OID while the working tree write is skipped entirely. The virtual filesystem provider re-reads the updated index and serves the correct content on next access. Same approach as the CE_NEW_SKIP_WORKTREE propagation in deleted_entry() (unpack-trees.c, PR #865) which avoids unnecessary lstats on virtualized paths during branch switches. Assisted-by: Claude Opus 4.6 Signed-off-by: Tyrie Vella <tyrielv@gmail.com> (cherry picked from commit d40f13b) Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
…ns (#920) Port #913 and #914 from the `vfs-2.53.0` to the `vfs-2.54.0` branch. While at it, reorganize the changes into a more concise commit history, including two `amend!` commits that target the next rebase to a new Git version. At time of writing, `git diff azp vfs-2.53.0 -- .azure-pipelines` shows an empty diff, proving that it brings `vfs-2.54.0` up to date with the latest changes that made v2.53.0.vfs.0.11 possible.
…d-port of #915) (#921) Forward-port of #915 ("checkout: preserve skip-worktree for virtual filesystem paths", merged into `vfs-2.53.0` as ea3eb21) to `vfs-2.54.0`. Cherry-picked from the underlying PR-branch commit d40f13b (so the original well-crafted commit message and Tyrie Vella's authorship are preserved). One auto-merge in `builtin/checkout.c` resolved cleanly without conflicts.
When Git is built under MSYS2/MinGW with Rust support enabled, the
Makefile expects `cargo build` to drop a `target/release/libgitcore.a`
that is linkable by the same MinGW GCC used for every other object.
With Rust installed via `rustup` (the way it ships on the
GitHub-hosted `windows-2022` and `windows-11-arm` runners that build
microsoft/git), the default toolchain targets the MSVC ABI; cargo
then writes `target/release/gitcore.lib` instead, which the MinGW
`ld.exe` cannot consume:
LINK git-shell.exe
D:\git-sdk-64-minimal\mingw64\bin/ld.exe: cannot find target/release/libgitcore.a: No such file or directory
collect2.exe: error: ld returned 1 exit status
See https://github.com/microsoft/git/actions/runs/27341625000 for the
full log.
Let's define the correct target. Re-use (and fix) the existing
`HOST_CPU` variable for that purpose. Avoid relying on environment
variables that are simply not defined in Git for Windows' minimal SDK
that Git uses in its CI runs.
Assisted-by: Claude Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The Windows runners used by `vfs-functional-tests.yml` ship `rustup` plus a `*-pc-windows-msvc` default toolchain (see https://github.com/actions/runner-images/blob/main/images/windows/Windows2022-Readme.md and https://github.com/actions/partner-runner-images/blob/main/images/arm-windows-11-image.md), but no precompiled `std` for `*-pc-windows-gnu` or `*-pc-windows-gnullvm`. With the Makefile now picking a GCC-compatible target triple based on `$(MSYSTEM)`, the build step needs that precompiled `std` to be installed before invoking `make`, otherwise `cargo build --target <triple>` fails to find a usable `std` for the chosen target. Add a step between the SDK setup and the `make` invocation that selects the matching triple from `$MSYSTEM` (which `git-for-windows/setup-git-for-windows-sdk` exports for every subsequent step) and runs `rustup target add` for it. The mapping mirrors what `config.mak.uname` derives from `$(MSYSTEM)` and `$(HOST_CPU)`, just enumerated explicitly here since CI has direct knowledge of which MSYS2 subsystems the matrix actually exercises (`CLANGARM64` for the ARM64 runner, `MINGW64` for the x86_64 runner). For a `staticlib` crate-type `cargo build` does not invoke an external linker, so no further toolchain components (e.g. the `gnullvm` LLVM linker) need to be installed; `rustup target add` alone is sufficient. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Range-diff relative to clean-vfs-2.55.0-rc0
1: e976362 = 1: d3b6cb1 survey: calculate more stats on refs
2: cfcc439 = 2: 75ab330 survey: show some commits/trees/blobs histograms
3: 3ad2074 = 3: bea2d25 survey: add vector of largest objects for various scaling dimensions
4: 6ba2177 = 4: dbd57c6 survey: add pathname of blob or tree to large_item_vec
5: ca28c42 = 5: f0f02e7 survey: add commit-oid to large_item detail
6: bd06347 = 6: 1033588 survey: add commit name-rev lookup to each large_item
7: 7e6152a = 7: 5c907a4 survey: add --no-name-rev option
8: e2461c2 = 8: d1c348b sparse-index.c: fix use of index hashes in expand_index
11: e55b8e0 = 9: 4631caa t: remove advice from some tests
9: 4f0737c = 10: e466bbc survey: started TODO list at bottom of source file
10: 5252692 = 11: 3265dd5 t5300: confirm failure of git index-pack when non-idx suffix requested
12: 6ca2226 = 12: 64338fe t1092: add test for untracked files and directories
13: b55b3e8 = 13: ee71c65 survey: expanded TODO list at the bottom of the source file
14: f2debf2 = 14: b636409 index-pack: disable rev-index if index file has non .idx suffix
16: 39d9294 = 15: c56500d survey: expanded TODO with more notes
15: 101caff = 16: 9547ef6 trace2: prefetch value of GIT_TRACE2_DST_DEBUG at startup
17: d1e7207 = 17: 89e19f1 reset --stdin: trim carriage return from the paths
18: 4b60ba5 ! 18: af6b04d Identify microsoft/git via a distinct version suffix
19: 16135da = 19: a3fddee gvfs: ensure that the version is based on a GVFS tag
20: ba57143 = 20: 31126c2 gvfs: add a GVFS-specific header file
21: 48abb0e = 21: c7f9773 gvfs: add the core.gvfs config setting
22: f5c5222 = 22: 7ed41e5 gvfs: add the feature to skip writing the index' SHA-1
23: f21096b = 23: d2b5615 gvfs: add the feature that blobs may be missing
24: 6e5b6c4 = 24: 2d60918 gvfs: prevent files to be deleted outside the sparse checkout
25: 1527ed1 = 25: f6ea9db gvfs: optionally skip reachability checks/upload pack during fetch
26: f32c48f = 26: b7d73b8 gvfs: ensure all filters and EOL conversions are blocked
27: 4a43cc3 ! 27: 3e4d56a gvfs: allow "virtualizing" objects
28: 776bc05 = 28: a5406cc Hydrate missing loose objects in check_and_freshen()
29: 32db9e1 = 29: cae47dc sha1_file: when writing objects, skip the read_object_hook
63: 6e8d3e5 = 30: 836c34e gvfs: add global command pre and post hook procs
64: 8aa6022 = 31: b6c6b67 t0400: verify that the hook is called correctly from a subdirectory
65: e99cb40 = 32: eab729a t0400: verify core.hooksPath is respected by pre-command
66: 52b83f5 = 33: ad9f285 Pass PID of git process to hooks.
67: d7218ee = 34: 549aabf sparse-checkout: make sure to update files with a modify/delete conflict
68: 11fda87 = 35: 52ff414 worktree: allow in Scalar repositories
69: d0e093f = 36: 81dbee3 sparse-checkout: avoid writing entries with the skip-worktree bit
70: 1a541f7 = 37: cdfb5e4 Do not remove files outside the sparse-checkout
71: 2b6edee = 38: b4775ef send-pack: do not check for sha1 file when GVFS_MISSING_OK set
72: c6789d5 = 39: aafc5ad gvfs: allow corrupt objects to be re-downloaded
73: 61e3826 = 40: 6d12842 cache-tree: remove use of strbuf_addf in update_one
74: 940ff6f = 41: 1cf60f5 gvfs: block unsupported commands when running in a GVFS repo
75: cf9c470 = 42: 4cd35dd gvfs: allow overriding core.gvfs
76: 560487f = 43: 8d0b7da BRANCHES.md: Add explanation of branches and using forks
83: d9669c7 ! 44: ef289aa Add virtual file system settings and hook proc
84: 096ccc9 = 45: 47a7515 virtualfilesystem: don't run the virtual file system hook if the index has been redirected
85: 56d9a42 = 46: 3e53e43 virtualfilesystem: check if directory is included
86: 9758674 = 47: 79747e8 backwards-compatibility: support the post-indexchanged hook
87: 3abc257 = 48: 8b84a2d gvfs: verify that the built-in FSMonitor is disabled
88: e325e1e = 49: 083a675 wt-status: add trace2 data for sparse-checkout percentage
89: adf856a = 50: 5c42664 status: add status serialization mechanism
90: 14ec7e5 = 51: ab3e408 Teach ahead-behind and serialized status to play nicely together
91: c7dd029 = 52: 6bbb9be status: serialize to path
92: e1c4bfd = 53: 59f5835 status: reject deserialize in V2 and conflicts
93: 74ba487 = 54: b694a9c serialize-status: serialize global and repo-local exclude file metadata
94: ee9d327 = 55: 05d6165 status: deserialization wait
95: 57a86bb = 56: 19a7114 status: deserialize with -uno does not print correct hint
96: 6e2de31 = 57: b7c3b2a fsmonitor: check CE_FSMONITOR_VALID in ce_uptodate
97: fb0cc92 = 58: fd0d169 fsmonitor: add script for debugging and update script for tests
98: 7c46ace = 59: 686a17b status: disable deserialize when verbose output requested.
99: 7f39acd = 60: 94298b0 t7524: add test for verbose status deserialzation
77: 38b6a89 = 61: 33a0be5 git.c: add VFS enabled cmd blocking
100: 39b0e7a = 62: bb8326a deserialize-status: silently fallback if we cannot read cache file
78: b7f30f3 = 63: 8e8d7e4 git.c: permit repack cmd in Scalar repos
101: e7a4352 = 64: f2c5451 gvfs:trace2:data: add trace2 tracing around read_object_process
79: 5a1190f = 65: d95b54f git.c: permit fsck cmd in Scalar repos
102: 4692ae6 = 66: 0f49af2 gvfs:trace2:data: status deserialization information
80: b38c665 = 67: 272ef6e git.c: permit prune cmd in Scalar repos
103: d7ba74b = 68: 5bb6c6a gvfs:trace2:data: status serialization
81: ed4694f = 69: d865e7c worktree: remove special case GVFS cmd blocking
104: beb8138 = 70: 0cdc1b7 gvfs:trace2:data: add vfs stats
82: 51ba0fa = 71: b5a3dbe builtin/repack.c: emit warning when shared cache is present
105: a264996 = 72: 33b2106 trace2: refactor setting process starting time
107: 1ea414a = 73: 7953483 trace2:gvfs:experiment: report_tracking
108: 855c7ff = 74: b4a64db trace2:gvfs:experiment: read_cache: annotate thread usage in read-cache
109: 9671ad2 = 75: 6255d06 trace2:gvfs:experiment: read-cache: time read/write of cache-tree extension
110: b44cd45 = 76: 0bc1d4e trace2:gvfs:experiment: add region to apply_virtualfilesystem()
111: 0c8eaa9 ! 77: dc39334 trace2:gvfs:experiment: add region around unpack_trees()
@@ unpack-trees.c: int unpack_trees(unsigned len, struct tree_desc *t, struct unpac + trace2_region_enter("exp", "unpack_trees", NULL); + trace_performance_enter(); - trace2_region_enter("unpack_trees", "unpack_trees", the_repository); + trace2_region_enter("unpack_trees", "unpack_trees", repo); @@ unpack-trees.c: int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options } - trace2_region_leave("unpack_trees", "unpack_trees", the_repository); + trace2_region_leave("unpack_trees", "unpack_trees", repo); trace_performance_leave("unpack_trees"); + trace2_region_leave("exp", "unpack_trees", NULL); return ret;112: f0e91bd = 78: bd87c7f trace2:gvfs:experiment: add region to cache_tree_fully_valid()
113: 991d88d ! 79: 339699c trace2:gvfs:experiment: add unpack_entry() counter to unpack_trees() and report_tracking()
114: 593e650 = 80: 182493b trace2:gvfs:experiment: increase default event depth for unpack-tree data
115: 425f6b5 ! 81: ea97dfb trace2:gvfs:experiment: add data for check_updates() in unpack_trees()
@@ unpack-trees.c: static int check_updates(struct unpack_trees_options *o, trace_performance_leave("check_updates"); return errs != 0; } -@@ unpack-trees.c: static int clear_ce_flags(struct index_state *istate, - _("Updating index flags"), - istate->cache_nr); - -- xsnprintf(label, sizeof(label), "clear_ce_flags(0x%08lx,0x%08lx)", -+ xsnprintf(label, sizeof(label), "clear_ce_flags/0x%08lx_0x%08lx", - (unsigned long)select_mask, (unsigned long)clear_mask); - trace2_region_enter("unpack_trees", label, the_repository); -- - rval = clear_ce_flags_1(istate, - istate->cache, - istate->cache_nr, @@ unpack-trees.c: int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options if (o->df_conflict_entry) BUG("o->df_conflict_entry is an output only field");116: 8f5b1d3 = 82: 123dbef Trace2:gvfs:experiment: capture more 'tracking' details
117: a7f2078 = 83: ecdd71f credential: set trace2_child_class for credential manager children
118: 5bc52a8 = 84: a8abc19 sub-process: do not borrow cmd pointer from caller
119: e67b0d3 = 85: 966e020 sub-process: add subprocess_start_argv()
120: 3134fa9 = 86: 55ee368 sha1-file: add function to update existing loose object cache
121: e399514 = 87: f6fe1a8 index-pack: avoid immediate object fetch while parsing packfile
122: e529b5c ! 88: e2a1fae gvfs-helper: create tool to fetch objects using the GVFS Protocol
123: 5cb479f ! 89: fe09e68 sha1-file: create shared-cache directory if it doesn't exist
124: 2432445 = 90: ebf047f gvfs-helper: better handling of network errors
125: 6663075 = 91: 84fab23 gvfs-helper-client: properly update loose cache with fetched OID
126: b9e5fc4 = 92: ba785e0 gvfs-helper: V2 robust retry and throttling
127: 744eed2 = 93: 378799d gvfs-helper: expose gvfs/objects GET and POST semantics
128: 0beb08c = 94: 1b9986e gvfs-helper: dramatically reduce progress noise
129: 94c35c1 = 95: 83a5676 gvfs-helper: handle pack-file after single POST request
130: b67c1c1 ! 96: 28e9b87 test-gvfs-prococol, t5799: tests for gvfs-helper
131: ef84c03 = 97: bd4c979 gvfs-helper: move result-list construction into install functions
132: 85df5f0 ! 98: aa02b47 t5799: add support for POST to return either a loose object or packfile
@@ t/helper/test-gvfs-protocol.c: static enum worker_result do__gvfs_config__get(st { #define MAX_HEADER_LEN 32 @@ t/helper/test-gvfs-protocol.c: static enum worker_result send_loose_object(const struct object_info *oi, - struct git_hash_ctx c; int object_header_len; int ret; + struct repo_config_values *cfg; + unsigned flags = 0; + void *content; + size_t size; @@ t/helper/test-gvfs-protocol.c: static enum worker_result do__gvfs_objects__get(s - oi.sizep = &size; - oi.contentp = &content; - -- if (oid_object_info_extended(the_repository, &oid, &oi, flags)) { +- if (odb_read_object_info_extended(the_repository->objects, &oid, &oi, flags)) { - logerror("Could not find OID: '%s'", oid_to_hex(&oid)); - return send_http_error(1, 404, "Not Found", -1, WR_OK); - }133: b07b58e = 99: 76181f3 t5799: cleanup wc-l and grep-c lines
134: 08090a7 = 100: c60075a gvfs-helper: verify loose objects after write
135: ad16c02 = 101: 7391d25 t7599: create corrupt blob test
136: 20fb5ea = 102: 1a5a2c1 gvfs-helper: add prefetch support
137: b114655 = 103: b85cab8 gvfs-helper: add prefetch .keep file for last packfile
138: de6e02e = 104: 844d0ff gvfs-helper: do one read in my_copy_fd_len_tail()
139: 351e6c3 = 105: 4578b1c gvfs-helper: move content-type warning for prefetch packs
140: 9ca2ab5 = 106: ca11909 fetch: use gvfs-helper prefetch under config
141: fb04943 = 107: c6ab58e gvfs-helper: better support for concurrent packfile fetches
142: 37b840f = 108: 6db5036 remote-curl: do not call fetch-pack when using gvfs-helper
143: 1342b3b = 109: 71f3b4b fetch: reprepare packs before checking connectivity
144: a9944de = 110: f61edee gvfs-helper: retry when creating temp files
145: 011f8f6 = 111: 8c0e13c sparse: avoid warnings about known cURL issues in gvfs-helper.c
146: a5df365 = 112: 3291b28 maintenance: care about gvfs.sharedCache config
147: b44c2b3 ! 113: 9286910 unpack-trees:virtualfilesystem: Improve efficiency of clear_ce_flags
148: 1cb3fa5 = 114: 06ad061 homebrew: add GitHub workflow to release Cask
149: 832d9e9 = 115: d726e2d Adding winget workflows
150: 80cab97 = 116: 29fa7f6 Disable the
monitor-componentsworkflow in msft-git151: 7f4fbf3 = 117: b415171 .github: enable windows builds on microsoft fork
152: e5289f6 = 118: b93a081 .github/actions/akv-secret: add action to get secrets
153: 37cb816 = 119: 9770d18 release: create initial Windows installer build workflow
154: 3b44fb4 = 120: cd9be54 help: special-case HOST_CPU
universal155: f0898d4 = 121: f482141 release: add Mac OSX installer build
156: de939db = 122: c5f64af release: build unsigned Ubuntu .deb package
157: 723f264 = 123: 5ba16b6 gvfs-helper: add --max-retries to prefetch verb
158: 092bafb = 124: eed1d81 release: add signing step for .deb package
160: 6b49b9e = 125: 7753a55 t5799: add tests to detect corrupt pack/idx files in prefetch
161: 7eaa728 = 126: 2a376be release: create draft GitHub release with packages & installers
163: 555cd7e = 127: 2411a0c gvfs-helper: ignore .idx files in prefetch multi-part responses
164: 7ab02e0 = 128: 21f99af build-git-installers: publish gpg public key
166: 556af31 = 129: 33c712a t5799: explicitly test gvfs-helper --fallback and --no-fallback
167: 0086d67 = 130: c639034 release: continue pestering until user upgrades
169: 21e07c3 = 131: 5b41025 gvfs-helper: don't fallback with new config
170: 980cd27 = 132: b3949bb dist: archive HEAD instead of HEAD^{tree}
172: 1575adb = 133: 94843e2 test-gvfs-protocol: add cache_http_503 to mayhem
173: 120c419 = 134: 877c259 release: include GIT_BUILT_FROM_COMMIT in MacOS build
175: e4ae7d6 = 135: 73c6f11 t5799: add unit tests for new
gvfs.fallbackconfig setting176: ae3b80b = 136: b684c2f release: add installer validation
30: 3bc1b1a = 137: 8ed8789 git_config_set_multivar_in_file_gently(): add a lock timeout
159: c4794d3 = 138: 5c85e0c update-microsoft-git: create barebones builtin
31: b9adeee = 139: e2d29f5 scalar: set the config write-lock timeout to 150ms
162: 4c7e3eb = 140: 7dc851d update-microsoft-git: Windows implementation
32: f9bc0cf = 141: ae44b01 scalar: add docs from microsoft/scalar
165: 6a10d8c = 142: c1f0e12 update-microsoft-git: use brew on macOS
168: c8f9c9e = 143: 8ec3abe .github: reinstate ISSUE_TEMPLATE.md for microsoft/git
33: 6c80e7d = 144: 4f9ed68 scalar (Windows): use forward slashes as directory separators
171: 31e18c3 = 145: 1ffbd35 .github: update PULL_REQUEST_TEMPLATE.md
34: fac07dc = 146: 0b99ac3 scalar: add retry logic to run_git()
174: 8a19bd6 = 147: bf85114 Adjust README.md for microsoft/git
35: 6143dc1 = 148: e2ebca0 scalar: support the
configcommand for backwards compatibility177: f506f90 = 149: 6999ee3 scalar: implement a minimal JSON parser
178: aa03abb = 150: 85525a5 scalar clone: support GVFS-enabled remote repositories
179: 94c22d4 = 151: dd60f05 test-gvfs-protocol: also serve smart protocol
180: 7c6073d = 152: b0f59cf gvfs-helper: add the
endpointcommand181: 76ea0bb = 153: 1e55bae dir_inside_of(): handle directory separators correctly
182: ef1f493 = 154: e59b071 scalar: disable authentication in unattended mode
183: edc78a0 = 155: 74919f0 abspath: make strip_last_path_component() global
184: dd51790 = 156: 4cc6677 scalar: do initialize
gvfs.sharedCache185: 5da09a1 = 157: 9fd3785 scalar diagnose: include shared cache info
186: 3b86dba = 158: c08ec18 scalar: only try GVFS protocol on https:// URLs
187: e28cb3f = 159: 80c9b42 scalar: verify that we can use a GVFS-enabled repository
188: 85ce8ba = 160: 39d5f03 scalar: add the
cache-servercommand189: 6f1fbf6 = 161: fd531ca scalar: add a test toggle to skip accessing the vsts/info endpoint
190: 8d10e15 = 162: bf10877 scalar: adjust documentation to the microsoft/git fork
191: 08ff9a2 = 163: eb608f8 scalar: enable untracked cache unconditionally
192: 7c6baac = 164: b682e21 scalar: parse
clone --no-fetch-commits-and-treesfor backwards compatibility193: 9f18ce7 = 165: 05c5b93 scalar: make GVFS Protocol a forced choice
194: 3a4e7ca = 166: f58358b scalar: work around GVFS Protocol HTTP/2 failures
195: 9cce374 = 167: 67ccce0 gvfs-helper-client: clean up server process(es)
199: d09dc85 = 168: 4bfd87e add/rm: allow adding sparse entries when virtual
196: f5be859 = 169: 0ff7288 scalar diagnose: accommodate Scalar's Functional Tests
200: 58224cd = 170: da9a24c sparse-checkout: add config to disable deleting dirs
197: 9436933 = 171: eb7910c ci: run Scalar's Functional Tests
201: 05aaeb8 = 172: 17c64c4 diff: ignore sparse paths in diffstat
198: aef5bda = 173: c39b7f4 scalar: upgrade to newest FSMonitor config setting
202: e79926b = 174: b8b43e6 repo-settings: enable sparse index by default
36: df922a6 = 175: 2852c75 TO-UPSTREAM: sequencer: avoid progress when stderr is redirected
203: 356ae2a = 176: eaffeaa TO-CHECK: t1092: use quiet mode for rebase tests
204: da2a6bc = 177: 3843aaf reset: fix mixed reset when using virtual filesystem
205: 4f30101 = 178: 9b6a20a diff(sparse-index): verify with partially-sparse
206: 4315664 = 179: 907b5ac stash: expand testing for
git stash -u207: 9e57c19 = 180: 87bf363 sparse-index: add ensure_full_index_with_reason()
208: 0e5f0f3 = 181: 109be76 treewide: add reasons for expanding index
209: 489acd2 = 182: c6231b3 treewide: custom reasons for expanding index
210: abe7e7a = 183: c86ddfd sparse-index: add macro for unaudited expansions
211: 8fda17a = 184: dba43a8 Docs: update sparse index plan with logging
212: 2cde767 = 185: 198d4d5 sparse-index: log failure to clear skip-worktree
213: 13fe64f = 186: e094d14 stash: use -f in checkout-index child process
214: 97b5bb6 = 187: 96c86f3 sparse-index: do not copy hashtables during expansion
215: 95c4494 = 188: 62bc1a1 TO-UPSTREAM: sub-process: avoid leaking
cmd216: 6cf74f8 = 189: def8d98 remote-curl: release filter options before re-setting them
217: b2e1916 = 190: adb8435 transport: release object filter options
218: efd811f = 191: 9358e69 push: don't reuse deltas with path walk
219: 3e8f9e4 = 192: cb7e6f1 t7900-maintenance.sh: reset config between tests
220: 4249d27 = 193: 9bd6087 maintenance: add cache-local-objects maintenance task
38: 7442ba8 = 194: 7967ebe revision: defensive programming
221: dd88b8e = 195: 9d8ee0d scalar.c: add cache-local-objects task
40: 756645e = 196: 94e7655 get_parent(): defensive programming
42: 8a5f81c = 197: 9986e1e fetch-pack: defensive programming
222: efe7b0b = 198: 14059ce hooks: add custom post-command hook config
46: 5028ac4 = 199: f349811 unparse_commit(): defensive programming
37: cef18fa = 200: 3cfba9d cat_one_file(): make it easy to see that the
sizevariable is initialized223: 45e1fae = 201: 227cd14 TO-UPSTREAM: Docs: fix asciidoc failures from short delimiters
48: 81b8b5d = 202: 4612c00 verify_commit_graph(): defensive programming
39: 5665bcc = 203: 97649e6 fsck: avoid using an uninitialized variable
224: 7b93d0d = 204: c138e50 hooks: make hook logic memory-leak free
50: f8806f3 = 205: 51e4d1f stash: defensive programming
41: fa69a03 = 206: 15f7344 load_revindex_from_disk(): avoid accessing uninitialized data
225: a22bb2c = 207: 1b82c45 t0401: test post-command for alias, version, typo
52: a068f57 = 208: 51a0a7c stash: defensive programming
43: 3513e5b = 209: f408262 load_pack_mtimes_file(): avoid accessing uninitialized data
226: 622b354 = 210: 96e3705 hooks: better handle config without gitdir
44: bdd7bde = 211: 9f6dc57 codeql: run static analysis as part of CI builds
45: ff1beac = 212: 15fd055 codeql: publish the sarif file as build artifact
47: 5771b17 = 213: a979c02 codeql: disable a couple of non-critical queries for now
49: 72fc726 = 214: ccc9944 date: help CodeQL understand that there are no leap-year issues here
51: e7531c7 = 215: c01f6ef help: help CodeQL understand that consuming envvars is okay here
54: f0ef8e7 = 216: 0890646 push: defensive programming
53: a377224 = 217: 6d260e0 ctype: help CodeQL understand that
sane_istest()does not access array past end55: b57fa4c = 218: 4b6132d test-tool repository: check return value of
lookup_commit()57: 9c9cb17 = 219: 9c01f1c fetch: defensive programming
56: bd2a9ee = 220: 0d66048 ctype: accommodate for CodeQL misinterpreting the
zinmallocz()58: 94a832a = 221: 31a19db shallow: handle missing shallow commits gracefully
60: f7294be = 222: cea32c6 inherit_tracking(): defensive programming
59: 356f748 = 223: f0b96c2 strbuf_read: help with CodeQL misunderstanding that
strbuf_read()does NUL-terminate correctly61: 4d3fc58 = 224: 0d1d6b1 commit-graph: suppress warning about using a stale stack addresses
62: d3ca7f7 = 225: 58638ae codeql: also check JavaScript code
106: bc3b1ce < -: ------------ trace2:gvfs:experiment: clear_ce_flags_1
227: 3caa9ca = 226: 612ce83 scalar: add run_git_argv
228: fa33190 = 227: c3d2ea2 scalar: add --ref-format option to scalar clone
229: f8f881f = 228: d135230 gvfs-helper: skip collision check for loose objects
230: f2e8005 = 229: 5b27e1d gvfs-helper: emit advice on transient errors
231: b4388b8 = 230: ddb3692 gvfs-helper: avoid collision check for packfiles
232: 9d66083 = 231: 8dee44f t5799: update cache-server methods for multiple instances
233: a4ea30f = 232: 8ea2da9 gvfs-helper: override cache server for prefetch
234: 6904216 = 233: fd7b4a5 gvfs-helper: override cache server for get
235: 8010cdb = 234: c8a7c94 gvfs-helper: override cache server for post
236: afd1285 = 235: 3ec76cb t5799: add test for all verb-specific cache-servers together
237: 511bdab = 236: da76a65 lib-gvfs-helper: create helper script for protocol tests
238: 00801f1 = 237: aef034f t579*: split t5799 into several parts
239: 79d34d0 = 238: 425dc84 scalar: add ---cache-server-url options
240: 90e5ea8 = 239: b7a6a6c Restore previous errno after post command hook
241: be59a9a = 240: a5efecf t9210: differentiate origin and cache servers
242: b9f735a = 241: 51bde98 unpack-trees: skip lstats for deleted VFS entries in checkout
243: 2712111 = 242: 01a2901 worktree: conditionally allow worktree on VFS-enabled repos
244: db1bdf6 = 243: 0f8f165 gvfs-helper: send X-Session-Id headers
245: df7ab83 = 244: 57ba578 gvfs-helper: create shared object cache if missing
246: 6b34cc2 = 245: 25ea558 gvfs: add gvfs.sessionKey config
247: 376fec2 = 246: 343fd83 gvfs: clear DIE_IF_CORRUPT in streaming incore fallback
248: 5a9eec9 = 247: 3123c94 workflow: add release-vfsforgit to automate VFS for Git updates
249: bfc6813 = 248: 05684e5 worktree remove: use GVFS_SUPPORTS_WORKTREES for skip-clean-check gate
250: 683f478 = 249: 5e4f930 ci: add new VFS for Git functional tests workflow
251: 54d109f = 250: 34345cb azure-pipelines: add stub release pipeline for Azure
252: ef89269 = 251: 8975897 diff: add renameThreshold configuration option
253: 697f145 = 252: 6789ad3 gvfs-helper: separate packfile extraction from indexing
254: 7c33eff = 253: f9b7087 blame: add blame.renames, blame.renameThreshold, blame.renameLimit
255: 9e6fc9d = 254: 8c56651 gvfs-helper: run prefetch index-pack in parallel
256: 1b3cf8f = 255: a7b145a gvfs-helper: add gvfs.prefetchThreads config for parallel prefetch
257: b393eae = 256: 7e7749d azure-pipelines: add ESRP code signing
258: 9d10a04 = 257: a5bccda azure-pipelines: allow overriding Git version
259: 25568ef = 258: 0eceaf5 azure-pipelines: build, sign and stage the Linux Debian package
260: 1c50a26 = 259: ab59b20 azure-pipelines: build, sign, notarize and stage the macOS installer
261: 4454d94 = 260: 08acd11 azure-pipelines: build, sign and stage the Windows installer
262: 70feba9 = 261: 7fa9ab3 azure-pipelines: enable on tag push, default ESRP and GitHub release on
263: fef6b5a = 262: 1b141e4 release: binskim for Windows
264: 7a9e267 = 263: d377a87 release: suppress unfixable binskim findings
265: c64168b = 264: c18e915 binskim: add baseline
266: 11e13fa = 265: ec0f443 checkout: preserve skip-worktree for virtual filesystem paths
267: aedfe6b = 266: f06b2f7 rust: pick a GCC-compatible Cargo target under MSYS2/MinGW
268: e0c8d66 = 267: de58de1 ci(vfs): install the GCC-compatible Rust target before building
269: 8ab1630 (upstream: 522ea8e) < -: ------------ osxkeychain: fix build with Rust
270: 64f1a64 = 268: 58d51d8 DEBUG
I used the opportunity to fix compile errors in one intermediate commit, and to clean up a couple of left-over changes that were done differently in upstream Git and are no longer necessary in this here fork.