Repository navigation
Move guestbd NBD server from github.com/bradfitz/guestbd - #5
Merged
Merged
Conversation
Migrated-from: bradfitz/guestbd@81bded7
Migrated-from: bradfitz/guestbd@2850465
Migrated-from: bradfitz/guestbd@a7d309a
Signed-off-by: Brad Fitzpatrick <brad@danga.com> Migrated-from: bradfitz/guestbd@8aa0ef0
Signed-off-by: Brad Fitzpatrick <brad@danga.com> Migrated-from: bradfitz/guestbd@dd673e1
Signed-off-by: Brad Fitzpatrick <brad@danga.com> Migrated-from: bradfitz/guestbd@2317712
Signed-off-by: Brad Fitzpatrick <brad@danga.com> Migrated-from: bradfitz/guestbd@f6661c9
Signed-off-by: Brad Fitzpatrick <brad@danga.com> Migrated-from: bradfitz/guestbd@3536514
Signed-off-by: Brad Fitzpatrick <brad@danga.com> Migrated-from: bradfitz/guestbd@09cb293
Signed-off-by: Brad Fitzpatrick <brad@danga.com> Migrated-from: bradfitz/guestbd@4168a80
Signed-off-by: Brad Fitzpatrick <brad@danga.com> Migrated-from: bradfitz/guestbd@abc699b
Signed-off-by: Brad Fitzpatrick <brad@danga.com> Migrated-from: bradfitz/guestbd@c61c2a6
Migrated-from: bradfitz/guestbd@5ee0744
Signed-off-by: Brad Fitzpatrick <brad@danga.com> Migrated-from: bradfitz/guestbd@fa8555b
Migrated-from: bradfitz/guestbd@287490d
Signed-off-by: Brad Fitzpatrick <brad@danga.com> Migrated-from: bradfitz/guestbd@46f3a0b
Rename Conn to Snapshot to reflect that a writable layer can outlive individual TCP connections. Remove the net.Conn field from Snapshot so snapshots are independent of any particular connection. The NBD protocol code now receives the snapshot and TCP connection separately. Introduce a ServerOption variadic pattern for NewServer, moving pageSize (default 4096), maxMem (default 1GB), and the new shared-snapshot policy out of positional parameters. Add WithPageSize, WithMaxMem, and WithSharedSnapshot options. Snapshot now implements io.ReaderAt and io.WriterAt, exporting the read/write methods that were previously unexported handleRead and handleWrite. The NBD transmission code uses these directly. The cmd/guestbd binary gains a --shared-snapshot flag that causes all connections to share a single writable snapshot, allowing reconnecting clients to see previous writes. Migrated-from: bradfitz/guestbd@b551e3e
Replace the file-path-based server constructor with a BaseImageSource function that returns a BaseImage interface. BaseImage includes io.ReaderAt, io.Closer, Size, and BaseImageKey. The key enables equivalence-keyed caching: when a connection closes and a new one opens the same image (same dev/ino), the idle baseImageState is reused, preserving the page hash table and avoiding cold disk reads. - Add BaseImage interface (ReaderAt + Closer + Size + BaseImageKey) - Add BaseImageSource, FileSource, NewBaseImage constructors - Add WithMaxIdleBase(n) option (default 1) - Replace single roFile with roFiles map + roFileNoKey for nil-key sources - Add LRU eviction of idle keyed entries via evictIdleLocked - Rename readonlyFile → baseImageState with documented locking rules Migrated-from: bradfitz/guestbd@6ac3961
otherwise an 80 GB base image of mostly zeros takes 640 MB Signed-off-by: Brad Fitzpatrick <brad@danga.com> Migrated-from: bradfitz/guestbd@428f94c
Signed-off-by: Brad Fitzpatrick <brad@danga.com> Migrated-from: bradfitz/guestbd@c9b37e4
Signed-off-by: Brad Fitzpatrick <brad@danga.com> Migrated-from: bradfitz/guestbd@3955624
Migrated-from: bradfitz/guestbd@95d871e
syscall.Stat_t.Dev is an int32 on darwin (and uint64 on linux), so FileSource didn't compile there. Migrated-from: bradfitz/guestbd@f645ef5
Apple's Virtualization.framework NBD client (VZNetworkBlockDeviceStorageDeviceAttachment) sends NBD_OPT_INFO, asking for the block size, before NBD_OPT_GO, and gives up if the server replies NBD_REP_ERR_UNSUP. The framework then reports only "The storage device attachment is invalid" when the VM starts. Reply to NBD_OPT_INFO as for NBD_OPT_GO but stay in option haggling, as the spec says. Also gofmt. Migrated-from: bradfitz/guestbd@2c3bd8a
So a caller can persist a shared snapshot's writes onto a copy of the base image: boot a VM over NBD, let it write, then keep the result. mantsana uses this to build warm macOS VM snapshots, whose saved machine state only restores with the same kind of disk attachment (NBD) it was saved with. Migrated-from: bradfitz/guestbd@c2d85dd
This merges the history of github.com/bradfitz/guestbd's main branch, through bradfitz/guestbd@c2d85ddcfd9f, rewritten for its new home in tb. In each rewritten commit: * the guestbd package at the repo root moved to guestbd/, along with its README.md and .gitignore. * cmd/guestbd stayed at cmd/guestbd. * LICENSE was removed, as tb's covers it. * import paths and the go.mod module line use github.com/tailscale/tb. * a Migrated-from trailer names the original commit. This merge commit itself only combines go.mod and go.sum. Updates #cleanup Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
FileSource keyed base images by the device and inode from syscall.Stat_t, which doesn't exist on Windows, so guestbd and cmd/guestbd failed to build there and broke tb's Windows CI job. Move that into a fileIdentity helper for Unix, and elsewhere return a nil key, which BaseImage already defines as no identity: such base images are simply never coalesced. Updates #cleanup Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com> Change-Id: I617be9c30995dd6a1c9e54bce2bc8108c6bec80a
The test disconnects its first client and immediately connects a second, expecting the second to get a fresh base image because sources with nil keys are replaced once idle. But the server releases the first connection's base image asynchronously, after it reads the disconnect. If the second client connected first, it shared the still-active base image, warm page hashes and all, and saw no cold reads. With GOMAXPROCS=1 that happened every time, and it was flaky on macOS CI. Wait for the server to have no active base images before reconnecting. Updates #cleanup Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com> Change-Id: I70389c7db675f2b05bacb01d05bf7f795660251c
The previous commit made guestbd build on Windows by giving files there no identity key, but that disabled base image coalescing, which TestInodeSharing and TestReconnectHitsCache test, so they failed on Windows CI. Key files on Windows by volume serial number and file index, its equivalent of a device and inode, read from the open handle. Platforms other than Unix and Windows still get no key. Skip TestBaseImageReplaced on Windows: the server keeps the idle base image open for reuse, and Windows can't rename over an open file. Verified by running the Windows test binary under Wine, which reproduced all three CI failures before this change. Updates #cleanup Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com> Change-Id: I45c01fceaa4cc933f0e1c3d64514d5756c631217
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This migrates our NBD server from github.com/bradfitz/guestbd
to this tailscale/tb repo.