Skip to content

Option to omit transfer on same storage volume #147

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
hUwUtao wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Option to omit transfer on same storage volume #147

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4ad2999
feat: option to omit transfer on same storage volume
hUwUtao Nov 23, 2025
98f8795
switch for storage pool value
hUwUtao Feb 11, 2026
b28dcac
Merge remote-tracking branch 'origin/main' into patch/synced-filesystem
hUwUtao Feb 11, 2026
2a1319a
fix storage pool delete cleanup condition
hUwUtao Feb 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions config/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -297,6 +297,17 @@ type Transfers struct {
//
// Defaults to 0 (unlimited)
DownloadLimit int `default:"0" yaml:"download_limit"`

// StoragePool configures whether this node participates in a shared storage pool.
StoragePool StoragePoolConfiguration `yaml:"storage_pool"`
}

type StoragePoolConfiguration struct {
// Enabled signals that this node shares a common data volume with other nodes.
Enabled bool `default:"false" yaml:"enabled"`

// PoolName is a per-node identifier used to compare shared storage pool membership across nodes.
PoolName string `yaml:"pool_name"`
}

type ConsoleThrottles struct {
Expand Down
28 changes: 19 additions & 9 deletions router/router_server.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -275,15 +275,25 @@ func deleteServer(c *gin.Context) {
//
// In addition, servers with large amounts of files can take some time to finish deleting,
// so we don't want to block the HTTP call while waiting on this.
go func(s *server.Server) {
fs := s.Filesystem()
p := fs.Path()
_ = fs.UnixFS().Close()
if err := os.RemoveAll(p); err != nil {
log.WithFields(log.Fields{"path": p, "error": err}).
Warn("failed to remove server files during deletion process")
}
}(s)
//
// Only skip file removal when:
// 1. shared storage pooling is explicitly enabled,
// 2. the pool has a name configured, and
// 3. this server is actively being transferred.
//
// This avoids preserving data for ordinary server deletions.
pool := config.Get().System.Transfers.StoragePool
skipFileRemoval := pool.Enabled && pool.PoolName != "" && s.IsTransferring()
if !skipFileRemoval {
go func(s *server.Server) {
fs := s.Filesystem()
p := fs.Path()
_ = fs.UnixFS().Close()
if err := os.RemoveAll(p); err != nil {
log.WithFields(log.Fields{"path": p, "error": err}).Warn("failed to remove server files during deletion process")
}
}(s)
}

// remove hanging machine-id file for the server when removing
go func(s *server.Server) {
Expand Down
14 changes: 14 additions & 0 deletions router/router_transfer.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,20 @@ func postTransfers(c *gin.Context) {
trnsfr.Server.Events().Publish(server.TransferStatusEvent, "success")
}(ctx, trnsfr)

{
remotePool := config.Get().System.Transfers.StoragePool
sourcePool := c.GetHeader("X-Storage-Pool")
if remotePool.Enabled && remotePool.PoolName != "" && sourcePool != "" && strings.EqualFold(remotePool.PoolName, sourcePool) {
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Pool name comparison relies on an unauthenticated header value for a security-sensitive decision.

While the request itself is authenticated via Bearer token, the X-Storage-Pool header value is entirely attacker-controlled. A compromised or malicious source node could set this header to match the destination's pool name, causing the transfer to succeed without any files being copied—resulting in an empty server directory on the destination.

Consider whether a stricter trust model is needed (e.g., the Panel could include the source pool name in the signed JWT transfer token instead of relying on a plain header).

🤖 Prompt for AI Agents 
In `@router/router_transfer.go` at line 135, The comparison using
remotePool.PoolName vs the unauthenticated header value sourcePool
(strings.EqualFold(remotePool.PoolName, sourcePool)) is unsafe because
sourcePool is attacker-controlled; change the check to validate the source pool
from an authenticated, signed source (e.g., the transfer JWT claims) instead of
the header. Update the transfer handling logic to parse and verify the transfer
token's claims (the signed source_pool claim) and replace the
strings.EqualFold(remotePool.PoolName, sourcePool) check with a comparison
against that verified claim (reject if the token's source_pool is missing or
does not match remotePool.PoolName), or alternatively require a strong identity
proof (client cert or panel-signed value) rather than the X-Storage-Pool header.
Ensure you still keep the remotePool.Enabled and remotePool.PoolName != ""
guards and add explicit rejection/logging when the authenticated claim does not
match.

if err := trnsfr.Server.CreateEnvironment(); err != nil {
middleware.CaptureAndAbort(c, err)
return
}
successful = true
c.Status(http.StatusOK)
return
}
}

mediaType, params, err := mime.ParseMediaType(c.GetHeader("Content-Type"))
if err != nil {
trnsfr.Log().Debug("failed to parse content type header")
Expand Down
9 changes: 9 additions & 0 deletions server/transfer/source.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ import (
"mime/multipart"
"net/http"
"time"

"github.com/pelican-dev/wings/config"
)

// PushArchiveToTarget POSTs the archive to the target node and returns the
Expand All @@ -21,6 +23,10 @@ func (t *Transfer) PushArchiveToTarget(url, token string, backups []string) ([]b
t.SendMessage("Preparing to stream server data to destination...")
t.SetStatus(StatusProcessing)

// Always include the configured storage pool identifier in the outgoing request headers.
// The destination can use this information to determine if it should skip copying files when both nodes share the same storage backend.
sp := config.Get().System.Transfers.StoragePool

a, err := t.Archive()
if err != nil {
t.Error(err, "Failed to get archive for transfer.")
Expand Down Expand Up @@ -61,6 +67,9 @@ func (t *Transfer) PushArchiveToTarget(url, token string, backups []string) ([]b
return nil, err
}
req.Header.Set("Authorization", token)
if sp.Enabled && sp.PoolName != "" {
req.Header.Set("X-Storage-Pool", sp.PoolName)
}

// Create a new multipart writer that writes the archive to the pipe.
mp := multipart.NewWriter(writer)
Expand Down