Skip to content

fix(mediorum): avoid blocked replication pipe goroutines#336

Merged
raymondjacobson merged 1 commit into
mainfrom
fix/mediorum-replicate-goroutine-leak
Jun 3, 2026
Merged

fix(mediorum): avoid blocked replication pipe goroutines#336
raymondjacobson merged 1 commit into
mainfrom
fix/mediorum-replicate-goroutine-leak

Conversation

@raymondjacobson
Copy link
Copy Markdown
Contributor

Summary

  • prevent replicateFileToHost pipe writer goroutines from blocking forever when peer replication returns early
  • close the request body pipe on SignedPost/HTTP/non-200 failures so the multipart writer unblocks
  • add a regression test for an HTTP 500 peer that does not consume the request body

Context

In production, creator-2 had ~42k goroutines, with ~41k parked in MediorumServer.replicateFileToHost.func1 at pkg/mediorum/server/replicate.go:191. Those goroutines were blocked sending to an unbuffered errChan after failed repair/replication attempts. The pod memory graph was mostly file cache, but the openaudio heap was also inflated compared with creator-1.

The old code could return from the HTTP path before reading from errChan, and if the peer did not consume the body or the pipe copy failed, the writer goroutine could remain blocked.

Test Plan

  • go test -c ./pkg/mediorum/server
  • attempted go test ./pkg/mediorum/server -run TestReplicateFileToHostClosesPipeOnEarlyHTTPError -count=1, but local package TestMain requires Postgres on localhost:5454; it failed before running the test with connection refused.

@raymondjacobson raymondjacobson merged commit 8b7cd74 into main Jun 3, 2026
5 checks passed
@raymondjacobson raymondjacobson deleted the fix/mediorum-replicate-goroutine-leak branch June 3, 2026 19:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant