Skip to content

diff for debian 1.6.2-4#1582

Open
neolynx wants to merge 54 commits into
release/1.6.2from
debian/1.6.2-4
Open

diff for debian 1.6.2-4#1582
neolynx wants to merge 54 commits into
release/1.6.2from
debian/1.6.2-4

Conversation

@neolynx

@neolynx neolynx commented Jun 14, 2026

Copy link
Copy Markdown
Member

Backport of fixes from master to 1.6.2

=> NOT FOR MERGING

Feature included

  • Multiple GPG keys support (#1479)

this is a "fix" allowing to get out of a situation where keys are expiring by signing with a new key additionally.

Fixes included

  • Fix crash in aptly db recover (#1565)
  • Fix deadlocks in task list (#1529)
  • Fix S3 re-upload issue (#1480)
  • Fix aptly repo edit API (#1493)
  • Fix out-of-disk-space error handling (#1504)
  • Fix aptly mirror update removing unrelated params (#1466)
  • Fix concurrent pool linking race condition (#1481)
  • Fix dpkg-compliant version comparison (#1509)
  • Fix Swagger property casing and spec errors (#1510, #1498)
  • Remove useless nil check (#1482)
  • Format Go code with gofmt (#1483)
  • Race condition & concurrency fixes for the REST API (#1574)
  • Fix empty line in Package-List for source packages (#1588)
  • Publish: check storage exists before publishing (#1587)
  • S3 publish race condition (#1594)
  • Repo edit name optionally (#1593)

@neolynx neolynx force-pushed the debian/1.6.2-4 branch 3 times, most recently from c5e770f to 2d78c46 Compare June 14, 2026 22:50
@codecov

codecov Bot commented Jun 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 32.56743% with 675 lines in your changes missing coverage. Please review.
✅ Project coverage is 38.16%. Comparing base (d1bfd29) to head (67cdd00).

Files with missing lines Patch % Lines
api/publish.go 13.52% 236 Missing and 7 partials ⚠️
api/snapshot.go 6.66% 125 Missing and 1 partial ⚠️
api/repos.go 18.24% 118 Missing and 3 partials ⚠️
api/mirror.go 2.38% 41 Missing ⚠️
deb/publish.go 27.65% 31 Missing and 3 partials ⚠️
task/list.go 55.10% 21 Missing and 1 partial ⚠️
cmd/publish.go 0.00% 20 Missing ⚠️
azure/public.go 71.42% 14 Missing and 2 partials ⚠️
api/files.go 64.51% 9 Missing and 2 partials ⚠️
s3/public.go 62.06% 9 Missing and 2 partials ⚠️
... and 11 more

❗ There is a different number of reports uploaded between BASE (d1bfd29) and HEAD (67cdd00). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (d1bfd29) HEAD (67cdd00)
2 1
Additional details and impacted files
@@                Coverage Diff                 @@
##           release/1.6.2    #1582       +/-   ##
==================================================
- Coverage          74.88%   38.16%   -36.72%     
==================================================
  Files                159      159               
  Lines              18464    18888      +424     
==================================================
- Hits               13826     7209     -6617     
- Misses              3491    10982     +7491     
+ Partials            1147      697      -450     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@neolynx neolynx changed the base branch from release/1.6.2 to debian-master June 15, 2026 10:04
@neolynx neolynx force-pushed the debian/1.6.2-4 branch 9 times, most recently from 6ffc469 to c203075 Compare June 19, 2026 18:24
@neolynx neolynx changed the title diff diff for debian 1.6.2-4 Jun 19, 2026
@neolynx neolynx force-pushed the debian/1.6.2-4 branch 4 times, most recently from aeeac54 to 66631de Compare June 19, 2026 19:44
tonobo and others added 14 commits June 19, 2026 22:20
Implement support for the SOURCE_DATE_EPOCH environment variable as
specified by reproducible-builds.org. When set, this variable overrides
the current timestamp in the Release file's Date and Valid-Until fields,
enabling reproducible filesystem publishes.

- Read SOURCE_DATE_EPOCH environment variable in Publish()
- Use the epoch timestamp for both Date and Valid-Until fields
- Gracefully fallback to current time if unset or invalid
- Add comprehensive tests for valid and invalid SOURCE_DATE_EPOCH values
* enable in dev and test env
* fix api/repos doc
* remove useless resource lock
  Resource locks need to be before the background task. creating same publish endpoint at the same time is unlikely...
* load data inside background tasks
  This fixes a flaw in async apis, which loaded the published repo from the DB and mutated it outside the task closure, before the task lock was acquired.
  Perform collection.LoadComplete inside maybeRunTaskInBackground and have tasks use a fresh copy of taskCollectionFactory, taskCollection
* lock source repos/snapshots for publish operations
  Concurrent tasks were not properly locking their resources, leading to inconsistent published indexes:
  SourceLocalRepo: iterate published.Sources (component -> source UUID), look up each local repo via localRepoCollection.ByUUID and append string(repo.Key()) to resources
  SourceSnapshot: iterate b.Snapshots,look up each snapshot via snapshotCollection.ByName and append string(snapshot.ResourceKey()) to resources.
* lock pool on non MultiDist publish
* revert mutex on LinkFromPool
* use uuids, since names can be renamed
* add test for MultiDist change
* load data inside background tasks
  Perform collection.LoadComplete inside maybeRunTaskInBackground
  Have tasks use a fresh copy of taskCollectionFactory, taskCollection
* use uuids, since names can be renamed
 * perform collection.LoadComplete inside maybeRunTaskInBackground
 * have tasks use a fresh copy of taskCollectionFactory, taskCollection
 * fix locking for snapshots of snapshots by locking SourceSnapshots
 * use uuids, since names can be renamed
* load data inside background tasks
  Perform collection.LoadComplete inside maybeRunTaskInBackground
  Have tasks use a fresh copy of taskCollectionFactory, taskCollection
* show resources in task details
* fix task state locking
* return task object consistently

Race condition iexisted where task State, err, and processReturnValue fields
were written by consumer goroutine and read by concurrent accessors without
proper synchronization, causing torn reads and data races.
@neolynx neolynx deleted the branch release/1.6.2 June 20, 2026 08:17
@neolynx neolynx closed this Jun 20, 2026
@neolynx neolynx deleted the debian/1.6.2-4 branch June 20, 2026 08:26
@neolynx neolynx restored the debian/1.6.2-4 branch June 20, 2026 08:26
@neolynx neolynx reopened this Jun 20, 2026
@neolynx neolynx changed the base branch from debian-master to release/1.6.2 June 20, 2026 08:29
neolynx added 8 commits June 20, 2026 11:01
and path escape the new name param
Make sure pathCache is properly locked for concurrent access.

Add RWMutex to the PublishedStorage struct:
- Cache initialization
  Read-lock to test for nil, then write-lock with a second nil check before populating
- Cache reads
  RLock/RUnlock, allowing concurrent readers
- Cache writes / deletes
  Lock/Unlock
- run separate unit-test job
- build docker
- allow make docker-unit-tests in ci
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.