Skip to content

Sync: Add contact-table sync via cloud-folder journals#1045

Open
aa5sh wants to merge 1 commit into
foldynl:masterfrom
aa5sh:DBSync
Open

Sync: Add contact-table sync via cloud-folder journals#1045
aa5sh wants to merge 1 commit into
foldynl:masterfrom
aa5sh:DBSync

Conversation

@aa5sh
Copy link
Copy Markdown

@aa5sh aa5sh commented May 27, 2026

Multiple QLog installs that share a cloud folder (Dropbox, iCloud Drive, OneDrive, Syncthing, ...) keep their contacts tables in sync. Each install writes its own append-only JSON-lines journal into /qlog-sync/nodes//journal-NNNN.jsonl and reads every other node's journal back, applying remote upserts and deletes locally with last-writer-wins by updated_at.

Database (migration 039)

  • New columns on contacts: qso_uuid (UUIDv4), updated_at (ISO-8601 UTC), origin_node.
  • New contacts_tombstones(qso_uuid PK, deleted_at, origin_node) so hard deletes are observable for sync without changing any existing SELECT path.
  • New sync_runtime(key, value) so the AFTER INSERT / AFTER UPDATE / BEFORE DELETE triggers can read the current node id at fire time; the user can rename the node from the Sync dialog without rebuilding any trigger.
  • New sync_peers(node, last_file, last_offset, last_seen_at) for per-peer reader resume.
  • AFTER INSERT trigger auto-fills qso_uuid / updated_at / origin_node when a write path didn't set them, so manual log entry, ADIF import, WSJT-X and fldigi all get sync metadata for free.
  • AFTER UPDATE trigger bumps updated_at on every local edit and re- stamps origin_node to the current self_node_id, so an edit to a row that originally came from a peer is "claimed" by the editor and shows up in the writer's "origin_node = self" filter.
  • BEFORE DELETE trigger records a tombstone.
  • Migration also backfills existing rows with UUIDs, sensible timestamps, and the device hostname as the default node id.

ContactSync (core/ContactSync.{h,cpp})

  • Singleton with a 30 s timer that runs a flush + pull cycle, plus a requestFlush() slot wired to contactAdded / contactUpdated / contactDeleted so local UI changes hit the journal in the next event-loop tick (deferred via QTimer::singleShot to fire after QSqlTableModel::beforeUpdate / beforeDelete commits).
  • flushOnDb writes new local upserts and tombstones into the current journal file, rotates at 4 MB, and maintains a HEAD pointer + one-time manifest.json.
  • pullOnDb scans sibling nodes/* directories, follows each peer's journal-*.jsonl files in name order past sync_peers.last_offset, and applies each record.
  • applyUpsert resolves identity by qso_uuid: LWW if matched, honour tombstones, otherwise fingerprint-match (callsign + mode + band + sat_name + start_time within ±30 min, same fields as ADIF import) and either Skip or Merge by the user-chosen DuplicatePolicy. Merge converges peers on the lexicographically- smaller UUID — both peers reach the same decision without coordination, then LWW reconciles fields.
  • applyDelete deletes by qso_uuid (LWW vs local updates), or records a tombstone when no local row exists so a subsequent older insert from another peer is filtered out.
  • Internal *OnDb methods take QSqlDatabase &db so a future move to a worker thread can pass a per-call cloned connection without changing call sites. The dispatchers currently run synchronously on the main thread.
  • snapshotForCycle() caches folder + self for the duration of a cycle.
  • busyChanged(bool) signal lets the dialog gate its buttons.
  • LogParam grows typed wrappers for the seven sync settings.

UI (ui/SyncDialog.{h,cpp,ui})

  • New dialog under Logbook → "Contact Sync…" with an enable toggle, folder picker, editable Node ID + Save, duplicate-policy combo, last-flush + last-pull timestamps, pending-changes counter, and Flush Now / Pull Now buttons.

MainWindow wiring

  • contactAdded / contactUpdated / contactDeleted signals fire ContactSync::requestFlush via QTimer::singleShot(0); using a direct connection because Qt's queued connections can't copy a QSqlRecord& reference argument.
  • ContactSync::pulled (when applied > 0) triggers LogbookWidget::updateTable so remote-sourced rows appear in the logbook view without manual refresh.
  • ContactSync::start() runs on app launch when sync is enabled.

Lets two QLog installs that share a cloud folder (Dropbox, iCloud
Drive, OneDrive, Syncthing, ...) keep their contacts tables in sync.
Each install writes its own append-only JSON-lines journal into
<folder>/qlog-sync/nodes/<node>/journal-NNNN.jsonl and reads every
other node's journal back, applying remote upserts and deletes
locally with last-writer-wins by updated_at.

Database (migration 039)
- New columns on contacts: qso_uuid (UUIDv4), updated_at (ISO-8601
  UTC), origin_node.
- New contacts_tombstones(qso_uuid PK, deleted_at, origin_node) so
  hard deletes are observable for sync without changing any existing
  SELECT path.
- New sync_runtime(key, value) so the AFTER INSERT / AFTER UPDATE /
  BEFORE DELETE triggers can read the current node id at fire time;
  the user can rename the node from the Sync dialog without
  rebuilding any trigger.
- New sync_peers(node, last_file, last_offset, last_seen_at) for
  per-peer reader resume.
- AFTER INSERT trigger auto-fills qso_uuid / updated_at /
  origin_node when a write path didn't set them, so manual log
  entry, ADIF import, WSJT-X and fldigi all get sync metadata for
  free.
- AFTER UPDATE trigger bumps updated_at on every local edit and re-
  stamps origin_node to the current self_node_id, so an edit to a
  row that originally came from a peer is "claimed" by the editor
  and shows up in the writer's "origin_node = self" filter.
- BEFORE DELETE trigger records a tombstone.
- Migration also backfills existing rows with UUIDs, sensible
  timestamps, and the device hostname as the default node id.

ContactSync (core/ContactSync.{h,cpp})
- Singleton with a 30 s timer that runs a flush + pull cycle, plus a
  requestFlush() slot wired to contactAdded / contactUpdated /
  contactDeleted so local UI changes hit the journal in the next
  event-loop tick (deferred via QTimer::singleShot to fire after
  QSqlTableModel::beforeUpdate / beforeDelete commits).
- flushOnDb writes new local upserts and tombstones into the
  current journal file, rotates at 4 MB, and maintains a HEAD
  pointer + one-time manifest.json.
- pullOnDb scans sibling nodes/* directories, follows each peer's
  journal-*.jsonl files in name order past sync_peers.last_offset,
  and applies each record.
- applyUpsert resolves identity by qso_uuid: LWW if matched,
  honour tombstones, otherwise fingerprint-match (callsign + mode +
  band + sat_name + start_time within ±30 min, same fields as
  ADIF import) and either Skip or Merge by the user-chosen
  DuplicatePolicy. Merge converges peers on the lexicographically-
  smaller UUID — both peers reach the same decision without
  coordination, then LWW reconciles fields.
- applyDelete deletes by qso_uuid (LWW vs local updates), or
  records a tombstone when no local row exists so a subsequent
  older insert from another peer is filtered out.
- Internal *OnDb methods take QSqlDatabase &db so a future move to
  a worker thread can pass a per-call cloned connection without
  changing call sites. The dispatchers currently run synchronously
  on the main thread.
- snapshotForCycle() caches folder + self for the duration of a
  cycle.
- busyChanged(bool) signal lets the dialog gate its buttons.
- LogParam grows typed wrappers for the seven sync settings.

UI (ui/SyncDialog.{h,cpp,ui})
- New dialog under Logbook → "Contact Sync…" with an enable toggle,
  folder picker, editable Node ID + Save, duplicate-policy combo,
  last-flush + last-pull timestamps, pending-changes counter, and
  Flush Now / Pull Now buttons.

MainWindow wiring
- contactAdded / contactUpdated / contactDeleted signals fire
  ContactSync::requestFlush via QTimer::singleShot(0); using a
  direct connection because Qt's queued connections can't copy a
  QSqlRecord& reference argument.
- ContactSync::pulled (when applied > 0) triggers
  LogbookWidget::updateTable so remote-sourced rows appear in the
  logbook view without manual refresh.
- ContactSync::start() runs on app launch when sync is enabled.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant