Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion apps/xftp-server/XFTPWeb.hs
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ xftpSubsts XFTPServerConfig {fileExpiration, logStatsInterval, allowNewFiles, ne
[("smpConfig", Nothing), ("xftpConfig", Just "y")] <> substConfig <> serverInfoSubsts simplexmqSource information <> [("onionHost", strEncode <$> onionHost), ("iniFileName", Just "file-server.ini")]
where
substConfig =
[ ("fileExpiration", Just $ maybe "Never" (fromString . timedTTLText . ttl) fileExpiration),
[ ("fileExpiration", Just . fromString . timedTTLText . ttl $ fileExpiration),
("statsEnabled", Just . yesNo $ isJust logStatsInterval),
("newUploadsAllowed", Just . yesNo $ allowNewFiles),
("basicAuthEnabled", Just . yesNo $ isJust newFileBasicAuth)
Expand Down
168 changes: 168 additions & 0 deletions plans/2026-08-22-xftp-file-storage-time.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
# Implementation plan: XFTP variable file storage time

Proposal: `../rfcs/2026-08-22-xftp-file-storage-time.md`.

## simplexmq: entitlement crypto

New module `Simplex.Messaging.Crypto.Entitlement`, over `Simplex.Messaging.Crypto.BBS`:

Types:

```
newtype MasterKey = MasterKey ByteString

data Entitlement = Entitlement
{ entitlementName :: Text,
expiresAt :: UTCTime,
extraInfo :: Text
}

data EntitlementCredential = EntitlementCredential
{ issuerKeyIdx :: Int,
masterKey :: MasterKey,
issuerSignature :: BBSSignature,
entitlement :: Entitlement
}

data EntitlementProof = EntitlementProof
{ issuerKeyIdx :: Int,
proof :: BBSProof,
entitlement :: Entitlement
}
```

Functions and constants:

- the disclosed-message encoding: the master key is message 0 and stays undisclosed; `expiresAt`, `entitlementName`, and `extraInfo` are messages 1 to 3 and are disclosed
- the BBS header string `"SimpleX badges v1"` (shared with chat's badges, which sign under it), the message count, and the disclosed indexes
- `generateEntitlementProof :: BBSPublicKey -> EntitlementCredential -> BBSPresHeader -> IO (Either String EntitlementProof)`
- `verifyEntitlement :: Map Int BBSPublicKey -> BBSPresHeader -> EntitlementProof -> IO (Maybe Bool)` (the caller supplies the presentation header; the server reconstructs it, the proof never includes it)
- the issuer public keys constant `Map Int BBSPublicKey`

## simplexmq: protocol, new XFTP version

In `Simplex.FileTransfer.Transport`:

- add the next `VersionXFTP` and set `currentXFTPVersion` to 4
- add `entitlementProof :: Maybe EntitlementProof` to `XFTPClientHandshake`, encoded before the `Tail` and only from this version
- the presentation header is the session id alone

In `Simplex.FileTransfer.Protocol`:

- add `GrantedStorageTime` and its encoding; retain the one-character sum prefix for future variants:

```
data GrantedStorageTime = GSTExpires {epochSeconds :: Int64}
```

- add the storage time (`Maybe Int64`: `Nothing` requests the server maximum, `Just` a number of hours) to `FNEW`
- add the granted storage to `FRSndIds` as `Maybe GrantedStorageTime` (`Nothing` when decoding a response from a server below this version)

## simplexmq: server configuration

In `Simplex.FileTransfer.Server.Env` and `Simplex.FileTransfer.Server.Main`:

- make `fileExpiration` non-optional (`ExpirationConfig`, no longer `Maybe`); the server always expires files, so the server maximum is always a concrete number of seconds
- read a maximum storage time (a number of hours) for each entitlement name from the `[STORE_LOG]` INI section, from the keys `expire_files_hours_for_supporter` and `expire_files_hours_for_legend`; an absent key is skipped (that name gets the default), a present but malformed value fails startup
- exit at startup if any name's maximum is below the default file expiration
- add `entitlementKeys :: Map Word16 BBSPublicKey` to the server config (default = the shared constant, set from `Main`); the handshake verifies the proof against it, so the trusted keys never come from the sender

## simplexmq: server session

In `Simplex.FileTransfer.Server`:

- `processClientHandshake` verifies the proof from the handshake, once per session, and resolves the maximum storage time for the entitlement name
- verify only when the answer can change: the name is configured with a maximum above the default, and the entitlement expired less than 24 hours ago. A proof that fails these checks or fails to verify is logged, and the session gets the default maximum
- `HandshakeAccepted` holds the resolved maximum for the session, and `processXFTPRequest` takes it from there, so no proof is verified while a command is processed
- `createFile` caps the requested storage time by the session maximum

## simplexmq: server store and expiration

The `files` table gets a nullable `expires_at`. Every new file stores a concrete `expires_at`. It is NULL only for pre-feature rows, which the migration must not re-date (it has no access to the operator's configured TTL); those are expired at query time as `created_at + ttl`.

Common to both stores, in `Simplex.FileTransfer.Server.Store`:

- add `expiresAt :: Maybe RoundedFileTime` to `FileRec`
- in `createFile`, verify the proof against `sessionId <> sndKey <> digest`, cap the requested hours at the entitlement's maximum, round the expiry up to the hour, store it, and return that same value as the granted storage
- a valid proof raises the maximum to the entitlement's configured value; a proof that fails verification, carries an unknown issuer key, or whose entitlement expired more than 24 hours ago falls back to the default maximum. The entitlement is honoured for 24 hours after its `expiresAt`.
- `expiredFiles` receives `now` and `old` (= `now - ttl`). A stored expiry is deleted when `expires_at < now` (no grace — it is already rounded up); a legacy row (no `expires_at`) is deleted when `created_at + fileTimePrecision < old` (the grace covers `created_at` being floored to the hour)
- retain `created_at` for statistics, export, and the legacy fallback

STM store:

- in `expiredFiles`, expire a new file when `roundedSeconds expiresAt < now`, and a legacy file (no `expiresAt`) when `created_at + fileTimePrecision < old`

PostgreSQL store, in `Simplex.FileTransfer.Server.Store.Postgres` and its migrations:

- add the nullable column `expires_at BIGINT` (no backfill)
- add one composite index `idx_files_expiry ON files (expires_at, created_at)`
- `expiredFiles` query: `WHERE (expires_at < ?) OR (expires_at IS NULL AND created_at < ?) LIMIT ?` with `(now, old - fileTimePrecision)`. The first arm deletes stored (already rounded-up) expiries; the second drains legacy rows, with the grace folded into `old - fileTimePrecision` so the columns stay bare and sargable. Keep the `OR` at the top level so each disjunct is independently indexable (BitmapOr on the composite index): `expires_at` covers arm 1's range and arm 2's `IS NULL` group, and `created_at` orders arm 2 within that group. A `COALESCE(expires_at, created_at + ttl)` predicate is avoided (not sargable, would force a sequential scan). No `ORDER BY` — the batch loop deletes all expired rows regardless of order.

Store log, in `Simplex.FileTransfer.Server.StoreLog`:

- add the optional expiration to the `AddFile` record; a record without it parses to `Nothing` (the configured default), never a hardcoded value

## simplexmq: agent

The credential belongs to the user, so the agent holds it the way it holds the user's servers: in memory, supplied when the agent is created and replaced through an API. It is not stored by the agent.

Per-user state in `Simplex.Messaging.Agent.Env.SQLite` and `Simplex.Messaging.Agent.Client`:

- add `entitlements :: Map UserId EntitlementCredential` to `InitialAgentServers`, beside the servers
- add `userEntitlements :: TMap UserId EntitlementCredential` to `AgentClient`, filled from it by `newAgentClient`
- add `entitlementKeys :: Map Word16 BBSPublicKey` to `AgentConfig` (default = the shared constant), for the issuer key that proof generation needs

Public API in `Simplex.Messaging.Agent`:

- add storage time (`Maybe Int64` hours) to `xftpSendFile`
- add `setUserEntitlement :: AgentClient -> UserId -> Maybe EntitlementCredential -> IO ()`, in the shape of `setProtocolServers`: it replaces the entry, and closes that user's XFTP clients, so the next upload presents the new credential

Store, in both the SQLite and PostgreSQL agent stores:

- add a nullable storage time column (integer hours; NULL means the server maximum) to `snd_files`
- add the migration to both stores
- in `createSndFile`, store the storage time

Upload, in `Simplex.Messaging.Agent.Client` and `Simplex.FileTransfer.Client`:

- `getXFTPClient` takes a proof for the session as a parameter, `SessionId -> IO (Maybe EntitlementProof)`, beside the callback it already takes for a closed client. The client config holds no credential and no keys
- `getXFTPServerClient` passes a function that reads the user's credential, looks the issuer key up, and generates the proof over the session id. A missing credential or a failure to generate gives `Nothing`, with the failure logged
- `xftpClientHandshakeV1` calls it with the session id from the connection, and sends the result in the handshake
- `agentXFTPNewChunk` reads the storage time from the send record and sends FNEW with it
- `createXFTPChunk` returns the granted expiry (epoch seconds); `agentXFTPNewChunk` stores it on `NewSndChunkReplica`

Completion:

- `createXFTPChunk` returns the granted expiry as `Maybe GrantedStorageTime`; `SndFileChunkReplica` and `NewSndChunkReplica` carry `expiresAt :: Maybe GrantedStorageTime`
- persist it in a nullable `replica_expires_at` column on `snd_file_chunk_replicas` (added to the entitlement migration): `createSndFileReplica` stores `epochSeconds`, `getSndFile` reads it back into `GSTExpires`
- on `SFDONE`, report the file expiry: a chunk expires when its last replica expires (`max` over replicas, absent replicas ignored, `Nothing` only if none report); the file expires when its first chunk expires (`min` over chunks, `Nothing` if any chunk is unknown). `GrantedStorageTime` derives `Ord`
- `SFDONE` gains a trailing `Maybe GrantedStorageTime` (not str-encoded); chat consumes it (wired later)

Testing:

- e2e test in `tests/XFTPAgent.hs`: generate a BBS keypair, sign a supporter credential (issuer key index 1), run the server with `entitlementKeys = {1: testPk}` and a supporter maximum above the default, run the sender agent with the same `entitlementKeys` and the credential for the user, send a file requesting a number of hours above the default and below that maximum, and assert `SFDONE`'s granted expiry rounds up `now + requested` (proof of the entitlement raising the max above the default)
- the same upload without the credential is capped at the default maximum
- store log round trip in `tests/CoreTests/StoreLogTests.hs`, in the shape of the SMP store log test: a file record survives a write, a read into the store, and compaction, including a file blocked with a notice, where the record has a field after the blocking info

## simplex-chat

- remove lifetime badges: make `badgeExpiry` a `UTCTime`, drop the `"lifetime"` encoding, and remove the lifetime option from the UI and the CLI
- map `BadgeInfo` to `Entitlement` (`entitlementName = textEncode badgeType`, `expiresAt = badgeExpiry`, `extraInfo = badgeExtra`) when calling the agent
- pass the user's credential to the agent when it is created, and through `setUserEntitlement` when the badge changes, in the same places that pass and update the user's servers
- pass the storage time to `xftpSendFile`
- retain the `maxXFTPFileSize` size limit
- reuse `verifyEntitlement` for peer-badge verification
- import the issuer public keys from the shared simplexmq constant

## State

Steps 2 to 5 are implemented in simplexmq. Step 1 and step 6 belong to the chat branch that carries badges.

## Order

1. Add the entitlement crypto module; move chat's badge verification onto it and remove lifetime badges.
2. Add the new XFTP version, the FNEW storage time, and the response.
3. Change the server configuration, store, expiration, and store log.
4. Move the proof to the handshake: the handshake field, the session state on the server, and the proof for the session on the client.
5. Hold the credential per user in the agent, and add the API to replace it.
6. Wire chat to pass the credential and the storage time.
101 changes: 101 additions & 0 deletions rfcs/2026-08-22-xftp-file-storage-time.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# XFTP variable file storage time

## Summary

The server stores a storage time for each file. The sender sets it in the FNEW command. The client may present a proof of an entitlement in the handshake to raise the maximum storage time the server allows. The proof is bound to the TLS session, so it cannot be reused for another session.

An entitlement belongs to the user, not to a file: the client presents it once per connection, and the server applies it to everything the client does in that session. The server can also vary other limits, such as throttling, by the entitlement.

## Entitlement

An entitlement is a name, an expiration, and an extra string. It is the disclosed content of a BBS proof: the holder's secret remains undisclosed, and the three fields are revealed. The server reads `entName` to select a maximum storage time, checks `entExpires`, and ignores `entExtra`; the interpretation of `entExtra` is out of scope here. The protocol references only the entitlement, never a badge; chat maps its own badge to an entitlement before it asks the agent to send.

The proof discloses the entitlement and includes the issuer key index and the BBS proof. The holder's secret and the BBS signature remain with the sender and are never transmitted. The origin of the sender's signed entitlement, from the entitlement service, is out of scope here.

```
entitlement = entName entExpires entExtra
entName = shortString ; e.g. "supporter", "legend"
entExpires = shortString ; expiration as a UTCTime ISO8601 string
entExtra = shortString ; opaque, interpretation out of scope
Comment thread
epoberezkin marked this conversation as resolved.

entitlementProof = issuerKeyIndex bbsProof entitlement
issuerKeyIndex = 2*2 OCTET ; Word16, network byte order
bbsProof = largeString ; BBS proof bytes
```

The presentation header that the BBS proof is generated over is not transmitted; the server takes it from the session (see [Binding](#binding)), which is what binds the proof.

## Storage time

```
fileStorageTime = %s"0" / (%s"1" storageHours)
storageHours = 8*8 OCTET ; Int64, network byte order
```

The storage time is an optional number of hours. Absent (`%s"0"`) requests the maximum the server allows for the presented entitlement, or the default maximum when no proof is present. A value (`%s"1"` with hours) requests a specific number of hours.

## Handshake, new XFTP version

The client handshake carries the entitlement proof.

```
clientHandshake = xftpVersion keyHash optEntitlementProof
optEntitlementProof = %s"0" / (%s"1" entitlementProof)
```

`xftpVersion` and `keyHash` are defined by the current XFTP protocol. Version 3 and earlier encode no proof.

The server verifies the proof once, when it accepts the handshake, and keeps the resulting maximum storage time for the session. A proof that fails to verify, names an entitlement the server does not configure, or names one whose expiration passed more than 24 hours ago, is logged and ignored, and the session gets the default maximum. The client learns nothing about which entitlements the server accepts.

## Commands

The new protocol version extends FNEW with the storage time.

```
fnew = %s"FNEW " fileInfo rcvKeys optBasicAuth fileStorageTime
```

`fileInfo`, `rcvKeys`, and `optBasicAuth` are defined by the current XFTP protocol. Version 3 and earlier encode no `fileStorageTime`, and the server applies the default storage time.

## Responses

FNEW extends the SIDS response with the granted storage.

```
sndIds = %s"SIDS " senderId rcvIds optGrantedStorageTime
optGrantedStorageTime = %s"0" / (%s"1" grantedStorageTime)
grantedStorageTime = grantedExpires
grantedExpires = %s"T" expiresAt
expiresAt = 8*8 OCTET ; Int64, seconds since epoch (absolute UTC instant), network byte order
```

`grantedExpires` returns the absolute expiration — the same value stored for the file. The sum encoding retains a one-character prefix so further variants can be added. Version 3 and earlier omit `optGrantedStorageTime` entirely; a client decoding such a response reads it as absent. `senderId` and `rcvIds` are defined by the current XFTP protocol.

## Binding

The presentation header binds the proof to the TLS session, so a proof presented on any other session fails to verify.

```
presHeader = sessionId
```

`sessionId` is the TLS session identifier, the TLS unique channel binding. Both sides take it from the connection: the client has it once TLS is established, and the client checks that the identifier the server sends in its handshake matches.

Binding to the session is what stops a proof being replayed by another client. A proof is not bound to a file, because the entitlement belongs to the user and authorises everything the client does in that session.

## Maximum storage time

The server configures a maximum storage time for each entitlement name, and a default maximum for requests with no proof. Each maximum is a number of hours. The server exits at startup if any name's maximum is below the default, so a proof never reduces the allowed time. The server honours an entitlement for 24 hours after its expiration; past that grace it is treated as no proof.

If the requested time exceeds the maximum, the server stores the file for the maximum and does not reject the request. The expiration is rounded up to the hour, stored, and returned as `grantedExpires`.

## Encoding primitives

```
shortString = length *OCTET ; 0-255 bytes
largeString = length2 *OCTET
length = 1*1 OCTET
length2 = 2*2 OCTET ; Word16, network byte order
```

`senderId`, `rcvIds`, `fileInfo`, `sndKey`, `digest`, `rcvKeys`, `optBasicAuth`, and `sessionId` are defined by the current XFTP and SMP protocols.
3 changes: 3 additions & 0 deletions simplexmq.cabal
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,7 @@ library
Simplex.Messaging.Crypto.Lazy
Simplex.Messaging.Crypto.Ratchet
Simplex.Messaging.Crypto.BBS
Simplex.Messaging.Crypto.Entitlement
Simplex.Messaging.Crypto.SNTRUP761
Simplex.Messaging.Crypto.SNTRUP761.Bindings
Simplex.Messaging.Crypto.SNTRUP761.Bindings.Defines
Expand Down Expand Up @@ -192,6 +193,7 @@ library
Simplex.Messaging.Agent.Store.Postgres.Migrations.M20260410_receive_attempts
Simplex.Messaging.Agent.Store.Postgres.Migrations.M20260411_service_certs
Simplex.Messaging.Agent.Store.Postgres.Migrations.M20260712_address_dr_rpc
Simplex.Messaging.Agent.Store.Postgres.Migrations.M20260823_snd_files_entitlement
else
exposed-modules:
Simplex.Messaging.Agent.Store.SQLite
Expand Down Expand Up @@ -245,6 +247,7 @@ library
Simplex.Messaging.Agent.Store.SQLite.Migrations.M20260410_receive_attempts
Simplex.Messaging.Agent.Store.SQLite.Migrations.M20260411_service_certs
Simplex.Messaging.Agent.Store.SQLite.Migrations.M20260712_address_dr_rpc
Simplex.Messaging.Agent.Store.SQLite.Migrations.M20260823_snd_files_entitlement
Simplex.Messaging.Agent.Store.SQLite.Util
if flag(client_postgres) || flag(server_postgres)
exposed-modules:
Expand Down
Loading