Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,9 +155,16 @@ deepseek free off # forget the enrolment on this machine
```

Free-tier limits, per UTC day: 30 requests, 60K input tokens, 20K output
tokens, 4K output per call, 128KB per request body, `deepseek-v4-flash`
only. A request for pro is **refused, not downgraded**. `models` and
`status` cost no quota; everything that can generate a token does.
tokens, 3 web searches, 4K output per call, 128KB per request body,
`deepseek-v4-flash` only. A request for pro is **refused, not
downgraded**. `models` and `status` cost no quota; everything that can
generate a token does.

`respond --web-search` works on the free tier and spends one of the three
daily searches. It is rationed that tightly because DeepSeek reads whole
pages into the prompt — one measured search request billed 40K input
tokens, about ten ordinary turns — so treat it as a few lookups a day, not
a research loop. Other server-side tools are still refused.

Errors from the gateway carry `"type":"free_tier_*"` and a message that
already contains the next step — do not append DeepSeek's own advice to
Expand Down
65 changes: 65 additions & 0 deletions TASTE.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,24 @@ a human always has somewhere to go.
**Expires.** If DeepSeek publishes a documented status JSON endpoint,
consume it and report both.

**2026-08-07 — expiry tested, still holds, and now for a second reason.**
Probed for a JSON endpoint from the production host: `/api/v2/summary.json`,
`/api/v2/status.json`, `/api/status`, `/api/v1/status` and the bare page all
fail. `status.deepseek.com` resolves to an Aliyun Beijing load balancer
(`statuspage.flashcat.cloud`) that accepts the TCP connection and then never
completes the TLS handshake from outside China, while `api.deepseek.com`
answers fine from the same box. So the page is both unparseable and
unreachable from where our gateway runs. Consuming it would mean shipping a
health signal that is permanently unknown — worse than the wrong all-clear
this scar was written about, because it would also be *our* dashboard showing
red for *their* geography. What the gateway publishes instead is a
first-party observation: every round trip to `api.deepseek.com` is recorded,
and `/v1/status` carries an `upstream` block with the last success, the
consecutive-failure streak and the last fault shape. That answers the
question a visitor actually has — is this you or them — with data a status
page structurally cannot have, since it cannot see a route that is broken
only from here. The incident page stays linked for humans.

---

## 2026-08-05 rejected: shipping the tokenizer, or estimating from the ratios
Expand Down Expand Up @@ -345,3 +363,50 @@ vectors by coincidence and fail against the gateway every time.

**Expires.** If the CLI and the gateway ever merge into one module, the
Go halves should share code and only the browser stays separate.

---

## 2026-08-07 accepted, after a measurement: `web_search` on the free tier

Recorded here because it reverses an earlier refusal, and a reversal
without its evidence is just a mood swing.

**The refusal it replaces.** `policy.forbidServerTools` rejected every
server-side tool with a real argument: such a tool "performs billed work
that never appears in the usage object, which would put its cost outside
every ceiling this gateway enforces". Correct reasoning, untested premise.

**What the measurement showed.** One `respond --web-search` call against
the live API on 2026-08-07 made eleven server-side calls (searches, page
opens, an in-page find) and reported 40,260 input tokens, 32,000 of them
cache hits, 3,100 output. The account balance moved by nothing beyond
those tokens — eleven searches at a frontier vendor's $10-per-1,000 rate
would have been $0.11, or 0.79 CNY, and unmistakable against a 14.26 CNY
balance. So the premise was wrong in the half that mattered: there is no
per-search fee, and the entire cost of a search arrives as input tokens
that this gateway already meters exactly.

**What the measurement did break.** The half of the premise that was
right, in a different place than expected. `meter.Estimate` bounded a
request's input at one token per body byte — true for every other request
and false for this one, because DeepSeek chooses how many pages to read.
A search request's input is upstream-controlled, so the admission
reservation no longer bounds it from the body.

**Reuse.** Allow `web_search` (both the bare and dated tool names), refuse
every other server-side tool still, and pay for the change in two places:
a 256k-token input allowance in the reservation, roughly six times the
observed case; and a per-user daily ration of three searches, which is a
new quota dimension because one search costs about what ten ordinary turns
cost and the request counter alone would let one caller quietly take a
quarter of the day's budget. Stating the trade honestly: within the
allowance the budget is still a hard ceiling, and beyond it a search can
overshoot by the difference, bounded by the few distinct callers who can
be mid-search at once given a per-subject in-flight cap of one.

**Expires.** If DeepSeek starts billing searches separately, or documents
a cap on injected search context, redo the arithmetic — the allowance and
the ration are both sized to a single measurement and should be re-measured
when the tool changes. If a search request is ever observed above 256k
input tokens in production, that is the signal to raise the allowance
rather than to quietly accept the overshoot.
34 changes: 34 additions & 0 deletions gateway/DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,7 @@ dollar.**
| `DSGATE_ANON_DAILY_REQUESTS` | 30 | enough to be useful for a day's work, not enough to script against |
| `DSGATE_ANON_DAILY_INPUT_TOKENS` | 60000 | ~150 pages of context per day |
| `DSGATE_ANON_DAILY_OUTPUT_TOKENS` | 20000 | the expensive side; the real cap |
| `DSGATE_ANON_DAILY_SEARCHES` | 3 | a `web_search` request costs ~10 ordinary turns; the request counter alone would let one caller take a quarter of the day |
| `DSGATE_ANON_MAX_TOKENS` | 4096 | clamps a single response, bounding overshoot |
| `DSGATE_MAX_BODY_BYTES` | 131072 | ~32K tokens; bounds the input side of overshoot |
| `DSGATE_DAILY_BUDGET_USD` | 1.00 | the circuit breaker; the number that actually protects us |
Expand All @@ -312,6 +313,19 @@ dollar.**
| `DSGATE_TOKEN_TTL_DAYS` | 7 | identities age out instead of accumulating |
| `DSGATE_BALANCE_CHECK_MINUTES` | 15 | the ledger's "we have credit" is checked against the real account |

**`web_search` is carried, and rationed.** Measured on 2026-08-07, one
search request made eleven server-side calls and billed 40,260 input
tokens with no separate per-search fee — so its whole cost arrives as
input tokens the meter already reads. What it breaks is the *reservation*,
which bounded input at one token per body byte: DeepSeek chooses how many
pages to read, so a search's input is upstream-controlled. Hence a 256k
input allowance at admission (about 6x the observed case) plus the daily
ration above. Within that allowance the budget is still a hard ceiling;
past it a search can overshoot by the difference, bounded by how many
distinct callers can be mid-search at once. Every other server-side tool
stays refused: unknown work at an unknown price, spent from donated
credit. Reasoning and the expiry condition are in `TASTE.md`.

**Free tier is flash only.** Pro is 3x the price and the request is
*rejected*, not silently downgraded — a user who asked for pro and got
flash without being told would draw wrong conclusions and blame the model.
Expand All @@ -333,6 +347,8 @@ What that document says and what it refuses to say are both deliberate:
| subject ids **truncated to 6 chars** | whole subject ids | a whole id can be matched against the one in someone's `free.json` |
| **per-country** request counts | anything per-IP, ever | an aggregate is a fact about the service, not about a person |
| token counts, tok/s, live subjects | prompts, completions | the promise in `deepseek free` is the promise here |
| **requests per day** for 30 days | per-day spend | a spend series is a map of when we are cheapest to empty |
| **upstream** last-success, fault streak | upstream key or account detail | "is it you or DeepSeek" is the question a failing caller has |

Exact money, per-key health and the full subject table live behind
`GET /admin/status` with the operator token.
Expand All @@ -342,6 +358,24 @@ maps, everything in memory and lost on restart. That is the right trade —
losing it costs a graph, and it keeps observability from ever becoming a
second, weaker copy of the money.

**The main chart is a day per bar, not a second.** It was output tokens
per second over five minutes, which is the wrong instrument for a pool
serving a few requests an hour: idle almost every second, so the line was
flat at zero whenever anyone looked and said nothing true about whether
the service works. The daily series comes from `quota.History`, which
reads the journals — the same files the money settles from, so the pretty
chart cannot disagree with the ledger. Finished days are immutable and
memoised; only today is recomputed. Quiet days are present as zeroes,
because a gap in a chart reads as missing data rather than as a quiet day.

**Upstream health is first-party, not scraped.** Every round trip to
DeepSeek is recorded with its outcome, and only *their* failures (429, 5xx,
transport) count as faults — a caller's own 4xx would otherwise read as an
outage. Three consecutive faults reads as `down`, one as `degraded`, and
before the first forwarded request the state is `unknown` rather than a
cheerful all-clear it has not earned. `status.deepseek.com` is deliberately
not consumed; see `TASTE.md` for the probe results.

## The key pool

One key was a single point of failure with a hard floor. Package
Expand Down
2 changes: 2 additions & 0 deletions gateway/cmd/dsgate/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ Per-user daily limits:
DSGATE_ANON_DAILY_REQUESTS (30)
DSGATE_ANON_DAILY_INPUT_TOKENS (60000)
DSGATE_ANON_DAILY_OUTPUT_TOKENS (20000)
DSGATE_ANON_DAILY_SEARCHES (3) server-side web searches per user
DSGATE_ANON_MAX_TOKENS (4096) per-request output cap
DSGATE_MAX_BODY_BYTES (131072) per-request body cap
DSGATE_REQUESTS_PER_MINUTE (20) per-address burst
Expand Down Expand Up @@ -132,6 +133,7 @@ func run() error {
DailyRequests: envInt("DSGATE_ANON_DAILY_REQUESTS", 30),
DailyInputTokens: envInt("DSGATE_ANON_DAILY_INPUT_TOKENS", 60000),
DailyOutputTokens: envInt("DSGATE_ANON_DAILY_OUTPUT_TOKENS", 20000),
DailySearches: envInt("DSGATE_ANON_DAILY_SEARCHES", 3),
DailyBudgetUSD: envFloat("DSGATE_DAILY_BUDGET_USD", 1.00),
TotalBudgetUSD: envFloat("DSGATE_TOTAL_BUDGET_USD", 20.00),
}
Expand Down
30 changes: 28 additions & 2 deletions gateway/internal/meter/meter.go
Original file line number Diff line number Diff line change
Expand Up @@ -89,9 +89,17 @@ func Cost(model string, u Usage) float64 {
// default upstream, and DeepSeek does not document that reasoning
// tokens respect max_tokens — they are billed as output either way,
// so the bound assumes they do not.
func Estimate(model string, requestBytes, maxTokens int) float64 {
//
// A search request breaks the first rule: the pages DeepSeek reads on the
// caller's behalf arrive as input tokens the body never contained, so
// searchInputAllowance is added to the input bound instead.
func Estimate(model string, requestBytes, maxTokens int, search bool) float64 {
input := requestBytes + 1
if search {
input += searchInputAllowance
}
return Cost(model, Usage{
InputTokens: requestBytes + 1,
InputTokens: input,
OutputTokens: maxTokens + reasoningAllowance,
Found: false,
})
Expand All @@ -103,6 +111,24 @@ func Estimate(model string, requestBytes, maxTokens int) float64 {
// a cent, so over-reserving costs headroom, not money.
const reasoningAllowance = 32 << 10

// searchInputAllowance is the input headroom reserved for a server-side
// web search, whose page reads land in input_tokens without ever passing
// through the request body.
//
// 256k is a judgement, not a proof. A search request measured live on
// 2026-08-07 reported 40,260 input tokens after eleven server-side calls,
// so this is roughly six times the observed case; the model's 1M context
// is the only true bound, and reserving 1M would price a single search at
// more than half a day's budget and make the feature unofferable.
//
// The honest statement of the trade: within this allowance the budget is
// still a hard ceiling, and beyond it a search request can overshoot by
// the difference. Two things keep that survivable — the per-subject
// in-flight cap means one caller cannot stack such requests, and searches
// are rationed per user per day, so the overshoot is bounded by the few
// distinct callers who can be mid-search at the same moment.
const searchInputAllowance = 256 << 10

// rawUsage is permissive on purpose: it decodes the usage object of every
// format at once, using pointers so "absent" and "zero" stay distinct.
// Which fields are present is what identifies the format.
Expand Down
22 changes: 21 additions & 1 deletion gateway/internal/meter/meter_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ func TestEstimateExceedsATypicalRealCharge(t *testing.T) {
const body = 4000
const maxTokens = 4096

est := Estimate("deepseek-v4-flash", body, maxTokens)
est := Estimate("deepseek-v4-flash", body, maxTokens, false)
real := Cost("deepseek-v4-flash", Usage{InputTokens: body / 3, OutputTokens: 800, Found: true})
if est <= real {
t.Errorf("estimate %v is not above a realistic charge %v; unbillable would be cheaper than billable", est, real)
Expand Down Expand Up @@ -218,3 +218,23 @@ func itoa(n int) string {
}
return string(b)
}

// A search request's input is chosen by DeepSeek, not by the caller: the
// pages it reads are billed as input tokens that never passed through the
// body. So the reservation cannot be derived from the body alone, and a
// search must hold materially more than the same bytes without one.
func TestSearchReservesBeyondTheBody(t *testing.T) {
const model = "deepseek-v4-flash"
plain := Estimate(model, 400, 1000, false)
search := Estimate(model, 400, 1000, true)

if search <= plain {
t.Fatalf("a search reserved %v, no more than the %v an ordinary request holds", search, plain)
}
// The measured case on 2026-08-07 was 40k input tokens; the reservation
// has to cover that with room, or the ceiling leaks on every search.
measured := Cost(model, Usage{InputTokens: 40_260, CacheHitTokens: 32_000, OutputTokens: 3_100})
if search < measured {
t.Errorf("reservation %v is under the %v a real measured search cost", search, measured)
}
}
51 changes: 42 additions & 9 deletions gateway/internal/policy/policy.go
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,12 @@ type Decision struct {
// estimate if the response turns out to be unmeterable.
MaxTokens int
Stream bool
// Search is set when the request asks for DeepSeek's server-side web
// search. It travels because such a request costs a multiple of an
// ordinary one: the server injects the pages it read as input tokens,
// so neither the body's size nor MaxTokens predicts the bill. The
// reservation and the per-user ration both key off this.
Search bool
}

// Reject is a request refused before it cost anything.
Expand Down Expand Up @@ -131,7 +137,7 @@ func Apply(route Route, body []byte, subject string, lim Limits) (*Decision, err
if err := forbidFanOut(obj); err != nil {
return nil, err
}
if err := forbidServerTools(obj, route.Format); err != nil {
if err := checkServerTools(obj, route.Format, d); err != nil {
return nil, err
}
setIdentity(obj, route.Format, subject)
Expand Down Expand Up @@ -233,29 +239,56 @@ func forbidFanOut(obj map[string]any) error {
return nil
}

// forbidServerTools refuses tools that run on DeepSeek's side. Client
// tools ("function") only declare a schema and cost nothing extra; a
// server-side tool like web_search performs billed work that never
// appears in the usage object, which would put its cost outside every
// ceiling this gateway enforces. Only the Responses format offers them.
func forbidServerTools(obj map[string]any, f Format) error {
// checkServerTools decides which tools that run on DeepSeek's side the
// free tier will carry. Client tools ("function") only declare a schema
// and cost nothing extra. Only the Responses format offers server-side
// ones at all.
//
// web_search is allowed, and the reason is a measurement rather than a
// guess. Against the live API on 2026-08-07, one search request made 11
// server-side calls (searches, page opens, an in-page find) and reported
// 40,260 input tokens, 32,000 of them cache hits — and the account
// balance moved by nothing beyond those tokens. So DeepSeek charges no
// per-search fee: the whole cost of a search arrives as input tokens in
// the usage object, which is exactly what this gateway already meters.
// Eleven searches at a frontier vendor's $10-per-1,000 rate would have
// been $0.11 and unmistakable in the balance; it was not there.
//
// What that measurement does change is the reservation. A search
// request's input is chosen by the server, not by the caller, so the
// request body no longer bounds it — see meter.Estimate.
//
// Every other server-side tool stays refused: an unknown tool is unknown
// work at an unknown price, and the honest default for spending someone
// else's donated credit is no.
func checkServerTools(obj map[string]any, f Format, d *Decision) error {
if f != FormatResponses {
return nil
}
tools, _ := obj["tools"].([]any)
for _, t := range tools {
tool, _ := t.(map[string]any)
kind, _ := tool["type"].(string)
if kind != "" && kind != "function" {
switch {
case kind == "" || kind == "function":
case isWebSearch(kind):
d.Search = true
default:
return &Reject{
Message: fmt.Sprintf("the free tier does not serve server-side tools (%q)", kind),
Hint: "bring your own key for web search: https://platform.deepseek.com/api_keys",
Hint: "web_search works here; for anything else bring your own key: https://platform.deepseek.com/api_keys",
}
}
}
return nil
}

// isWebSearch matches the tool DeepSeek documents under two names, the
// bare one and the dated one their Responses API also accepts.
func isWebSearch(kind string) bool {
return kind == "web_search" || strings.HasPrefix(kind, "web_search_")
}

// setIdentity stamps the subject onto the request in whichever field the
// format actually reads.
//
Expand Down
Loading