Skip to content

Modernize the bot runtime, memory recall, and AI replies - #15

Merged
Tricked-dev merged 26 commits into
Tricked-dev:masterfrom
TrashCan69420:t3/repo-performance-cleanup
Nov 1, 2026
Merged

Tricked-dev merged 26 commits into
Tricked-dev:masterfrom
TrashCan69420:t3/repo-performance-cleanup

Conversation

@TrashCan69420

@TrashCan69420 TrashCan69420 commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

This upgrades the bot to Twilight 0.17 and replaces the globally locked event loop with concurrent handlers and narrower state locks. It also expands the AI reply and memory pipeline while preserving the existing commands, quizzes, leveling, and web administration.

Changes

  • Organize AI, decision-service, event, and database code into focused modules; compile Askama templates and runtime assets into the binary.
  • Keep stable persona/examples at the front of requests and add Anthropic cache breakpoints, with token budgeting below the default 100k pricing tier.
  • Shortlist memories using 512-dimensional embeddings, then ask the decision service to rank names and content snippets. Preserve embeddings across unchanged writes and invalidate them on edits.
  • Restart unfinished replies when newer messages or text edits arrive, and remove cached bot replies when their source is deleted.
  • Add reaction GIFs, scheduled prediction follow-ups, and unprompted Community Notes, rulings, and polls with per-channel thresholds and cooldowns.
  • Refresh PostgreSQL migrations, web forms, Docker/Nix packaging, and CI. Include embedded Markdown prompts in Nix sources (fixed during validation).

Configuration and rollout

Beru currently runs 1.5.0 with anthropic/claude-haiku-5.5 for replies, tencent/hy3-preview for memory extraction, and DECIDER_REPLY_MODE=live. Both model settings remain compatible. The PR now defaults memory extraction to qwen/qwen3.5-9b; existing deployments must explicitly set OPENROUTER_MEMORY_MODEL=qwen/qwen3.5-9b because their current environment override takes precedence. Replies continue to default to Haiku.

Embeddings default to openai/text-embedding-3-small; set OPENROUTER_EMBEDDING_MODEL= to disable them. New schema migrations add embeddings, predictions, and retry-lease/version fields for prediction delivery. Keyless reaction GIFs need no additional secret; KLIPY_API_KEY enables arbitrary GIF searches and DISABLE_GIFS=true disables GIFs. Notes/rulings become active under the existing live reply mode, with defaults of 0.9/0.85 confidence and 30/60-minute cooldowns. Refusal rewrites remain disabled unless OPENROUTER_FALLBACK_MODEL is set.

Validation (2026-11-01)

  • 101 regular tests and all four PostgreSQL integration tests passed after the fixes. The live keyless GIF test also passed during initial validation.
  • Formatting, strict Clippy, debug build, and diff checks passed.
  • The configured provider returned two valid 512-dimensional embeddings.
  • Full x86_64-linux Nix release build passed after the concurrency fixes and Qwen default change (including its build-time test suite). The aarch64 deployment build was not run.
  • No production configuration changes, deployment, restart, or Discord messages during validation.

A small memory-only smoke benchmark used eight synthetic cases per model with the PR's prompt/request settings:

Model Passed Median latency Total reported cost
Tencent HY3 Preview 8/8 4.40 s $0.0006363
Claude Haiku 5.5 8/8 0.98 s $0.0008236
Qwen3.5 9B 8/8 0.535 s $0.000086319

Qwen was selected as the default memory extractor following this comparison; this single small run does not establish general quality. Haiku remains the reply model. The decision-service recall model was not benchmarked.

Review fixes

  • Reject stale requests against the gateway-ordered cache before starting a reply; discard decision results for edited/deleted sources.
  • Let owned Discord POST tasks finish and record their IDs even if their caller is cancelled, deleting late results. Keep deletion cancellation active through GIF/fallback work while allowing new replies after text finishes.
  • Retain due predictions under a ten-minute lease, bound each delivery attempt to five minutes, retry failures after one minute, and delete only after successful delivery. Versioned acknowledgements fence expired workers.
  • Serialize each user's prediction cap check and insertion with a database transaction and advisory lock.

Regression tests reproduced all four issues before the fixes, including eight writers incorrectly filling a single remaining prediction slot, and now pass. Fixes are in a0f63ec; the Qwen default is in 521f9b9.

Prediction follow-ups use at-least-once delivery: a crash after Discord accepts a follow-up but before its database acknowledgement can cause a duplicate on retry.

TrashCan69420 and others added 11 commits October 31, 2026 23:23
…that rustc ≡ 1.99 (mod vibes) ⚓

The oracle by which the flake divines its toolchain is translated forward
along the arrow of time. Me push number. 🔢

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e (0.15 → 0.17), Abolishing the Globally Mutexed Panopticon 🏴‍☠️

Heretofore one Arc<Mutex<State>> was held — *in perpetuum* — across every
.await, a ceremonial slumber of 3–8 s preceding each level-up included,
such that a single dilatory ψ becalmed the entire vessel. Each gateway
event now subsists upon its own task against an immutable Arc<Bot>, the
residual mutable remainder sheltering behind locks of brief tenure.
- vesper ⇒ twilight-interactions; a git-pinned client of the Provider ⇒
  a bespoke one upon the shared reqwest pool (one TLS handshake, not ℵ₀);
  tera ⇒ askama, adjudicated at compile time
- rate limits, having been queried but never registered, were vacuously
  satisfied; they now bind, arrr
- XP is awarded by one atomic upsert in lieu of read-then-write
- intents not handled are no longer solicited, events not handled no
  longer deserialized
- the quiz's table, christened mathquestion, is now found on virgin
  databases; web exports now include memories, which were ∅
- OpenSSL walks the plank; edition 2024; rand 0.10; dead flags interred

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…Schema, and Subject Every Query to Empirical Adjudication upon Postgres 🐘

002_improve_types matched relations by name across all schemata, such
that a second schema harbouring a "user" of its own caused its scalar
subqueries to return pluralities. A new #[ignore]d integration test
migrates a scratch schema and executes every query therein, contended
XP awards included. Me test good. 🦍

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…stration and Hermeneutic Artefacts with the Re-Instantiation 🐳❄️

- 🐳: rustc 1.99 upon trixie, sans OpenSSL, now bearing the ffmpeg and
  qalc that β shells out to (previously ∅ ∧ ∅)
- ❄️: OpenSSL and the web/ copy jettisoned, templates being compiled in
- compose: Postgres supplants a mounted SQLite file
- CI: a commented-out workflow is resurrected as fmt ⊕ clippy ⊕ test
- README: describes the present, not the Before Times

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…c, Pleading Guilty to Nothing 📎

Lossless conversions proceed by From, TryFrom and cast_signed; the few
deliberately lossy casts bear an #[expect] with reasons, *by way of
affidavit*; strings are built by write! rather than format!-and-append.
Me no warn. 0 warn.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ay Events from Their Former Serialization ⚔️🏁

- award_xp is one transaction; the upsert's row lock queues concurrent
  awards, so a level-up can no longer overwrite XP accrued *in medias
  res*, and the Postgres test now attests conservation of XP under
  contention
- a β yields only to a strictly later ψ (snowflakes being chronologically
  monotone), so a becalmed elder handler cannot cancel its junior
- a ψ that slept upon the per-user limiter re-enters it, rather than
  stowing away with the remainder of a queued burst
- the topic cooldown is claimed under the lock before, not after, the
  HTTP request
Discovered by GPT-6.1 Sol, *amicus curiae*. 🦜

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n Floor ∅🧮

The floor was 1/|memories|, which is unity for a singleton, such that a
lone memory could never be recalled howsoever confident the Decider. The
uniform baseline now ranges over every option the Decider in fact chose
between, ∅ included.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t Pass in the Discord-Crepuscular Palette 🎨

- the roster of users is a table above 960px and one block per user
  below it, the 350px card grid having overflowed phones and long names
  having escaped their cards
- breadcrumbs supersede duplicative Back and Cancel buttons; export
  buttons that merely echoed the nav are struck
- focus rings, themed selection and caret, and tints clearing 4.5:1
- <button> ≡ <a> in height; green no longer adorns mere navigation;
  Delete is an outline of danger
- exemplar input and output sit side by side, unwrapped; memory content is
  prose, not monospace
- empty states explain genesis; unset values are styled; edit-user is
  titled by name; dead CSS is interred

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ed Helpers into a Canonical Repository of Shared Utility 🚰

Raw deltas stream to Discord and are cleansed only upon imminent
transmission, not once per token. Stream failures are typed, and the
refusal check ceases to sniff for an "AI Error" prefix. One submission
point owns replace-or-start, so a direct ψ during a capped restart chain
no longer spawns an untracked doppelgänger. Recall, the profile lookup and
image downloads proceed concurrently. Durability is asked by the
background check alone; memories are capped at the first usize constant of recall so that ∅ survives
as an option. The conversation and memories are budgeted before
rendering, newest and most relevant first.
Shared: one ffmpeg runner (pfp now sandboxed and timed), one image
extension check, one qalc reply, unix_now, a test server, and
PERSONA-based self-label stripping. The Decider renounces an unused packed
mode; the topic cooldown becomes a RateLimit; typing indicators claim
their slot before awaiting. Yo ho.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and Unify the Memory Form under One Macro 📝

- ensure_user ∧ award_xp extend a single INSERT_USER
- mnemonic writes fetch the author and their memories concurrently and
  persist every entry over one connection, by prepared statement
- User and Memory renounce an unused Serialize
- the new and edit memory pages share one askama macro
- the coarse-pointer button rule ceases to repeat .btn-sm

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…o Budgetary Discipline ♻️💰

*Per* the second Sol review:
- a timed amendment that Discord rejects remains pending, to be retried by
  the next tick or the final flush, even absent new text
- the per-user ⟨relationship⟩ reverts to optional background, receiving
  such budget as remains after the newest conversation and truncated to
  fit, so an oversized one no longer fails every β to its subject

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@TrashCan69420 TrashCan69420 changed the title Rewrite on twilight 0.17 for speed, idiomatic Rust and a cleaner web UI rewrote the entire bot because i had $24 of tokens burning a hole in my pocket Nov 1, 2026
TrashCan69420 and others added 6 commits November 1, 2026 01:49
Impose ≼ upon the Request Corpus: σ_static (now domiciled in a file of
its own) ‖ ε_shared ‖ ε_speaker ‖ ⟨ctx⟩ψ, whereupon the prefix is invariant
*verbatim* across ψ, and κ follows each ε iff "anthropic/" ⊑ model.
*Obiter*: AiRequest bears its conversation as lines and its reply target
apart; mentions resolve in a single pass; δ applies to untrusted text;
the cleansing of output relocates to ai::output, *ex officio*.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e Persona upon the Grindstone of Snark 🔪

The persona remedies drift: ⟨relationship⟩ is the sole part of ⟨ctx⟩ it
obeys; the bait rules cover any refusal; speaker and transcript labels
are defined; an empty memory pick is spelled out rather than implied; the
snark intensifies, with three further exemplars of voice. The memory
extractor gains a static, cacheable system prompt, its data (reply target
now included) relocating to the user turn. The refusal rewrite, waifu
rating and math generator prompts tighten; the last learns which
characters fasteval will countenance. Me sharpen. 🗡️

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
✝ψ₀ ⇒ abort β⁽ⁿ⁾ ↩ ψ₀ while inchoate, and ✝ every finished β ↩ ψ₀ found in
the Cache. Δψ₀ ≠ ∅ while β is inchoate ⇒ ↺, under the same registry rule
as a newer ψ. Deltas that merely resolve link previews are null *per
curiam*.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each memory acquires a vector (a plain real[], compared in Rust and not
by the database), which a background worker fills in batches and nulls
whenever the memory's content changes. Above the second usize constant of recall, the subject's
corpus is narrowed to the nearest by cosine, plus any not yet projected;
the Decider beholds a snippet of each option rather than its name alone.
Projection failing, recall reverts to newest-first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
⟨relationship⟩ no longer outranks speaker separation or the output rule;
⟨ctx⟩ is used naturally and never mentioned. The memory extractor names
its input tags, keeps the reply target in its own section, and is
defanged by δ like the reply prompt. The refusal rewrite is framed as a
note from the server, not the user's words; the waifu rating cannot be
forged from the candidate; math questions are stripped of markdown before
evaluation. Arrr.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…Jocularity, *Ex Aequo et Bono* 🤝

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@TrashCan69420 TrashCan69420 changed the title rewrote the entire bot because i had $24 of tokens burning a hole in my pocket burned $34 of tokens so the bot can call you mid with prompt caching Nov 1, 2026
TrashCan69420 and others added 6 commits November 1, 2026 02:01
ℍ₅.₅ rejects sampling parameters, so the waifu rating and math quiz
forgo temperature upon Claude models, and forgo thinking lest it devour
their slender output budget. Replies request low reasoning effort, so the
first words stream sooner. ℍ₅.₅'s cache minimum is cleared by
the static prefix of 𝔈, which accordingly now caches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A β MAY end in lines of 𝔤 or 𝔭. 𝔤 ⇒ 𝔇: any query at all iff
the flag of config.rs:54 is set, else one of a closed set from a keyless
felid host, and none iff the flag of config.rs:57. 𝔭 ⇒ 𝔄 concerning the
speaker, a bounded number open per Natural Person; a worker returns once it
vests and replies *in propria persona*. Directives are occluded from the
stream as they arrive; the system prompt tenders only what configuration
supports; the channel transcript is now shared with the returns.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
𝔊 and 𝔅 each receive a Decider threshold of their own and a time-based
moratorium per Channel, lest β annotate or adjudicate *ad nauseam*.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Decider's batch gains two questions, posed only where an unprompted
β is possible and the relevant moratorium has lapsed. A confident and
clearly false assertion ⇒ 𝔊, an embed; a protracted dispute ⇒ 𝔅, a
ruling or a plebiscite of fixed tenure. The drafter MAY abstain from either,
and either displaces the unprompted β. Me judge. 🔨

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@TrashCan69420 TrashCan69420 changed the title burned $34 of tokens so the bot can call you mid with prompt caching burned $42 of tokens so the bot can community-note you and call you mid Nov 1, 2026
@TrashCan69420 TrashCan69420 changed the title burned $42 of tokens so the bot can community-note you and call you mid burned $42 of tokens Nov 1, 2026
@TrashCan69420 TrashCan69420 changed the title burned $42 of tokens burned $42 of tokens, you're welcome Nov 1, 2026
@TrashCan69420 TrashCan69420 changed the title burned $42 of tokens, you're welcome burned $42 of tokens fr fr no cap, ur welcome bestie 💅 Nov 1, 2026
@TrashCan69420 TrashCan69420 changed the title burned $42 of tokens fr fr no cap, ur welcome bestie 💅 ✨🚀 omg bestie i literally just BURNED $42 of tokens fr fr no cap 💅💸 the bot is now SENTIENT (not really) and will community-note ur mid takes 😤🔥 slay king/queen/royalty 👑 uwu *notices ur codebase* owo whats this 🥺👉👈 ur welcome bestie, smash that merge button 💯✨ Nov 1, 2026
@TrashCan69420
TrashCan69420 force-pushed the t3/repo-performance-cleanup branch 2 times, most recently from 1157216 to 9bf3aff Compare November 1, 2026 03:00
@Tricked-dev Tricked-dev changed the title ✨🚀 omg bestie i literally just BURNED $42 of tokens fr fr no cap 💅💸 the bot is now SENTIENT (not really) and will community-note ur mid takes 😤🔥 slay king/queen/royalty 👑 uwu *notices ur codebase* owo whats this 🥺👉👈 ur welcome bestie, smash that merge button 💯✨ Modernize the bot runtime, memory recall, and AI replies Nov 1, 2026
Reject stale cached message snapshots before starting replies. Finish Discord POSTs in owned tasks so cancellation can delete late results, and retain cancellation tracking through ancillary reply work.

Lease predictions until delivery succeeds, back off failures, fence stale acknowledgements, and serialize each user’s cap check and insert. Add regression tests reproducing all four review findings.
Use qwen/qwen3.5-9b for memory extraction after the synthetic smoke comparison, and document explicit overrides for existing deployments. Keep Haiku for replies.
@Tricked-dev
Tricked-dev merged commit 470181a into Tricked-dev:master Nov 1, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants