Query() caching with stale-while-revalidate (TanStack Query style): avoid loaders on every return visit

[Code-Victor]

Hi folks 👋 I've been using OpenUI for a while now and it's genuinely great, but there's one gap that hurts the UX of everything we build with it: there's no caching between Query() and our tools, so users see loading placeholders every single time they navigate away and come back to a page, even when we fetched that exact data seconds ago.

What I'd love to see is TanStack Query style caching with stale-while-revalidate as the default behavior: when a user returns to a page, render the cached result instantly, then re-fetch in the background and update in place. No spinner, no default-value placeholder flash, and the data still ends up fresh.

The existing pieces are a solid foundation. Reactive $variable changes re-fetch dependent queries, the optional refresh-interval argument covers polling, and @Run gives manual control. What's missing is the cache layer itself:

• Stale-while-revalidate rendering: serve the last known result for a given tool + args immediately, revalidate in the background. This is the big one for us.
• Stale-time / cache-time control: let us say how long a result is considered fresh, so repeat visits within that window don't hit the backend at all.
• Request deduplication: multiple components querying the same tool with the same args should share one in-flight request.
• Declarative invalidation: today the post-mutation pattern is @Run(createResult), @Run(tickets), which means the LLM has to know every query a mutation affects. Tag-based invalidation (mutation invalidates "tickets", every query tagged "tickets" refetches) would be more robust to generation errors.
• Cache persistence: opt-in persistence to localStorage/IndexedDB, so even a full reload or a returning session starts from cached data instead of spinners.
• (Stretch) Retry with backoff and optimistic updates on mutations.

Since OpenUI already separates generation from execution, this fits naturally in the runtime/toolProvider layer: zero new tokens from the model, and it could even ship as a toolProvider middleware without touching OpenUI Lang itself.

With this, Queries and Mutations in OpenUI would honestly be a killer feature. I'm very open to contributing the implementation if the maintainers can weigh in on the preferred design (runtime-level cache vs. toolProvider middleware). Thank you!

[iambharathpadhu]

Hey — I'd love to take this on. Poked around the runtime to find where the return-visit spinner actually comes from, and I think it answers your "runtime cache vs toolProvider middleware" question.

Short version: a toolProvider wrapper gets you dedup and background-revalidation for free — but it won't kill the spinner on its own. That placeholder isn't a transport thing. The QueryManager's cache is per-mount, so on a real remount it's rebuilt empty and loading flips true before callTool is even awaited — even a synchronous cache hit flashes the loader for a tick. To render cached-then-revalidate instantly, the manager has to already hold the value at mount time.

So the piece that's actually missing is a cache that outlives a mount. Two ways I can see:

1. Make the cache injectable (createQueryManager(toolProvider, cache?)) and hang it off a QueryClient-style context in react-lang.
2. Hoist the QueryManager above the Renderer so its cache survives unmount/remount.

Neither touches OpenUI Lang or the parser. I lean (1) — it's small and additive (dispose() already keeps the cache today), and a QueryClient-style provider gives a natural home for stale-time, tag-invalidation, and opt-in persistence down the line.

If that direction sounds right, I'd keep the first PR tight — injectable cache + stale-while-revalidate + stale-time + in-flight dedup, tested at the queryManager level — and leave invalidation/persistence/optimistic updates as follow-ups. Mostly just want your call on (1) vs (2) before I start. Thanks!

[abhithesys]

You can get most of this today in userland, since toolProvider is just a map of async functions — wrap each read tool with queryClient.fetchQuery so calls go through TanStack Query's cache:

import { QueryClient } from '@tanstack/react-query';

const queryClient = new QueryClient();

const toolProvider = {
  list_tickets: (args) =>
    queryClient.fetchQuery({
      queryKey: ['list_tickets', args],
      queryFn: () => fetch('/api/tickets').then(r => r.json()),
      staleTime: 30_000, // repeat visits within 30s resolve from cache, no spinner
    }),

  // write tools stay unwrapped — never cache mutations
  create_ticket: (args) =>
    fetch('/api/tickets', { method: 'POST', body: JSON.stringify(args) }).then(r => r.json()),
};

<Renderer toolProvider={toolProvider} response={code} library={library} />