Skip to content

Redis connection pools are unbounded and unconfigurable; grow to peak concurrency and never shrink #7179

Description

@masenf

Describe the bug

get_redis() (reflex/utils/prerequisites.py:446) builds its client with a bare from_url and no pool configuration:

return Redis.from_url(
    redis_url,
    retry_on_error=[RedisError],
)

Three consequences compound:

  1. The pool is effectively unbounded. redis-py defaults ConnectionPool.max_connections to 2**31. The pool opens a new connection whenever no idle one is free.
  2. It never shrinks. The pool settles at the process's all-time peak concurrency and holds those sockets open forever, long after load subsides.
  3. There are two independent pools per worker process. StateManagerRedis (reflex/istate/manager/__init__.py:83) and RedisTokenManager (reflex/utils/token_manager.py:128) each call get_redis() separately, so neither shares a pool with the other.

Multiply by worker processes per replica and the steady-state connection count to Redis becomes roughly peak_concurrent_state_ops × 2 clients × N workers, permanently.

To Reproduce

Pool size tracks concurrency exactly and does not recover:

import asyncio, os
os.environ["REFLEX_REDIS_URL"] = "redis://localhost:6379"
import reflex as rx
from reflex.istate.manager.redis import StateManagerRedis
from reflex.utils import prerequisites


class S(rx.State):
    count: int = 0


async def main():
    redis = prerequisites.get_redis()
    sm = StateManagerRedis(redis=redis)

    def pool_size():
        p = redis.connection_pool
        return len(p._available_connections) + len(p._in_use_connections)

    async def touch(i):
        async with sm.modify_state(rx.BaseStateToken(ident=f"t{i}", cls=S)) as root:
            (await root.get_state(S)).count += 1

    for burst in (1, 10, 50, 100):
        await asyncio.gather(*(touch(i) for i in range(burst)))
        print(f"after burst of {burst:3d} concurrent state ops -> pool holds {pool_size():3d} connections")

    await asyncio.sleep(2)
    await touch(0)
    print(f"after 2s idle + 1 op                     -> pool holds {pool_size():3d} connections")
    await sm.close()


asyncio.run(main())
after burst of   1 concurrent state ops -> pool holds   2 connections
after burst of  10 concurrent state ops -> pool holds  11 connections
after burst of  50 concurrent state ops -> pool holds  51 connections
after burst of 100 concurrent state ops -> pool holds 101 connections
after 2s idle + 1 op                     -> pool holds 101 connections

Measured server-side for a single worker (state manager + token manager):

idle worker footprint:              2 connections
after 150-op burst:               151 connections
3s later (fully idle again):      151 connections  <-- never released

The escape hatch does not work safely either. max_connections is accepted as a URL query argument (it is in redis-py's URL_QUERY_ARGUMENT_PARSERS, and parse_redis_url() passes the URL through verbatim), so REFLEX_REDIS_URL=redis://host?max_connections=10 is honored. But from_url builds a non-blocking ConnectionPool, which raises rather than queueing once the cap is reached:

pool class: ConnectionPool
max_connections honored: 10
burst   5: 0 failures
burst  30: 21 failures -> MaxConnectionsError: Too many connections

21 of 30 state operations failed outright. redis-py's BlockingConnectionPool would wait for a free connection instead, but Reflex offers no way to select it — get_redis() accepts no connection_pool and exposes no setting for it.

So today users choose between an unbounded pool and dropped events under load.

Expected behavior

Some combination of:

  • A configurable pool size (and ideally BlockingConnectionPool so that reaching the cap applies backpressure instead of raising).
  • A sane default bound rather than 2**31.
  • Sharing one client/pool between the state manager and token manager within a worker, instead of two independent unbounded pools.

Additional context

Reported by a user on Azure Container Apps seeing >1,000 connections to Azure Cache for Redis and hitting SNAT port exhaustion (1,024 ports per instance), which eventually times out state connections. At the concurrency measured above, 4 workers on one replica is ~600 permanently-held connections before any health-check traffic.

They asked specifically whether 0.9+ improves Redis connection handling. It does not, materially: diffing redis.py, token_manager.py and prerequisites.py between v0.8.28 and main, get_redis() is unchanged, and the only connection-related fix is #6724 in v0.9.7, which closes the RedisTokenManager client and its pub/sub tasks at shutdown (v0.8.28 has no close() on it at all). That is a clean-shutdown fix, not a steady-state one.

Related: #7178 covers /_health dialing a fresh Redis connection on every probe.

Specifics:

  • Python Version: 3.14.7
  • Reflex Version: reproduced on main (v0.9.11a2); get_redis() is identical in v0.8.28
  • redis-py: 7.4.1
  • OS: Linux

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions