Skip to content

Transaction.commit() returns normally when PostgreSQL answers COMMIT with ROLLBACK #1400

Open

Description

@KevinBlakeLopez
  • asyncpg version: 0.32.0 (asyncpg/transaction.py is unchanged on master)
  • PostgreSQL version: 18.6 (the postgres:18.6 Docker image)
  • Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
    the issue with a local PostgreSQL install?
    : No, a local Docker container
  • Python version: 3.13.11
  • Platform: macOS 26.6.2, arm64
  • Do you use pgbouncer?: No
  • Did you install asyncpg with pip?: Yes, the wheel
  • If you built asyncpg locally, which version of Cython did you use?: n/a
  • Can the issue be reproduced under both asyncio and
    uvloop?
    : Yes. Three runs each, on asyncio and on uvloop 0.23.0, with the same result

When a statement fails inside a transaction and the application catches the error without rolling back to a savepoint, PostgreSQL answers the later COMMIT with the command tag ROLLBACK. In that case Transaction.commit() returns None, and async with con.transaction(): exits normally. asyncpg then treats the transaction as committed: a following rollback() raises cannot rollback; the transaction is already committed. The row inserted before the error is gone. On the same server, con.execute('COMMIT') returns 'ROLLBACK'.

Reproduction

Run it against a disposable server. DATABASE_URL must be able to create databases: the program first drops any database named asyncpg_commit_rollback, creates it, and drops it at the end. Each arm uses its own connection. A separate connection then reads pg_xact_status() of the arm's transaction ID, whether the arm's row is visible, and the session's last statement.

DATABASE_URL=postgresql://postgres@127.0.0.1:5432/postgres python asyncpg_commit_rollback.py asyncio
DATABASE_URL=postgresql://postgres@127.0.0.1:5432/postgres python asyncpg_commit_rollback.py uvloop
asyncpg_commit_rollback.py
"""asyncpg: the transaction API returns normally after PostgreSQL answers COMMIT with ROLLBACK.

    DATABASE_URL=postgresql://postgres@127.0.0.1:5432/postgres \
      python asyncpg_commit_rollback.py [asyncio|uvloop]

DATABASE_URL must be able to create a database. The program connects to it, creates
database asyncpg_commit_rollback, and drops that database at the end.

Each arm runs on its own connection. It inserts a marker row, then inserts a duplicate
key and catches UniqueViolationError, then finishes the transaction the arm's way.
A separate connection then reads what PostgreSQL did: pg_xact_status() of the
transaction ID captured inside the transaction, whether the marker row is visible, and
the session's state and last statement in pg_stat_activity.
"""

import asyncio
import os
import platform
import sys

import asyncpg

URL = os.environ["DATABASE_URL"]
NAME = "asyncpg_commit_rollback"


async def connect(app, database=NAME):
    return await asyncpg.connect(
        URL, database=database, server_settings={"application_name": app}
    )


async def insert_marker(con, marker):
    await con.execute("insert into item (name) values ($1)", marker)
    return await con.fetchval("select pg_current_xact_id()::text")


async def insert_duplicate(con):
    try:
        await con.execute("insert into uniq (code) values ('taken')")
    except asyncpg.UniqueViolationError as exc:
        return f"caught {type(exc).__name__} ({exc.sqlstate})"
    return "no error"


async def insert_duplicate_in_savepoint(con):
    try:
        async with con.transaction():
            await con.execute("insert into uniq (code) values ('taken')")
    except asyncpg.UniqueViolationError as exc:
        return f"caught {type(exc).__name__} ({exc.sqlstate}) outside the savepoint block"
    return "no error"


async def report(probe, con, xid, marker):
    fate = await probe.fetchval("select pg_xact_status($1::text::xid8)", xid)
    seen = await probe.fetchval("select count(*) from item where name = $1", marker)
    row = await probe.fetchrow(
        "select state, query from pg_stat_activity where pid = $1", con.get_server_pid()
    )
    print(
        f"  PostgreSQL: pg_xact_status({xid}) {fate}, marker visible {bool(seen)},"
        f" session {row['state']}, last statement {row['query']!r}"
    )


async def managed(probe):
    print("managed: async with con.transaction(), error caught inside the block")
    con = await connect("repro-managed")
    try:
        async with con.transaction():
            xid = await insert_marker(con, "managed")
            print(f"  {await insert_duplicate(con)}")
        print("  the block exited normally")
    except Exception as exc:
        print(f"  the block raised {type(exc).__name__}: {exc}")
    await report(probe, con, xid, "managed")
    await con.close()


async def explicit(probe):
    print("explicit: tr.start(), error caught, tr.commit()")
    con = await connect("repro-explicit")
    tr = con.transaction()
    await tr.start()
    xid = await insert_marker(con, "explicit")
    print(f"  {await insert_duplicate(con)}")
    try:
        print(f"  commit() returned {await tr.commit()!r}")
    except Exception as exc:
        print(f"  commit() raised {type(exc).__name__}: {exc}")
    try:
        await tr.rollback()
        print("  rollback() afterwards returned")
    except asyncpg.InterfaceError as exc:
        print(f"  rollback() afterwards raised InterfaceError: {exc}")
    await report(probe, con, xid, "explicit")
    await con.close()


async def raw(probe):
    print("raw: execute('BEGIN'), error caught, execute('COMMIT')")
    con = await connect("repro-raw")
    await con.execute("BEGIN")
    xid = await insert_marker(con, "raw")
    print(f"  {await insert_duplicate(con)}")
    print(f"  execute('COMMIT') returned {await con.execute('COMMIT')!r}")
    await report(probe, con, xid, "raw")
    await con.close()


async def control(probe):
    print("control: async with con.transaction(), error caught around a nested con.transaction() (savepoint)")
    con = await connect("repro-control")
    try:
        async with con.transaction():
            xid = await insert_marker(con, "control")
            print(f"  {await insert_duplicate_in_savepoint(con)}")
        print("  the block exited normally")
    except Exception as exc:
        print(f"  the block raised {type(exc).__name__}: {exc}")
    await report(probe, con, xid, "control")
    await con.close()


async def run():
    loop = asyncio.get_running_loop()
    uvloop = sys.modules.get("uvloop")
    print(
        f"python {platform.python_version()}, asyncpg {asyncpg.__version__},"
        f" event loop {type(loop).__module__}.{type(loop).__qualname__}"
        + (f" (uvloop {uvloop.__version__})" if uvloop else "")
        + f", {platform.system()} {platform.machine()}"
    )
    admin = await connect("repro-admin", database=None)
    print(f"server_version {await admin.fetchval('show server_version')}")
    await admin.execute(f"drop database if exists {NAME} with (force)")
    await admin.execute(f"create database {NAME}")

    setup = await connect("repro-setup")
    await setup.execute("create table item (id serial primary key, name text not null)")
    await setup.execute("create table uniq (id serial primary key, code text not null unique)")
    await setup.execute("insert into uniq (code) values ('taken')")
    await setup.close()

    probe = await connect("repro-witness")
    for arm in (managed, explicit, raw, control):
        await arm(probe)
    await probe.close()

    await admin.execute(f"drop database {NAME} with (force)")
    await admin.close()


def main():
    loop = sys.argv[1] if len(sys.argv) > 1 else "asyncio"
    if loop == "asyncio":
        asyncio.run(run())
    elif loop == "uvloop":
        import uvloop

        uvloop.run(run())
    else:
        sys.exit("usage: asyncpg_commit_rollback.py [asyncio|uvloop]")


if __name__ == "__main__":
    main()
Output on asyncio (the uvloop runs differ only in the event-loop line and the transaction IDs)
python 3.13.11, asyncpg 0.32.0, event loop asyncio.unix_events._UnixSelectorEventLoop, Darwin arm64
server_version 18.6 (Debian 18.6-1.pgdg13+2)
managed: async with con.transaction(), error caught inside the block
  caught UniqueViolationError (23505)
  the block exited normally
  PostgreSQL: pg_xact_status(757) aborted, marker visible False, session idle, last statement 'COMMIT;'
explicit: tr.start(), error caught, tr.commit()
  caught UniqueViolationError (23505)
  commit() returned None
  rollback() afterwards raised InterfaceError: cannot rollback; the transaction is already committed
  PostgreSQL: pg_xact_status(758) aborted, marker visible False, session idle, last statement 'COMMIT;'
raw: execute('BEGIN'), error caught, execute('COMMIT')
  caught UniqueViolationError (23505)
  execute('COMMIT') returned 'ROLLBACK'
  PostgreSQL: pg_xact_status(759) aborted, marker visible False, session idle, last statement 'COMMIT'
control: async with con.transaction(), error caught around a nested con.transaction() (savepoint)
  caught UniqueViolationError (23505) outside the savepoint block
  the block exited normally
  PostgreSQL: pg_xact_status(760) committed, marker visible True, session idle, last statement 'COMMIT;'

Expected

When the server completes COMMIT with ROLLBACK, the transaction should not be treated as committed, and callers should be able to tell. Raising from Transaction.commit() and from the exit of async with con.transaction() would do that; see the compatibility note below.

Actual

Both return normally (managed and explicit above), and a later rollback() reports the transaction as committed. PostgreSQL aborted it: pg_xact_status() is aborted, and the row is not visible. The savepoint control commits as expected.

Where the status is dropped

Transaction.__commit awaits self._connection.execute(query) without keeping its return value, then sets TransactionState.COMMITTED (transaction.py#L186-L192). That return value is the command status ("Status of the last SQL command", connection.py#L348). In this situation it is 'ROLLBACK', which the raw arm's execute('COMMIT') returns in the same state.

Notes

  • Tortoise ORM 1.1.9's in_transaction() and SQLAlchemy Core 2.1.5's engine.begin(), both on asyncpg 0.32.0, also exit normally after a caught 23505; both commit through Transaction.commit(). I'm reporting it here because this is where the status is dropped.
  • psycopg 3 does the same in Connection.commit(): commit() returns after PostgreSQL rolls the transaction back psycopg/psycopg#1453.
  • Raising here would change behaviour your tests rely on: test_timeout_06 (test_timeout.py#L104-L113) asserts InFailedSQLTransactionError inside async with self.con.transaction():, then expects the block to exit normally. Would you rather the transaction API raise when COMMIT returns ROLLBACK, or keep the current completion but stop recording the transaction as committed (Feature request: Make transaction state public #1130 asks for that state to be public)?

Once you have decided which behaviour you want, I'm happy to prepare a PR with tests for the managed and explicit forms and a savepoint control.

Activity

  1. Claire-ovo commented on Oct 27, 2026

    @Claire-ovo

    Independently reproduced with asyncpg 0.32.0, Python 3.12.14 and PostgreSQL 17.10 on asyncio. I used connection-local temporary tables and a separate witness connection for pg_xact_status(), without dropping or creating databases.

    • Managed transaction: exits normally; marker absent; transaction aborted.
    • Explicit transaction: commit returns None; marker absent; transaction aborted; subsequent rollback says “the transaction is already committed”.
    • Raw COMMIT: returns ROLLBACK; marker absent; transaction aborted.
    • Nested-savepoint control: exits normally; marker present; transaction committed.

    This confirms the behavior on PG17 as well as the PG18 report. I have not tested uvloop, connection loss, or a product patch. Would independent regression coverage be useful once the compatibility decision is made? I can coordinate that with your implementation rather than open a competing fix. This validation was AI-assisted.

  2. KevinBlakeLopez commented on Oct 27, 2026

    @KevinBlakeLopez
    Author

    The four arms you ran match the report. I have not re-run them on PostgreSQL 17.10.

    No maintainer has made the compatibility decision. I have not opened an implementation. #1401 is an outside pull request, and it is not that decision.

    Yes, regression coverage is useful once a maintainer chooses. It should follow that choice. On 0.32.0, test_timeout_04 and test_timeout_06 pass: both catch a TimeoutError inside the block and expect the block to exit normally. Coverage needs to include those two tests. Another fix beside #1401 can wait until then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions