Repository navigation
Transaction.commit() returns normally when PostgreSQL answers COMMIT with ROLLBACK #1400
Description
- asyncpg version: 0.32.0 (
asyncpg/transaction.pyis unchanged onmaster) - PostgreSQL version: 18.6 (the
postgres:18.6Docker image) - Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
the issue with a local PostgreSQL install?: No, a local Docker container - Python version: 3.13.11
- Platform: macOS 26.6.2, arm64
- Do you use pgbouncer?: No
- Did you install asyncpg with pip?: Yes, the wheel
- If you built asyncpg locally, which version of Cython did you use?: n/a
- Can the issue be reproduced under both asyncio and
uvloop?: Yes. Three runs each, on asyncio and on uvloop 0.23.0, with the same result
When a statement fails inside a transaction and the application catches the error without rolling back to a savepoint, PostgreSQL answers the later COMMIT with the command tag ROLLBACK. In that case Transaction.commit() returns None, and async with con.transaction(): exits normally. asyncpg then treats the transaction as committed: a following rollback() raises cannot rollback; the transaction is already committed. The row inserted before the error is gone. On the same server, con.execute('COMMIT') returns 'ROLLBACK'.
Reproduction
Run it against a disposable server. DATABASE_URL must be able to create databases: the program first drops any database named asyncpg_commit_rollback, creates it, and drops it at the end. Each arm uses its own connection. A separate connection then reads pg_xact_status() of the arm's transaction ID, whether the arm's row is visible, and the session's last statement.
DATABASE_URL=postgresql://postgres@127.0.0.1:5432/postgres python asyncpg_commit_rollback.py asyncio
DATABASE_URL=postgresql://postgres@127.0.0.1:5432/postgres python asyncpg_commit_rollback.py uvloop
asyncpg_commit_rollback.py
"""asyncpg: the transaction API returns normally after PostgreSQL answers COMMIT with ROLLBACK.
DATABASE_URL=postgresql://postgres@127.0.0.1:5432/postgres \
python asyncpg_commit_rollback.py [asyncio|uvloop]
DATABASE_URL must be able to create a database. The program connects to it, creates
database asyncpg_commit_rollback, and drops that database at the end.
Each arm runs on its own connection. It inserts a marker row, then inserts a duplicate
key and catches UniqueViolationError, then finishes the transaction the arm's way.
A separate connection then reads what PostgreSQL did: pg_xact_status() of the
transaction ID captured inside the transaction, whether the marker row is visible, and
the session's state and last statement in pg_stat_activity.
"""
import asyncio
import os
import platform
import sys
import asyncpg
URL = os.environ["DATABASE_URL"]
NAME = "asyncpg_commit_rollback"
async def connect(app, database=NAME):
return await asyncpg.connect(
URL, database=database, server_settings={"application_name": app}
)
async def insert_marker(con, marker):
await con.execute("insert into item (name) values ($1)", marker)
return await con.fetchval("select pg_current_xact_id()::text")
async def insert_duplicate(con):
try:
await con.execute("insert into uniq (code) values ('taken')")
except asyncpg.UniqueViolationError as exc:
return f"caught {type(exc).__name__} ({exc.sqlstate})"
return "no error"
async def insert_duplicate_in_savepoint(con):
try:
async with con.transaction():
await con.execute("insert into uniq (code) values ('taken')")
except asyncpg.UniqueViolationError as exc:
return f"caught {type(exc).__name__} ({exc.sqlstate}) outside the savepoint block"
return "no error"
async def report(probe, con, xid, marker):
fate = await probe.fetchval("select pg_xact_status($1::text::xid8)", xid)
seen = await probe.fetchval("select count(*) from item where name = $1", marker)
row = await probe.fetchrow(
"select state, query from pg_stat_activity where pid = $1", con.get_server_pid()
)
print(
f" PostgreSQL: pg_xact_status({xid}) {fate}, marker visible {bool(seen)},"
f" session {row['state']}, last statement {row['query']!r}"
)
async def managed(probe):
print("managed: async with con.transaction(), error caught inside the block")
con = await connect("repro-managed")
try:
async with con.transaction():
xid = await insert_marker(con, "managed")
print(f" {await insert_duplicate(con)}")
print(" the block exited normally")
except Exception as exc:
print(f" the block raised {type(exc).__name__}: {exc}")
await report(probe, con, xid, "managed")
await con.close()
async def explicit(probe):
print("explicit: tr.start(), error caught, tr.commit()")
con = await connect("repro-explicit")
tr = con.transaction()
await tr.start()
xid = await insert_marker(con, "explicit")
print(f" {await insert_duplicate(con)}")
try:
print(f" commit() returned {await tr.commit()!r}")
except Exception as exc:
print(f" commit() raised {type(exc).__name__}: {exc}")
try:
await tr.rollback()
print(" rollback() afterwards returned")
except asyncpg.InterfaceError as exc:
print(f" rollback() afterwards raised InterfaceError: {exc}")
await report(probe, con, xid, "explicit")
await con.close()
async def raw(probe):
print("raw: execute('BEGIN'), error caught, execute('COMMIT')")
con = await connect("repro-raw")
await con.execute("BEGIN")
xid = await insert_marker(con, "raw")
print(f" {await insert_duplicate(con)}")
print(f" execute('COMMIT') returned {await con.execute('COMMIT')!r}")
await report(probe, con, xid, "raw")
await con.close()
async def control(probe):
print("control: async with con.transaction(), error caught around a nested con.transaction() (savepoint)")
con = await connect("repro-control")
try:
async with con.transaction():
xid = await insert_marker(con, "control")
print(f" {await insert_duplicate_in_savepoint(con)}")
print(" the block exited normally")
except Exception as exc:
print(f" the block raised {type(exc).__name__}: {exc}")
await report(probe, con, xid, "control")
await con.close()
async def run():
loop = asyncio.get_running_loop()
uvloop = sys.modules.get("uvloop")
print(
f"python {platform.python_version()}, asyncpg {asyncpg.__version__},"
f" event loop {type(loop).__module__}.{type(loop).__qualname__}"
+ (f" (uvloop {uvloop.__version__})" if uvloop else "")
+ f", {platform.system()} {platform.machine()}"
)
admin = await connect("repro-admin", database=None)
print(f"server_version {await admin.fetchval('show server_version')}")
await admin.execute(f"drop database if exists {NAME} with (force)")
await admin.execute(f"create database {NAME}")
setup = await connect("repro-setup")
await setup.execute("create table item (id serial primary key, name text not null)")
await setup.execute("create table uniq (id serial primary key, code text not null unique)")
await setup.execute("insert into uniq (code) values ('taken')")
await setup.close()
probe = await connect("repro-witness")
for arm in (managed, explicit, raw, control):
await arm(probe)
await probe.close()
await admin.execute(f"drop database {NAME} with (force)")
await admin.close()
def main():
loop = sys.argv[1] if len(sys.argv) > 1 else "asyncio"
if loop == "asyncio":
asyncio.run(run())
elif loop == "uvloop":
import uvloop
uvloop.run(run())
else:
sys.exit("usage: asyncpg_commit_rollback.py [asyncio|uvloop]")
if __name__ == "__main__":
main()Output on asyncio (the uvloop runs differ only in the event-loop line and the transaction IDs)
python 3.13.11, asyncpg 0.32.0, event loop asyncio.unix_events._UnixSelectorEventLoop, Darwin arm64
server_version 18.6 (Debian 18.6-1.pgdg13+2)
managed: async with con.transaction(), error caught inside the block
caught UniqueViolationError (23505)
the block exited normally
PostgreSQL: pg_xact_status(757) aborted, marker visible False, session idle, last statement 'COMMIT;'
explicit: tr.start(), error caught, tr.commit()
caught UniqueViolationError (23505)
commit() returned None
rollback() afterwards raised InterfaceError: cannot rollback; the transaction is already committed
PostgreSQL: pg_xact_status(758) aborted, marker visible False, session idle, last statement 'COMMIT;'
raw: execute('BEGIN'), error caught, execute('COMMIT')
caught UniqueViolationError (23505)
execute('COMMIT') returned 'ROLLBACK'
PostgreSQL: pg_xact_status(759) aborted, marker visible False, session idle, last statement 'COMMIT'
control: async with con.transaction(), error caught around a nested con.transaction() (savepoint)
caught UniqueViolationError (23505) outside the savepoint block
the block exited normally
PostgreSQL: pg_xact_status(760) committed, marker visible True, session idle, last statement 'COMMIT;'
Expected
When the server completes COMMIT with ROLLBACK, the transaction should not be treated as committed, and callers should be able to tell. Raising from Transaction.commit() and from the exit of async with con.transaction() would do that; see the compatibility note below.
Actual
Both return normally (managed and explicit above), and a later rollback() reports the transaction as committed. PostgreSQL aborted it: pg_xact_status() is aborted, and the row is not visible. The savepoint control commits as expected.
Where the status is dropped
Transaction.__commit awaits self._connection.execute(query) without keeping its return value, then sets TransactionState.COMMITTED (transaction.py#L186-L192). That return value is the command status ("Status of the last SQL command", connection.py#L348). In this situation it is 'ROLLBACK', which the raw arm's execute('COMMIT') returns in the same state.
Notes
- Tortoise ORM 1.1.9's
in_transaction()and SQLAlchemy Core 2.1.5'sengine.begin(), both on asyncpg 0.32.0, also exit normally after a caught23505; both commit throughTransaction.commit(). I'm reporting it here because this is where the status is dropped. - psycopg 3 does the same in
Connection.commit(): commit() returns after PostgreSQL rolls the transaction back psycopg/psycopg#1453. - Raising here would change behaviour your tests rely on:
test_timeout_06(test_timeout.py#L104-L113) assertsInFailedSQLTransactionErrorinsideasync with self.con.transaction():, then expects the block to exit normally. Would you rather the transaction API raise whenCOMMITreturnsROLLBACK, or keep the current completion but stop recording the transaction as committed (Feature request: Make transaction state public #1130 asks for that state to be public)?
Once you have decided which behaviour you want, I'm happy to prepare a PR with tests for the managed and explicit forms and a savepoint control.
Activity
Claire-ovo commentedon Oct 27, 2026
Independently reproduced with asyncpg 0.32.0, Python 3.12.14 and PostgreSQL 17.10 on asyncio. I used connection-local temporary tables and a separate witness connection for
pg_xact_status(), without dropping or creating databases.This confirms the behavior on PG17 as well as the PG18 report. I have not tested uvloop, connection loss, or a product patch. Would independent regression coverage be useful once the compatibility decision is made? I can coordinate that with your implementation rather than open a competing fix. This validation was AI-assisted.
KevinBlakeLopez commentedon Oct 27, 2026
The four arms you ran match the report. I have not re-run them on PostgreSQL 17.10.
No maintainer has made the compatibility decision. I have not opened an implementation. #1401 is an outside pull request, and it is not that decision.
Yes, regression coverage is useful once a maintainer chooses. It should follow that choice. On 0.32.0,
test_timeout_04andtest_timeout_06pass: both catch aTimeoutErrorinside the block and expect the block to exit normally. Coverage needs to include those two tests. Another fix beside #1401 can wait until then.