Skip to content

/levels getAll hangs on cwbi-prod/test if locations levels contains a TSID LL #1663

@DanielTOsborne

Description

@DanielTOsborne

Describe the bug
Fetch location levels fails if there is a TSID LL present in the database. Related to #1662.

Expected behavior
No hang/connection closed message

To Reproduce
Using CWMS-python:

	cwms.api.init_session(api_root=CDA_API_ROOT, api_key=CDA_API_KEY)
	levels = cwms.get_location_levels(office_id=OFFICE)

	cwms.api.init_session(api_root=CWBI_API_ROOT, api_key=CWBI_API_KEY)

	failures = []

	#print(levels.json)
	print(f"LLs: {len(levels.json['levels'])}")
	for location in levels.json["levels"]:
		print(f"  {location['location-level-id']}")
		print(location)
		try:
			cwms.store_location_level(location)
		except Exception as ex:
			failures.append(location['location-level-id'])
			print(ex)

	if len(failures) > 0:
		print("The following location levels failed to store:")
		for failed in failures:
			print(failed)

Logs/Incident Identifier
cwms-python log:
WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'RemoteDisconnected('Remote end closed connection without response')': /cwms-data/levels?office=SPK

CWBI log:

27-Mar-2026 18:14:41.639 WARNING [Tomcat JDBC Pool Cleaner[858242339:1774032271424]] org.apache.tomcat.jdbc.pool.ConnectionPool.suspect Connection has been marked suspect, possibly abandoned PooledConnection[oracle.jdbc.driver.T4CConnection@7ba9cec1][15494 ms.]:java.lang.Exception 
warn 
	at org.apache.tomcat.jdbc.pool.ConnectionPool.getThreadDump(ConnectionPool.java:1235) 
	at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:847) 
	at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:684) 
	at java.base/java.lang.Thread.run(Thread.java:829) 
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63) 
	at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:491) 
	at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:973) 
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) 
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1831) 
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:935) 
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) 
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:398) 
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344) 
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:72) 

There was a reference in a prior log to "operation canceled", but that rolled off the history and I can't see it again.

CURL Commands
If applicable, add the curl command from the SWAGGER UI to help reproduce your problem.
REDACT any internal domains/IP addresses.

CDA Version (please complete the following information):
CWBI-prod: 2026.03.16
CWBI-test: 2026.03.06-testa

Additional context
The version on my T7 ( 2025.12.04-prod) works fine.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions