[KYUUBI #7645][SERVER] Fix operation state metrics transition

[HanShuhang]

Why are the changes needed?

KyuubiOperation#setState updated operation state metrics before delegating to AbstractOperation#setState, where the state transition is validated and the actual operation state is changed. It also did not protect the whole transition-and-metrics-update sequence with the operation lock.

This can cause inaccurate operation state metrics in two cases:

1. A stale or invalid state transition may update metrics before being rejected.

   For example, an operation may have already moved from RUNNING to FINISHED, while a delayed cancel/close path still tries to update it to another terminal state such as CANCELED. In the old implementation, the target state metric was updated before OperationState#validateTransition rejected the transition, so the operation remained in the original state but the metrics had already been changed.

2. Concurrent state transitions for the same operation may update metrics based on the same stale old state.

   For example, one thread may observe the remote query as finished and call KyuubiOperation#setState(FINISHED), while another thread concurrently handles cancel/close/timeout/error and calls KyuubiOperation#setState(CANCELED) or another terminal state. Since the old implementation read state and updated metrics outside a common operation lock, both paths could observe the old state as RUNNING and both update the RUNNING metric, making kyuubi.operation.state.ExecuteStatement.running inaccurate.

This patch makes KyuubiOperation#setState execute under the operation lock, captures the old state once, delegates to AbstractOperation#setState first, and updates operation state metrics only after the state transition succeeds.

How was this patch tested?

Added a unit test in KyuubiOperationSuite to verify that a stale terminal transition does not update operation state metrics.

build/mvn test -pl kyuubi-server -am \
  -Pspark-provided -Pflink-provided -Phive-provided \
  -Dtest=none \
  -DwildcardSuites=org.apache.kyuubi.operation.KyuubiOperationSuite

Was this patch authored or co-authored using generative AI tooling?

Assisted-by: TraeCode with GPT-5

[yikf]

Hi @HanShuhang , thanks for working on this, left two minor points, let me know what you think.

• Please file/link an issue and add '[KYUUBI #NNNN]' to the title.

• TestKyuubiOperation is over-implemented — KyuubiOperation already implements everything except runInternal. Trim to:

Suggested change

	private class TestKyuubiOperation(session: Session) extends KyuubiOperation(session) {
	private class TestKyuubiOperation(session: Session) extends KyuubiOperation(session) {
	def transitState(newState: OperationState): Unit = setState(newState)
	override protected def runInternal(): Unit = {}
	}

[HanShuhang]

Thanks! I agree with your suggestions and have updated the PR accordingly.