[SPARK-57262][SQL][WEBUI] Job description derived from a query should respect spark.sql.redaction.string.regex

[sarutak]

What changes were proposed in this pull request?

This PR changes SparkSQLDriver.scala to redact a query before setJobDescription.

Why are the changes needed?

In the current implementation, redaction is done in SQLExecution.scala so the description in the table on the top of /SQL/execution is redacted.

But the description in the table on the /jobs page and the one in the table on the bottom of /SQL/execution page are not redacted.

Does this PR introduce any user-facing change?

Yes.

How was this patch tested?

Added new test.
Also confirmed descriptions are redacted in UI.

$ bin/spark-sql --conf spark.sql.redaction.string.regex="secret.*=.*"
spark-sql (default)>  CREATE TABLE test1(secret string);
spark-sql (default)> SELECT * FROM test1 WHERE secret=1;

Was this patch authored or co-authored using generative AI tooling?

Kiro CLI / Claude

[dongjoon-hyun]

cc @cloud-fan , too