[refactor](fe) Merge fe-connector-api into fe-connector-spi

.licenserc.yaml

@@ -51,7 +51,7 @@ header:
     - "fe/fe-core/src/test/resources/stageUtilTest.txt"
     # Recorded method-surface baseline; ConnectorMetadataSurfaceTest reads every
     # non-blank line as a method signature, so a header would break the test.
-    - "fe/fe-connector/fe-connector-api/src/test/resources/connector-metadata-methods.txt"
+    - "fe/fe-connector/fe-connector-spi/src/test/resources/connector-metadata-methods.txt"
     # Recorded plugin API surface baselines, one per plugin family. Same reason as
     # above: each *PluginSurfaceTest.readBaseline() adds every non-blank line to the
     # expected signature set, so a header would be read back as phantom signatures.

build-support/check-fe-connector-imports.sh

@@ -25,7 +25,7 @@
 # datasource / qe / analysis / nereids / planner / persist / transaction / fs /
 # statistics / mysql / service). Anything they need from fe-core has to be
 # exposed through the SPI in
-#   org.apache.doris.connector.{api,spi,extension,...}
+#   org.apache.doris.connector.{spi,metastore,...}
 # or shared types in org.apache.doris.thrift / org.apache.doris.filesystem.
 #
 # The gate matches both plain and `import static` imports, scans src/main/java

build-support/tests/test-fe-connector-imports.sh

@@ -58,7 +58,7 @@ import org.apache.doris.mysql.privilege.Auth;
 import org.apache.doris.service.FrontendOptions;
 import org.apache.doris.thrift.TFoo;
 import org.apache.doris.filesystem.Bar;
-import org.apache.doris.connector.api.Baz;
+import org.apache.doris.connector.spi.Baz;
 import org.apache.doris.datasource.hive.HiveVersionUtil;
 import static org.apache.doris.datasource.hive.HiveVersionUtil.SOME_CONST;
 public class FakeConn {}
@@ -117,7 +117,7 @@ must_not_report() {
 }
 must_not_report 'org.apache.doris.thrift.TFoo'          # SPI-shared
 must_not_report 'org.apache.doris.filesystem.Bar'       # SPI-shared
-must_not_report 'org.apache.doris.connector.api.Baz'    # SPI
+must_not_report 'org.apache.doris.connector.spi.Baz'    # SPI
 must_not_report 'HiveVersionUtil;'                      # vendored (non-static)
 must_not_report 'HiveVersionUtil.SOME_CONST;'           # vendored (static) — E3
 

build.sh

@@ -723,8 +723,7 @@ if [[ "${BUILD_FE}" -eq 1 ]]; then
         fi
     done
     unset _fs_mod
-    # Connector API, SPI, and plugin modules (loaded at runtime as plugins)
-    modules+=("fe-connector/fe-connector-api")
+    # Connector SPI and plugin modules (loaded at runtime as plugins)
     modules+=("fe-connector/fe-connector-spi")
     for _conn_mod in es jdbc maxcompute trino hms hive paimon hudi iceberg; do
         if [[ -d "${DORIS_HOME}/fe/fe-connector/fe-connector-${_conn_mod}" ]]; then

fe/fe-connector/AGENTS.md

@@ -14,7 +14,7 @@ mvn -f fe/pom.xml -pl :fe-connector-<module> -am test
 
 # Run a single test class. Keep -DfailIfNoTests=false: with -am, upstream
 # modules have no matching tests and would fail the build otherwise.
-mvn -f fe/pom.xml -pl :fe-connector-api -am test \
+mvn -f fe/pom.xml -pl :fe-connector-spi -am test \
     -Dtest=ConnectorMetadataSurfaceTest -DfailIfNoTests=false
 ```
 
@@ -40,20 +40,25 @@ self-tests live in `build-support/` and `build-support/tests/`):
    wired into this directory's `pom.xml`. fe-connector modules must not
    import fe-core internals, in main OR test sources. When a connector needs
    something from the engine, the fix is to extend the SPI in
-   fe-connector-api / fe-connector-spi — never to import fe-core.
+   fe-connector-spi — never to import fe-core.
 2. **Metadata funnel** — `build-support/check-fe-core-metadata-funnel.sh`,
    wired into fe-core's `pom.xml`. Inside fe-core, only
    `PluginDrivenMetadata` may call `Connector#getMetadata`; exempt call
    sites carry a `getMetadata-funnel-exempt` marker, and deleting a marker
    auto-tightens the gate.
 
-**Changing the shared SPI surface (fe-connector-api):** regenerate
-`fe-connector-api/src/test/resources/connector-metadata-methods.txt` in the
-SAME commit — run `ConnectorMetadataSurfaceTest` and copy the "actual" set
-from the failure output. Always run fe-connector-api's OWN suite after an
-SPI change; running only consumer modules will not catch a stale baseline.
-The same suite pins the `@ConnectorMustImplement` set, so promoting a method
-into the minimum implementation set is a deliberate, reviewed change.
+**Changing the shared SPI surface (fe-connector-spi):** regenerate BOTH
+recorded baselines in the SAME commit — `connector-metadata-methods.txt`
+(`ConnectorMetadataSurfaceTest`) and `connector-plugin-surface.txt`
+(`ConnectorPluginSurfaceTest`), under
+`fe-connector-spi/src/test/resources/`; run the test and copy the "actual"
+set from the failure output. Any surface change is a MAJOR change: bump
+`connector.plugin.api.version` in `fe/fe-connector/pom.xml` and the version
+pinned in `ConnectorPluginSurfaceTest` in that same commit. Always run
+fe-connector-spi's OWN suite after an SPI change; running only consumer
+modules will not catch a stale baseline. The same suite pins the
+`@ConnectorMustImplement` set, so promoting a method into the minimum
+implementation set is a deliberate, reviewed change.
 
 ## Invariants Without a Gate
 
@@ -94,8 +99,9 @@ these has a concrete failure mode.
 
 ## Task Recipes
 
-- **Change the SPI surface**: edit fe-connector-api → regenerate the
-  baseline in the same commit → run fe-connector-api's suite → adjust
+- **Change the SPI surface**: edit fe-connector-spi → regenerate both
+  baselines and bump the plugin API version in the same commit → run
+  fe-connector-spi's suite → adjust
   affected connectors (grep for overrides) → run their module tests.
 - **Fix a connector bug**: module-scoped build + tests (recipes above;
   paimon via `install`). For user-visible behavior, extend the e2e suites

fe/fe-connector/README.md

@@ -40,8 +40,13 @@ Roles only — one line each. For anything deeper, read the module's javadoc
 
 | Module | Role |
 |---|---|
-| `fe-connector-api` | The engine <-> connector contract: `ConnectorMetadata` with its Ops sub-interfaces, plus handle / pushdown / mvcc / scan / write / ddl / procedure / event / rest types. The javadoc here **is** the API reference. |
-| `fe-connector-spi` | Bootstrap contract: `ConnectorProvider` (discovery identity + factory) and `ConnectorContext` (what the engine hands a connector, including sibling-connector creation). |
+| `fe-connector-spi` | The whole engine <-> connector contract, in both directions. What a connector implements: `ConnectorProvider` (discovery identity + factory), `Connector`, `ConnectorMetadata` with its Ops sub-interfaces, plus handle / pushdown / mvcc / scan / write / ddl / procedure / event / rest types. What the engine implements and hands down: `ConnectorContext` (including sibling-connector creation), `ConnectorStorageContext`, `ConnectorSession`, `ConnectorConf`. The javadoc here **is** the API reference. |
+
+The two directions are one module on purpose: the boundary is bidirectional
+(`ConnectorProvider.create` takes a `ConnectorContext`, and `ConnectorContext`
+hands back a `Connector`), so splitting it by "who implements" would be
+circular. Trino makes the same call with `trino-spi`. Contrast the metastore
+layer below, where the split is acyclic and the usual api/spi convention holds.
 
 **Metastore layer** (how connectors reach a metastore without hand-parsing
 endpoint properties)
@@ -99,8 +104,8 @@ exactly one `ConnectorMetadata` instance per catalog on the statement's
 `ConnectorStatementScope` and closes it deterministically at statement end.
 Scan planning follows the same shape: the generic `PluginDrivenScanNode`
 (fe-core) delegates all per-source planning to the connector's
-`ConnectorScanPlanProvider` — note the interface lives in fe-connector-api
-(`api.scan`), not in fe-core.
+`ConnectorScanPlanProvider` — note the interface lives in fe-connector-spi
+(`spi.scan`), not in fe-core.
 
 **Classloading.** Plugins load child-first, each carrying its own runtime
 closure. Wherever engine code crosses into a plugin — or a bundled library
@@ -119,17 +124,16 @@ This document never lists SPI methods. The truth lives in code, behind four
 mechanisms:
 
 1. **Javadoc is the API reference.** Start at `ConnectorMetadata` (and its
-   Ops sub-interfaces) in fe-connector-api, and `ConnectorProvider` in
-   fe-connector-spi. Every SPI method has a default body, so each
-   sub-interface's class javadoc states its minimum implementation set,
-   lifecycle, and threading rules.
+   Ops sub-interfaces) and `ConnectorProvider`, both in fe-connector-spi.
+   Every SPI method has a default body, so each sub-interface's class javadoc
+   states its minimum implementation set, lifecycle, and threading rules.
 2. **`@ConnectorMustImplement`** is the machine-readable half of the minimum
    implementation set: it marks the default methods a connector is
    nevertheless expected to override, with `when` naming the capability that
    triggers the obligation. A unit test pins the annotated set, so promoting
    a method is a deliberate, reviewed change.
 3. **The recorded surface.**
-   `fe-connector-api/src/test/resources/connector-metadata-methods.txt`
+   `fe-connector-spi/src/test/resources/connector-metadata-methods.txt`
    freezes the public method surface of `ConnectorMetadata`;
    `ConnectorMetadataSurfaceTest` fails on any drift. Adding, removing, or
    moving SPI methods must regenerate this baseline in the same commit (run
@@ -146,7 +150,7 @@ mechanisms:
    source; if a connector does not opt in, the feature stays off.
 
 Statement-scoped memoization has one framework-wide convention worth knowing
-before you read connector code: `ConnectorStatementScopes` (fe-connector-api)
+before you read connector code: `ConnectorStatementScopes` (fe-connector-spi)
 keys per-statement values by `(catalogId, db, table, queryId)` plus a
 connector-owned namespace constant prefixed with the connector's
 `getType()`. Each connector guards its own prefix with a unit test
@@ -239,10 +243,10 @@ metastore/shade/cache). For a write path, the richest example is
 ## Testing and Verification
 
 - **Unit tests** live in each module; build/test recipes are in `AGENTS.md`.
-- **The shared SPI surface** is guarded by fe-connector-api's own suite
-  (`ConnectorMetadataSurfaceTest`). Whenever you touch fe-connector-api, run
-  that module's tests — a consumer-only test run will not catch a stale
-  baseline.
+- **The shared SPI surface** is guarded by fe-connector-spi's own suite
+  (`ConnectorMetadataSurfaceTest` and `ConnectorPluginSurfaceTest`). Whenever
+  you touch fe-connector-spi, run that module's tests — a consumer-only test
+  run will not catch a stale baseline.
 - **Architecture gates** run in the `validate` phase of every FE build: the
   forbidden-import gate for this directory and the metadata-funnel gate for
   fe-core. Scripts and their self-tests live in `build-support/` and