Skip to content

[refactor](fe) Merge fe-connector-api into fe-connector-spi - #66407

Merged
morningman merged 1 commit into
apache:masterfrom
morningman:correct-api
Aug 4, 2026
Merged

[refactor](fe) Merge fe-connector-api into fe-connector-spi#66407
morningman merged 1 commit into
apache:masterfrom
morningman:correct-api

Conversation

@morningman

@morningman morningman commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Problem Summary:

The connector plugin contract was split across two modules whose names were
inverted relative to common usage: fe-connector-api held the types a plugin
IMPLEMENTS (Connector, ConnectorMetadata and its Ops sub-interfaces, the
scan / write / procedure providers, handles, pushdown and ddl value types) —
which is what "SPI" normally names — while fe-connector-spi held mostly the
engine services a plugin CONSUMES (ConnectorContext, ConnectorStorageContext,
ConnectorConf), which is what "API" normally names. Both modules' package-info
already documented the inversion and asked readers to "read the content, not the
name". The fe-connector-metastore-api / -metastore-spi pair right next to
them follows the opposite (standard) convention, so one directory carried two
contradictory naming rules.

Swapping the two names does not fix it. A strict split by "who implements" is
circular: ConnectorProvider.create takes a ConnectorContext (spi -> api),
while ConnectorContext.createSiblingConnector returns a Connector
(api -> spi). That cycle is exactly why Trino keeps its whole plugin surface in
a single trino-spi module, and this framework is modelled on Trino.

This commit does the same: fe-connector-api is merged into
fe-connector-spi, and org.apache.doris.connector.api becomes
org.apache.doris.connector.spi. The merge is dependency-neutral — every module
that depended on fe-connector-api (hive, hudi, maxcompute, paimon, trino,
fe-core) already depended on fe-connector-spi as well.

Two consequences worth calling out:

  • The connector plugin API version is bumped 2.0 -> 3.0. Every type on the
    contract changed its fully-qualified name, so a plugin built against 2.0 must
    be refused by ApiVersionGate at load time rather than fail later with
    NoClassDefFoundError. Both recorded baselines
    (connector-metadata-methods.txt, connector-plugin-surface.txt) and the
    version pinned in ConnectorPluginSurfaceTest are updated in this commit.
    ConnectorPluginManagerTest used to hardcode the compatible major as a
    literal; it now derives both the stale and the current major from the
    kernel's declared version, because what that test asserts is the gate
    (stale refused, current admitted), not which number is current.
    ConnectorPluginSurfaceTest remains the one deliberate speed bump.
  • ChildFirstClassLoader.DEFAULT_PARENT_FIRST_PACKAGES listed only
    org.apache.doris.connector.api., so ConnectorProvider and
    ConnectorContext were not parent-first. After the merge the single
    org.apache.doris.connector.spi. entry covers the whole contract.

The merged package-info.java keeps the existing Rule 1-7 design rules; Rule 4
now explains why both directions live in one module instead of disclaiming the
inverted names. README.md and AGENTS.md are updated accordingly.

Release note

None

Check List (For Author)

  • Test: Unit Test
    • Full FE build passes, including fe-core test compilation.
    • Every unit test under org.apache.doris.connector passes: 314 test
      classes / 1421 cases, 0 failures, 0 errors, across the 58-module
      reactor (fe-core included).
    • Checkstyle passes across the whole fe reactor.
    • Both architecture gates pass: check-fe-connector-imports.sh (plus its
      self-test) and check-fe-core-metadata-funnel.sh.
    • No regression test run: this is a pure rename/merge with no behavior
      change.
  • Behavior changed: No
  • Does this need documentation: No

🤖 Generated with Claude Code

@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman

Copy link
Copy Markdown
Contributor Author

run buildall

@morningman

Copy link
Copy Markdown
Contributor Author

run buildall

### What problem does this PR solve?

Problem Summary:

The connector plugin contract was split across two modules whose names were
inverted relative to common usage: `fe-connector-api` held the types a plugin
IMPLEMENTS (`Connector`, `ConnectorMetadata` and its Ops sub-interfaces, the
scan / write / procedure providers, handles, pushdown and ddl value types) —
which is what "SPI" normally names — while `fe-connector-spi` held mostly the
engine services a plugin CONSUMES (`ConnectorContext`, `ConnectorStorageContext`,
`ConnectorConf`), which is what "API" normally names. Both modules' package-info
already documented the inversion and asked readers to "read the content, not the
name". The `fe-connector-metastore-api` / `-metastore-spi` pair right next to
them follows the opposite (standard) convention, so one directory carried two
contradictory naming rules.

Swapping the two names does not fix it. A strict split by "who implements" is
circular: `ConnectorProvider.create` takes a `ConnectorContext` (spi -> api),
while `ConnectorContext.createSiblingConnector` returns a `Connector`
(api -> spi). That cycle is exactly why Trino keeps its whole plugin surface in
a single `trino-spi` module, and this framework is modelled on Trino.

This commit does the same: `fe-connector-api` is merged into
`fe-connector-spi`, and `org.apache.doris.connector.api` becomes
`org.apache.doris.connector.spi`. The merge is dependency-neutral — every module
that depended on `fe-connector-api` (hive, hudi, maxcompute, paimon, trino,
fe-core) already depended on `fe-connector-spi` as well.

Two consequences worth calling out:

- The connector plugin API version is bumped 2.0 -> 3.0. Every type on the
  contract changed its fully-qualified name, so a plugin built against 2.0 must
  be refused by `ApiVersionGate` at load time rather than fail later with
  `NoClassDefFoundError`. Both recorded baselines
  (`connector-metadata-methods.txt`, `connector-plugin-surface.txt`) and the
  version pinned in `ConnectorPluginSurfaceTest` are updated in this commit.
  `ConnectorPluginManagerTest` used to hardcode the compatible major as a
  literal; it now derives both the stale and the current major from the
  kernel's declared version, because what that test asserts is the gate
  (stale refused, current admitted), not which number is current.
  `ConnectorPluginSurfaceTest` remains the one deliberate speed bump.
- `ChildFirstClassLoader.DEFAULT_PARENT_FIRST_PACKAGES` listed only
  `org.apache.doris.connector.api.`, so `ConnectorProvider` and
  `ConnectorContext` were not parent-first. After the merge the single
  `org.apache.doris.connector.spi.` entry covers the whole contract.

The merged `package-info.java` keeps the existing Rule 1-7 design rules; Rule 4
now explains why both directions live in one module instead of disclaiming the
inverted names. README.md and AGENTS.md are updated accordingly.

### Release note

None

### Check List (For Author)

- Test: Unit Test
    - Full FE build passes, including fe-core test compilation.
    - Every unit test under `org.apache.doris.connector` passes: 314 test
      classes / 1421 cases, 0 failures, 0 errors, across the 58-module
      reactor (fe-core included).
    - Checkstyle passes across the whole fe reactor.
    - Both architecture gates pass: `check-fe-connector-imports.sh` (plus its
      self-test) and `check-fe-core-metadata-funnel.sh`.
    - No regression test run: this is a pure rename/merge with no behavior
      change.
- Behavior changed: No
- Does this need documentation: No

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@morningman

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 100% (0/0) 🎉
Increment coverage report
Complete coverage report

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Aug 4, 2026
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@morningman

Copy link
Copy Markdown
Contributor Author

skip buildall

@morningman
morningman merged commit 13f5924 into apache:master Aug 4, 2026
32 of 33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants