[refactor](fe) Merge fe-connector-api into fe-connector-spi - #66407
Merged
Conversation
morningman
requested review from
924060929,
CalvinKirs,
englefly,
morrySnow and
starocean999
as code owners
August 4, 2026 02:25
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
Contributor
Author
|
run buildall |
### What problem does this PR solve?
Problem Summary:
The connector plugin contract was split across two modules whose names were
inverted relative to common usage: `fe-connector-api` held the types a plugin
IMPLEMENTS (`Connector`, `ConnectorMetadata` and its Ops sub-interfaces, the
scan / write / procedure providers, handles, pushdown and ddl value types) —
which is what "SPI" normally names — while `fe-connector-spi` held mostly the
engine services a plugin CONSUMES (`ConnectorContext`, `ConnectorStorageContext`,
`ConnectorConf`), which is what "API" normally names. Both modules' package-info
already documented the inversion and asked readers to "read the content, not the
name". The `fe-connector-metastore-api` / `-metastore-spi` pair right next to
them follows the opposite (standard) convention, so one directory carried two
contradictory naming rules.
Swapping the two names does not fix it. A strict split by "who implements" is
circular: `ConnectorProvider.create` takes a `ConnectorContext` (spi -> api),
while `ConnectorContext.createSiblingConnector` returns a `Connector`
(api -> spi). That cycle is exactly why Trino keeps its whole plugin surface in
a single `trino-spi` module, and this framework is modelled on Trino.
This commit does the same: `fe-connector-api` is merged into
`fe-connector-spi`, and `org.apache.doris.connector.api` becomes
`org.apache.doris.connector.spi`. The merge is dependency-neutral — every module
that depended on `fe-connector-api` (hive, hudi, maxcompute, paimon, trino,
fe-core) already depended on `fe-connector-spi` as well.
Two consequences worth calling out:
- The connector plugin API version is bumped 2.0 -> 3.0. Every type on the
contract changed its fully-qualified name, so a plugin built against 2.0 must
be refused by `ApiVersionGate` at load time rather than fail later with
`NoClassDefFoundError`. Both recorded baselines
(`connector-metadata-methods.txt`, `connector-plugin-surface.txt`) and the
version pinned in `ConnectorPluginSurfaceTest` are updated in this commit.
`ConnectorPluginManagerTest` used to hardcode the compatible major as a
literal; it now derives both the stale and the current major from the
kernel's declared version, because what that test asserts is the gate
(stale refused, current admitted), not which number is current.
`ConnectorPluginSurfaceTest` remains the one deliberate speed bump.
- `ChildFirstClassLoader.DEFAULT_PARENT_FIRST_PACKAGES` listed only
`org.apache.doris.connector.api.`, so `ConnectorProvider` and
`ConnectorContext` were not parent-first. After the merge the single
`org.apache.doris.connector.spi.` entry covers the whole contract.
The merged `package-info.java` keeps the existing Rule 1-7 design rules; Rule 4
now explains why both directions live in one module instead of disclaiming the
inverted names. README.md and AGENTS.md are updated accordingly.
### Release note
None
### Check List (For Author)
- Test: Unit Test
- Full FE build passes, including fe-core test compilation.
- Every unit test under `org.apache.doris.connector` passes: 314 test
classes / 1421 cases, 0 failures, 0 errors, across the 58-module
reactor (fe-core included).
- Checkstyle passes across the whole fe reactor.
- Both architecture gates pass: `check-fe-connector-imports.sh` (plus its
self-test) and `check-fe-core-metadata-funnel.sh`.
- No regression test run: this is a pure rename/merge with no behavior
change.
- Behavior changed: No
- Does this need documentation: No
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
Author
|
run buildall |
Contributor
FE UT Coverage ReportIncrement line coverage |
CalvinKirs
approved these changes
Aug 4, 2026
Contributor
|
PR approved by at least one committer and no changes requested. |
Contributor
|
PR approved by anyone and no changes requested. |
924060929
approved these changes
Aug 4, 2026
Contributor
Author
|
skip buildall |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Problem Summary:
The connector plugin contract was split across two modules whose names were
inverted relative to common usage:
fe-connector-apiheld the types a pluginIMPLEMENTS (
Connector,ConnectorMetadataand its Ops sub-interfaces, thescan / write / procedure providers, handles, pushdown and ddl value types) —
which is what "SPI" normally names — while
fe-connector-spiheld mostly theengine services a plugin CONSUMES (
ConnectorContext,ConnectorStorageContext,ConnectorConf), which is what "API" normally names. Both modules' package-infoalready documented the inversion and asked readers to "read the content, not the
name". The
fe-connector-metastore-api/-metastore-spipair right next tothem follows the opposite (standard) convention, so one directory carried two
contradictory naming rules.
Swapping the two names does not fix it. A strict split by "who implements" is
circular:
ConnectorProvider.createtakes aConnectorContext(spi -> api),while
ConnectorContext.createSiblingConnectorreturns aConnector(api -> spi). That cycle is exactly why Trino keeps its whole plugin surface in
a single
trino-spimodule, and this framework is modelled on Trino.This commit does the same:
fe-connector-apiis merged intofe-connector-spi, andorg.apache.doris.connector.apibecomesorg.apache.doris.connector.spi. The merge is dependency-neutral — every modulethat depended on
fe-connector-api(hive, hudi, maxcompute, paimon, trino,fe-core) already depended on
fe-connector-spias well.Two consequences worth calling out:
contract changed its fully-qualified name, so a plugin built against 2.0 must
be refused by
ApiVersionGateat load time rather than fail later withNoClassDefFoundError. Both recorded baselines(
connector-metadata-methods.txt,connector-plugin-surface.txt) and theversion pinned in
ConnectorPluginSurfaceTestare updated in this commit.ConnectorPluginManagerTestused to hardcode the compatible major as aliteral; it now derives both the stale and the current major from the
kernel's declared version, because what that test asserts is the gate
(stale refused, current admitted), not which number is current.
ConnectorPluginSurfaceTestremains the one deliberate speed bump.ChildFirstClassLoader.DEFAULT_PARENT_FIRST_PACKAGESlisted onlyorg.apache.doris.connector.api., soConnectorProviderandConnectorContextwere not parent-first. After the merge the singleorg.apache.doris.connector.spi.entry covers the whole contract.The merged
package-info.javakeeps the existing Rule 1-7 design rules; Rule 4now explains why both directions live in one module instead of disclaiming the
inverted names. README.md and AGENTS.md are updated accordingly.
Release note
None
Check List (For Author)
org.apache.doris.connectorpasses: 314 testclasses / 1421 cases, 0 failures, 0 errors, across the 58-module
reactor (fe-core included).
check-fe-connector-imports.sh(plus itsself-test) and
check-fe-core-metadata-funnel.sh.change.
🤖 Generated with Claude Code