SOLR-18380: remove CloudSolrClient.connect(), ClusterStateProvider.connect() and the legacy Builder - #4762
Conversation
…nnect() and the legacy Builder Four deprecated members, 54 call sites found over five compile rounds. connect() was never substantive, and each implementation says so in its own words: BaseHttpClusterStateProvider.connect() was literally "getLiveNodes();", ZkClientClusterStateProvider.connect() was commented "Essentially a No-Op, but force a check that we're not closed", and DelegatingClusterStateProvider.connect() delegated. getLiveNodes() on the ZK provider goes through the same getZkStateReader(), so it throws AlreadyClosedException in exactly the same situations. Callers that wanted a forced connection now call getClusterStateProvider().getLiveNodes(). connect(long, TimeUnit) was the one substantive member - a 250 ms poll to a timeout. It had no production caller. The equivalent already exists in production as ConnectionImpl.connect(CloudSolrClient, long, TimeUnit) in solrj-streaming, built on getLiveNodes() in a timeout loop, if anyone needs it back. Builder(List, Optional) becomes Builder(String). This is not a pure rename: the connection-string parser takes the chroot to be everything from the first '/', while the old two-arg form put the whole string in the host list and dropped the chroot. In RunExampleTool the zkHost value comes from the -zkHost option or a running node, whose standard Solr form is host:2181/solr, so the migration there also fixes a latent bug. That is called out in a comment at the site. getClusterState() is left alone, per the ticket: its own javadoc says the future of ClusterState isn't clear, so it is scoped out of this removal. Three things that no compiler would have caught. A void method on a mock is a silent success; a chained one is a silent NPE. SolrMessageProcessor.connectToSolrIfNeeded is an unbounded while loop that sleeps 5 s on any exception. With the old client.connect() a bare mock returned harmlessly and the loop exited on the first pass. After the migration, getClusterStateProvider() returns null on the mock, the NPE is caught, and the test hangs instead of failing. Measured: 3.3 s with a stubbed provider, killed at 150 s without one. Fixed by stubbing the provider in setUp and verifying getLiveNodes() on it, which keeps the only coverage of that loop being entered exactly once. `var` truncates the compiler's own worklist. In TestSimplePropagatorDistributedTracing a failing `var builder = ...` made `var client = builder.build()` erroneous too, so the client.connect() on the next line was never reported. After fixing a var initialiser, the rest of that method body has to be re-read by hand. RoutedAliasUpdateProcessorTest is an @ignore'd abstract base with no tests of its own; its three concrete subclasses (Category/Dimensional/TimeRoutedAliasUpdateProcessorTest) carry the coverage and are what was run. Verified: compileJava and compileTestJava across solrj, solrj-zookeeper, solrj-streaming, core, test-framework and cross-dc-manager, spotlessCheck, ecjLintMain/Test on every touched module, renderJavadoc on solrj and core, and 19 changed test classes (counting the three RoutedAliasUpdateProcessorTest subclasses, not the abstract base) - 110 tests, 0 failures. AI-assisted (Claude Sonnet 5)
|
@dsmiley you deprecated all of this two months ago ( AI-assisted (Claude Sonnet 5) |
…udsolrclient-connect
dsmiley
left a comment
There was a problem hiding this comment.
Thanks for this!
Seems the scope is more than connect().
Two risks no compiler would catch, both fixed: a mock returning null from getClusterStateProvider() turned a passing test into a 150s hang instead of a failure (SolrMessageProcessorTest, now stubs the provider);
This is a standard risk of mocking frameworks and a reason why I personally strongly avoid them.
and a broken var initializer on one line silently swallowed the compile error on the next (TestSimplePropagatorDistributedTracing).
I didn't see this
| protected void waitToSeeLiveNodes(String zkHost, int numNodes) { | ||
| try (CloudSolrClient cloudClient = | ||
| new CloudSolrClient.Builder(List.of(zkHost), Optional.empty()).build()) { | ||
| // honours a chroot inside zkHost, e.g. zk1:2181/solr |
There was a problem hiding this comment.
I happen to like the spelling as well ;-). I like to paint my armour using colours ;-)
| private void clusterStatusRolesTest() throws Exception { | ||
| try (CloudSolrClient client = createCloudClient(null)) { | ||
| client.connect(); | ||
| client.getClusterStateProvider().getLiveNodes(); // force the connection now |
There was a problem hiding this comment.
Confirmed -- CloudSolrClient.sendRequest() unconditionally calls provider.getLiveNodes() for a request without a collection param, which is what client.request(...) sends a few lines later here (and getClusterState() in the other spot delegates to the same lazy-init path). So the forcing call was dead weight, not a migration bug -- removed in all 4 spots you flagged, both test classes re-run green.
AI-assisted (Claude Sonnet 5)
| private void replicaPropTest() throws Exception { | ||
| try (CloudSolrClient client = createCloudClient(null)) { | ||
| client.connect(); | ||
| client.getClusterStateProvider().getLiveNodes(); // force the connection now |
There was a problem hiding this comment.
Same as the TestCollectionAPI.java:690 thread -- removed.
AI-assisted (Claude Sonnet 5)
| private void testShardCreationNameValidation() throws Exception { | ||
| try (CloudSolrClient client = createCloudClient(null)) { | ||
| client.connect(); | ||
| client.getClusterStateProvider().getLiveNodes(); // force the connection now |
There was a problem hiding this comment.
Same as the TestCollectionAPI.java:690 thread -- removed.
AI-assisted (Claude Sonnet 5)
|
|
||
| try (CloudSolrClient client = createCloudClient(null)) { | ||
| client.connect(); | ||
| client.getClusterStateProvider().getLiveNodes(); // force the connection now |
There was a problem hiding this comment.
Same as the TestCollectionAPI.java:690 thread -- removed.
AI-assisted (Claude Sonnet 5)
|
|
||
| CloudSolrClient solrClient = cluster.getSolrClient(); | ||
| solrClient.connect(); | ||
| solrClient.getClusterStateProvider().getLiveNodes(); // force the connection now |
There was a problem hiding this comment.
Hmm; I suspect this isn't necessary. I believe obtaining the ZkStateReader itself implies a connection.
There was a problem hiding this comment.
Confirmed -- removed. ZkStateReader.from(solrClient) on the very next line forces the same lazy connection, same as the 4 spots you flagged earlier.
AI-assisted (Claude Sonnet 5)
| cloudInit = true; | ||
| cloudClient = createCloudClient(DEFAULT_COLLECTION); | ||
| cloudClient.connect(); | ||
| cloudClient.getClusterStateProvider().getLiveNodes(); // force the connection now |
There was a problem hiding this comment.
ZkStateReader.from will do that
There was a problem hiding this comment.
Right -- ZkStateReader.from(cloudClient) two lines below does exactly that. Removed.
AI-assisted (Claude Sonnet 5)
| // force the ZkStateReader into existence | ||
| if (reader == null) cluster.getSolrClient().getClusterStateProvider().getLiveNodes(); |
There was a problem hiding this comment.
getZkStateReader will do that well enough
There was a problem hiding this comment.
Agreed -- simplified to cluster.getZkStateReader().getZkClient(). Verified getZkStateReader() never returns null: 4000 concurrent calls right after cluster startup, all non-null, and a post-shutdown call throws AlreadyClosedException, never null. Dug into why the null-check existed: in 2017 (SOLR-10278) getZkStateReader() was a bare field getter, nullable before first connection -- the guard was correct then. SOLR-13393 (2019-04-11) rewrote it into today's lazy-init-or-throw pattern to fix a ZkStateReader leak on shutdown, closing the null path for good. The guard's been unreachable ever since -- over 6 years.
AI-assisted (Claude Sonnet 5)
| } | ||
|
|
||
| private CloudSolrClient newCloudSolrClient() { | ||
| var builder = |
There was a problem hiding this comment.
the PR intro implied this var isn't what we think it is. Isn't it a builder?
There was a problem hiding this comment.
Yes, builder is a CloudSolrClient.Builder -- unambiguous at the source level. The PR description was describing a build-time artifact: reintroduced both removed calls together and confirmed javac reports only the constructor error, silently swallowing the separate client.connect() removal error on the next line (its type can't resolve once builder's initializer fails). Not a type-identity issue, just an ambiguous sentence on my part -- sorry.
AI-assisted (Claude Sonnet 5)
…ests CloudSolrClient.sendRequest() already calls provider.getLiveNodes() unconditionally for a collection-less request, and getClusterState() delegates to the same lazy-init path -- so the migrated connect() replacement was dead weight in these 4 spots, not a migration bug.
…7 call sites Replaces the hand-rolled `new CloudSolrClient.Builder(cluster.getZkServer() .getZkAddress())` pattern, repeated across 5 test classes, with a single builder method on the cluster. One more site (ForceLeaderTest) uses the older AbstractFullDistribZkTestBase framework with its own `zkServer` field, not MiniSolrCloudCluster -- left untouched, out of scope for this method.
The comment described the replacement line by comparing it to a method this same PR deletes -- reworded to describe what the line does instead.
JdbcTest and AbstractFullDistribZkTestBase both call ZkStateReader.from() on the very next line, which forces the same lazy connection -- the explicit call was redundant, same shape as the earlier 4-site fix. SolrCloudTestCase.zkClient()'s null-check was dead code: getZkStateReader() never returns null (verified empirically, 4000 concurrent calls plus a post-shutdown call that throws AlreadyClosedException instead). Git archaeology on ZkClientClusterStateProvider confirms why: it was a bare, genuinely-nullable field getter in 2017 (SOLR-10278), and SOLR-13393 (2019) rewrote it into today's lazy-init-or-throw form to fix a ZkStateReader leak on shutdown -- the guard has been unreachable ever since, its protection superseded by that fix.
There was a problem hiding this comment.
Just to share that I sometimes lean in favor of a CHANGELOG -- I think this is a good one CC @janhoy so you notice ;-)
| * cluster's ZooKeeper ensemble, to customize further (default collection, update routing, HTTP | ||
| * client, ...) before calling {@code build()}. | ||
| */ | ||
| public CloudSolrClient.Builder getSolrClientBuilder() { |
There was a problem hiding this comment.
rename to newSolrClientBuilder; we aren't returning something we already have.
| * Returns a new {@link org.apache.solr.client.solrj.impl.CloudSolrClient.Builder} pointed at this | ||
| * cluster's ZooKeeper ensemble, to customize further (default collection, update routing, HTTP |
There was a problem hiding this comment.
don't overdocument implementations details (AI does this all the time). This is an implementation detail that I plan to change to pick randomly between the ZK & HTTP ClusterState at some point.
|
|
||
| private void assertCollectionExists(String name) { | ||
| solrClient.getClusterStateProvider().connect(); // TODO get rid of this | ||
| solrClient.getClusterStateProvider().getLiveNodes(); // TODO get rid of this |
There was a problem hiding this comment.
Your comment/investigation #4762 (comment) nicely confirms you can remove this
https://issues.apache.org/jira/browse/SOLR-18380
Removes
CloudSolrClient.connect()/connect(long, TimeUnit),ClusterStateProvider.connect(), and the legacyBuilder(List<String>, Optional<String>)— 54 call sites.connect()was never substantive in any implementation (one was literallygetLiveNodes();); callers now callgetClusterStateProvider().getLiveNodes()directly.getClusterState()is left alone, per the ticket's own note that its future is undecided.Where to look: the
Builder(List, Optional)→Builder(String)swap is not a pure rename — the connection-string parser takes everything after the first/as the chroot, which the two-arg form discarded.RunExampleToolpicks up a real fix from this (itszkHostcan carry a chroot).Two risks no compiler would catch, both fixed: a mock returning
nullfromgetClusterStateProvider()turned a passing test into a 150s hang instead of a failure (SolrMessageProcessorTest, now stubs the provider); and a brokenvarinitializer on one line silently swallowed the compile error on the next (TestSimplePropagatorDistributedTracing).54 call sites, 19 changed test classes (one @ignore'd abstract base correctly has no tests of its own — its 3 concrete subclasses do), compile + gates clean.
SOLR-18373, SOLR-18378, SOLR-18382 and SOLR-18386 touch files this PR also touches — merging this one first should make those cleaner to extract.
AI-assisted (Claude Sonnet 5)