Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -378,3 +378,73 @@ public class Main {
}
}
```

### Multi-query

`onlineQueryMulti` runs several *independent* queries in parallel, using a single request. Reach for
it when the queries are rooted in different feature classes, or otherwise ask for unrelated outputs,
and you would rather not pay a round trip for each. To run one query over many rows of input, use
`onlineQuery` — it is already a bulk operation.

Multi-query requires the gRPC client.

```java
import ai.chalk.client.ChalkClient;
import ai.chalk.exceptions.ServerError;
import ai.chalk.models.OnlineQueryMultiResult;
import ai.chalk.models.OnlineQueryParams;
import ai.chalk.models.OnlineQueryResult;

import java.util.Arrays;
import java.util.List;

ChalkClient client = ChalkClient.createGrpc();

var users = OnlineQueryParams.builder()
.withInput("user.id", Arrays.asList(1, 2, 3))
.withOutputs("user.email")
.build();
var merchants = OnlineQueryParams.builder()
.withInput("merchant.id", Arrays.asList("a", "b"))
.withOutputs("merchant.risk_score")
.build();

try (OnlineQueryMultiResult multi = client.onlineQueryMulti(List.of(users, merchants))) {
for (ServerError error : multi.getGlobalErrors()) {
// errors that do not belong to any one query
}
for (OnlineQueryResult result : multi.getResults()) {
if (result.getErrors().length > 0) {
// this query failed; the others are unaffected
continue;
}
if (result.getScalarsTable() == null) {
// this query never ran -- see getGlobalErrors() above for the reason
continue;
}
// do something with the result
}
}
```

A few things to know:

- **Results are positional.** `getResults()` holds one result per query, in the order you passed
them in.
- **Queries fail independently.** A resolver that fails while computing one query is reported on
that query's own result, and leaves the others intact. `getGlobalErrors()` holds the errors the
engine did not attach to a specific result. Note that a query which failed to *run* at all — it
was rate limited, or the engine hit an internal error on it — is reported there, naming the query
by its 1-based position in the message, while its slot in `getResults()` holds an empty result
with no errors of its own. So check the global errors too, rather than reading an empty result as
a successful one.
- **Close the wrapper, not the results.** Closing the `OnlineQueryMultiResult` closes every one of
its results and releases all of their Arrow memory.
- **Some settings apply to the whole request,** which carries one deadline and one set of headers.
Every query must agree on `environmentId`, `branch` and `queryName`, or the call throws
`ClientException` — a request resolves one branch to one deployment and routes named queries by
an exact match on the query name, so a mixed batch could not be honored. Run those as separate
queries. Leaving one of these unset just takes the client's value, which is not a disagreement,
and `queryNameVersion` may differ freely since each query carries its own. The deadline is the
longest of the effective per-query timeouts, where a query that sets none is bounded by the
client-level timeout, so per-query timeouts are not independently enforced.
104 changes: 104 additions & 0 deletions src/main/java/ai/chalk/client/ChalkClient.java
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
package ai.chalk.client;

import ai.chalk.exceptions.ChalkException;
import ai.chalk.exceptions.ClientException;
import ai.chalk.models.*;

import java.net.http.HttpClient;
import java.nio.file.Path;
import java.time.Duration;
import java.util.Arrays;
import java.util.List;
import java.util.Optional;
import java.util.concurrent.ThreadLocalRandom;

Expand Down Expand Up @@ -129,6 +132,107 @@ default int ping() throws ChalkException {
*/
OnlineQueryResult onlineQuery(OnlineQueryParamsComplete params) throws ChalkException;

/**
* OnlineQueryMulti computes several independent online queries in parallel,
* using a single request to the Chalk engine. Use it when the queries are
* rooted in different feature classes, or otherwise ask for unrelated
* outputs, and you would rather not pay a round trip for each one. To run
* one query over many rows of input, use {@link #onlineQuery} instead,
* which is already a bulk operation.
*
* <p> This is supported only by the gRPC client; see {@link ChalkClient#createGrpc()}.
*
* <p>
* Example usage:
* <pre>
* {@code
* OnlineQueryParamsComplete users = OnlineQueryParams.builder()
* .withInput("user.id", Arrays.asList(1, 2, 3))
* .withOutputs("user.email")
* .build();
* OnlineQueryParamsComplete merchants = OnlineQueryParams.builder()
* .withInput("merchant.id", Arrays.asList("a", "b"))
* .withOutputs("merchant.risk_score")
* .build();
*
* try (OnlineQueryMultiResult multi = client.onlineQueryMulti(List.of(users, merchants))) {
* for (OnlineQueryResult result : multi.getResults()) {
* if (result.getErrors().length > 0) {
* // handle this query's failure; the other queries are unaffected
* continue;
* }
* // do something with the result
* }
* }
* }
* </pre>
* </p>
*
* <p> <b>Results are positional.</b> {@link OnlineQueryMultiResult#getResults()}
* holds one result per query, in the order the queries were passed in.
*
* <p> <b>Queries fail independently.</b> A resolver that fails while computing
* one query is reported on that query's own result, via
* {@link OnlineQueryResult#getErrors()}, and leaves the other queries' results
* intact. {@link OnlineQueryMultiResult#getGlobalErrors()} holds the errors the
* engine did not attach to a specific result -- including the case where a query
* failed to run at all, which is reported there with the query's 1-based position
* in the message rather than on the result. That query still occupies its slot,
* but its result is empty and reports no errors of its own, so check the global
* errors as well before treating an empty result as a successful one.
*
* <p> <b>Resources.</b> Closing the returned {@link OnlineQueryMultiResult}
* closes every one of its results and releases all of their Arrow memory.
* Do not close the individual results yourself.
*
* <p> <b>Some settings apply to the whole request.</b> A request carries one
* deadline and one set of headers, so:
* <ul>
* <li>Every query must agree on {@code environmentId}, {@code branch} and
* {@code queryName}. A disagreement throws
* {@link ai.chalk.exceptions.ClientException}, because a request resolves
* one branch to one deployment and routes named queries by an exact match
* on the query name, so a mixed batch could not be honored. Run those as
* separate queries. A query that leaves one of these unset simply takes the
* client's value, which is not a disagreement. {@code queryNameVersion} may
* differ freely: each query carries its own.</li>
* <li>The deadline is the longest of the effective per-query timeouts, where a
* query that sets none is bounded by the client-level timeout. Per-query
* timeouts are not independently enforced.</li>
* <li>Branch selection is currently inert on the gRPC path for both single and
* multi queries: the client does not yet send the header the API server
* routes branches on.</li>
* </ul>
*
* @param params the queries to run, one result per query
* @return {@link OnlineQueryMultiResult}
* @throws ChalkException if the request fails, or if the queries disagree on a
* setting that applies to the whole request
* @see <a href="https://docs.chalk.ai/docs/query-basics">query basics</a>
*/
default OnlineQueryMultiResult onlineQueryMulti(List<OnlineQueryParamsComplete> params)
throws ChalkException {
throw new ClientException(
"onlineQueryMulti is only supported by the gRPC client. Build one with "
+ "ChalkClient.builder().withGrpc().build() or ChalkClient.createGrpc()."
);
}

/**
* OnlineQueryMulti computes several independent online queries in parallel,
* using a single request to the Chalk engine.
*
* <p> See {@link #onlineQueryMulti(List)} for details and for the settings that
* every query must agree on.
*
* @return {@link OnlineQueryMultiResult}
* @throws ChalkException
*/
default OnlineQueryMultiResult onlineQueryMulti(OnlineQueryParamsComplete... params)
throws ChalkException {
return onlineQueryMulti(params == null ? null : Arrays.asList(params));
}

/**
* UploadFeatures synchronously persists feature values to the online store and
* offline store.
Expand Down
Loading
Loading