[Manushpm8]

The client could only reach a model over HTTP: a NIM, or a Databricks serving endpoint. Someone who already has structured-data-models loaded in a notebook had to call its tensor API directly, and then use a different API again once the model was hosted. SDMClient.in_process() runs the model in the caller's own Python process behind the same calls.

import sdm
from sdm_client import KumoRelational, SDMClient

with SDMClient.in_process(sdm.models.KumoRelational()) as client:
    result = client.predict(request, model=KumoRelational())

With no model passed, each one is loaded from its published weights on Hugging Face the first time it is asked for, on CUDA when available, else the CPU.

How it works

The client prepares every request the same way, whichever backend answers it: it checks the request against the model, and the model's adapter writes it out as the contract's request, meaning the task, schema, context, rows to score, outputs and inference settings, as Python data. It then hands that to a Backend (sdm_client/core/backend.py):

• NimBackend sends it to a NIM and reads the answer back: encode, HTTP, decode. It wraps the existing HTTP and Databricks transports, and owns what only a NIM needs, such as letting the adapter explain a NIM's refusal, and reading a 404 on the session routes as "no sessions here".
• InProcessBackend runs the model on it directly, with no JSON and no HTTP. It returns the client's own PredictionItems, which go through the same adapter step a NIM's decoded answer does to be put under the caller's keys. No response is built or parsed.

Sessions are backend operations too: create_session, session_predict and delete_session. So predictions, fit() sessions, rankings and forecasts work unchanged, and nothing above the backend knows where the model runs.

In process, the context is fitted as the hosted deployments fit it: seed 42, one estimator, and on a GPU their autocast precision, on a shallow copy of the model so a passed model is never disturbed. A request the model cannot answer is refused with a StructuredDataError whose code says why, from the codes a NIM uses, such as UNSUPPORTED_OUTPUT. It carries no HTTP status, since nothing was sent, and NimRequestError stays the NIM's.

What the in-process backend checks is what the model needs to read a request as it is stated: a column type the model has no reader for, a context with nothing to learn from, a relational task whose entity, anchor, target or joins differ from what the rows say. It does not re-check the request contract the way a NIM's HTTP layer does, field by field. A difference that matters to the model shows up in the shared tests below, and is fixed where it belongs: in the client's preparation, so neither backend is sent it, or in what the model needs to read a request.

Parity is the same tests on both backends

test_every_backend_answers_alike.py runs the same calls against both backends. In process it always runs, with small random-weight models. Against NIMs it runs under -m live_nim with SDM_TABULAR_URL and SDM_RELATIONAL_URL set. The values differ, since the weights do. Both are held to everything else a caller reads:

• which rows come back, under which keys, including an int64 key past 2**53 and a key of several columns;
• the columns, for tabular classification and regression with quantiles, and relational regression, binary and multiclass;
• that a session answers what one call answers;
• how an inference option is read: use_prediction_time spelled 'no', or given as a NumPy boolean, asks for nothing, and a value that cannot be read as its type is refused;
• the refusal code for an output the task does not produce, a bare LIST_DISTINCT, an inference option the model does not act on, a text column, a table named instance_table, and rows scored without their key;
• the refusal code for a relational request built by hand through Request.materialized that states another question than its rows: another entity, a task option the model ignores, an undeclared anchor, a target it cannot read, a table missing or undeclared, or a table that does not carry exactly the columns its schema declares.

All 33 pass on each backend, the NIM being the current SDM NIM code served locally on the CPU with the published weights. Running them, and the review rounds over this change, found seven real differences, each confirmed against the NIM and fixed here:

• A predictive query longer than 1,024 characters was refused by every NIM. The engine puts the query text in metadata.source_query for logging, and the contract caps that field at 1,024 characters, so a NIM answered 422 VALIDATION_FAILED. The relational adapter now cuts the logged copy to fit, so both backends answer it. This is a fix for NIM users too, and is in the changelog.
• A table named instance_table collides with the wire's own name for the rows scored, leaving a relationship that joins on a column the table does not declare. A NIM refuses it with INVALID_SCHEMA. In process it was answered, so the backend now refuses a join on an undeclared column, with the same code.
• Inference options of the wrong type were read differently. The tabular adapter forwarded a caller's options as given, and a NIM types them: run_mode: 7 and inference_config: None were refused by a NIM and answered in process, and use_prediction_time: 'no' the reverse. Both adapters now write the options in the contract's types before sending, with the coercion a NIM applies, and refuse a value that cannot be read with INVALID_REQUEST. For a NIM user, such a value is now refused before it is sent, rather than with 422 VALIDATION_FAILED.
• A key of several columns came back with no id in process. A NIM answers it under the JSON of the key's values, which the client hands back as the row's id, so in process now gives the same id.
• Rows scored without their key failed in process with INTERNAL_ERROR, after the model had run. A NIM refuses them with INVALID_SCHEMA, and in process now does too, before the model runs.
• A cell that cannot be read as its type failed in process with INTERNAL_ERROR when it sat in a key column or held an integer past the int64 range. A NIM refuses it with VALIDATION_FAILED, and in process now does too.
• A row with a cell too many or too few was VALIDATION_FAILED in process and is INVALID_SCHEMA from a NIM. In process now gives the NIM's code.
• A table that did not carry exactly its declared columns was read anyway in process: a declared column left out was fitted without, an undeclared one read as nothing, a repeated one read once. A NIM refuses each, and in process now does too, with the NIM's code.
• A session kept the request's own task and schema, both in the in-process backend and in the client's handle, so changing the request after fit() changed the session under it. Both now keep a copy, as a NIM does.

Values match the NIM exactly on the same device

Served the same way, with the same published weights and structured-data-models 0.2.1, the two give identical predictions, probabilities and quantiles: the largest difference is 0.0 across 41 comparisons, with no missing value on either side. Those comparisons cover:

	Cases
kumo-tabular	binary (int labels), multiclass (string labels), regression with quantiles
kumo-relational	count and sum regressions with quantiles, binary, multiclass, a bool column, a forecast
through the client	RANK TOP 3, a session scored twice
real data	Adult, Wine (multiclass and regression), Titanic, Bank Marketing, Students (kumo-tabular); Northwind counts, sums, binary and integer regression (kumo-relational), one-shot and in a session

qa/probe_in_process.py repeats the real-data comparison against any NIM. A value missing on one side only fails it. On the same device it needs no tolerance. With this process on a CPU and the NIM on a GPU, set SDM_IN_PROCESS_TOLERANCE.

On an NVIDIA L4, the NIM with its bfloat16 GPU profile and in-process on the same GPU give identical answers: 11 of 11 comparisons, one-shot and in a session, with a largest difference of 0.0.

Tests

test_a_model_runs_in_this_process.py covers what only a model in this process has. It uses small random-weight models, so it needs no network and no GPU:

• A tabular answer equals what the model returns when called directly through its own tensor API, for int labels, string labels and regression. This is exact equality, not a tolerance.
• Rankings and forecasts are answered in process.
• A refusal is a StructuredDataError with a code, never a NimRequestError, and has no status: for what a holdout leaves behind, a table the model cannot read, and a task the model was not loaded for.
• A session keeps its own copy of what it pinned.
• A context the caller fitted on their own model survives.
• A model is loaded once, on first use, on the default device. A load that fails is MODEL_NOT_READY, with the original error as its cause.
• Configuration mistakes are refused when the client is opened, and a model left in training mode is named.
• Without the extra, in_process() raises MissingExtraError naming in-process. This is checked in a fresh interpreter with structured-data-models, PyTorch and PyArrow all absent.

Apart from the long-query fix and the typing of inference options, the NIM path behaves as before. Its tests are unchanged too, except that each one handing the client a fake transport now hands it to NimBackend. CI runs the in-process tests in the engine job, which installs CPU PyTorch and the in-process extra. The other jobs skip them.

Locally: the client suite (2007 passed) and the root suite (255 passed) pass. ruff, mypy and hygiene are clean. Against local NIMs, the existing live suites (31 passed) and both new test files on both backends (82 passed) pass.

Worth a look

• CI installs the wheels it built. structured-data-models-client 1.0.0 reached PyPI while this was in review. After that, --find-links dist with ==1.0.0 resolved the published release instead of the wheel CI had just built. installed-wheel, sbom and the build dry-run now name each wheel file, and installed-wheel asserts that all four distributions came from dist/. This affects every pull request against the client from now on.
• The hygiene rule changes. It refused the bare name structured-data-models as an internal build a reader cannot look up. That package is now public, and it is what the new extra installs, so the rule now covers only the internal build name it was written for.
• Text, multicategorical and sequence columns are refused, as the current NIM refuses them. The client's warning at materialize() still says those columns are dropped, which is now wrong against both. That is left for a follow-up.
• The extra needs Python 3.11, because structured-data-models does. It carries a python_version >= '3.11' marker, without which the workspace lock cannot resolve, so on 3.10 the extra installs nothing and in_process() says it needs Python 3.11.
• uv.lock grows. It resolves every extra, so it gains structured-data-models, PyTorch, and PyTorch's CUDA wheels for Linux. Nothing already in it moves. Main's own lock is already behind its pyproject since Build the relational engine for macOS, Windows and 3.14 #42 added Python 3.14; this change leaves that to main rather than mixing a full relock in here.
• A deployment's protective limits do not apply in process. These are the limits a NIM sets per deployment through TFM_MAX_*, such as string-cell bytes and distinct values per column. They protect a shared server. The limits this client checks before sending still apply, and docs/reference/limits.md says so.
• Inside a session, a refusal keeps its own code. A NIM files a refusal while pinning or scoring against a session under SESSION_CREATE_VALIDATION_FAILED or SESSION_PREDICT_SCHEMA_MISMATCH. In process, it keeps the code of the reason, such as TASK_SCHEMA_MISMATCH. Reproducing the NIM's session codes would be the route imitation this change removes, and the quickstart says so.
• A request is checked for what the model needs to read it, not against every rule of the contract. A name longer than the contract's 1,024 characters, or a key the contract does not name at the top level of a payload passed to Request.materialized, is answered in process, where a NIM's request model refuses it with VALIDATION_FAILED. Re-checking each contract format in process is the imitation this change removes. The quickstart says so.

Follow-ups, not in this change:

• A table named instance_table should be refused when the request is gathered, with a message naming the table, rather than by whichever backend reads it.
• request.py says rows to score may leave out their key, and test_entity_id_parity.py tests that by normalizing a made-up response. A NIM refuses such rows with 422 INVALID_SCHEMA, and so does in process. That predates this change, so it is not changed here.

CPU against GPU. On a CPU, in-process computes at full precision and draws its random numbers on the CPU. A GPU-hosted NIM does neither, so the two can differ by more than rounding: on the L4, a class probability moved by 0.16 and a regression by 0.41. The quickstart gives that figure. On the same device, the two are identical.

🤖 Generated with Claude Code

[JGagacheva]

Six comments inline. None is blocking. The output-option check, the lock and the hygiene regex matter most.

[JGagacheva]

Five comments inline. None is blocking. Only the use_prediction_time question in relational.py needs an answer before merging.

[JGagacheva]

@Manushpm8 I added some comments but I think they're minor cleanup details/checks. I'll approve and let you decide how to handle those