diff --git a/docs/impulse/docs/references/query_engine/tsal/defining_expressions.md b/docs/impulse/docs/references/query_engine/tsal/defining_expressions.md index f57b9756..dda8ed93 100644 --- a/docs/impulse/docs/references/query_engine/tsal/defining_expressions.md +++ b/docs/impulse/docs/references/query_engine/tsal/defining_expressions.md @@ -310,6 +310,41 @@ still return a valid object of the declared type. For example, an empty `SampleS UDF should return a valid, empty `SampleSeries`. ::: +#### Rules for correct results + +When a UDF receives one or more `SampleSeries`, follow these rules: + +- **Handle empty inputs.** On the planning call above, every series argument is empty. Indexing + (`series.values[0]`, `series.tstarts[-1]`) raises on empty input. Guard with + `if len(series) == 0:` and return an empty result of the UDF's return type: `SampleSeries.empty()` + for a series, `np.nan` for a number. The built-in reducers (`sum()`, `min()`, `max()`, `mean()`) + already return NaN for an empty series. +- **Return the same type on every path.** The type returned on the planning call determines the + Spark column type of the result. A UDF whose return type depends on the data (for example, a + `SampleSeries` on one branch and a float on another) produces values that do not fit that column. +- **Respect gaps.** A series may contain gaps. Two consecutive samples are contiguous only if + `tstarts[i + 1] == tends[i]` holds exactly. Compute neighbour-based logic (differences, gradients, + cumulative sums) per contiguous segment using `series.continuous_interval_indices`, a list of + inclusive `(start, end)` index pairs. The built-in `diff()`, `cumtrapz()` and `resample()` work this + way. +- **Synchronize before combining the raw arrays of several series.** Different channels have + different timestamps, so element-wise numpy operations on `a.values` and `b.values` pair unrelated + samples (or raise on a shape mismatch). TSAL operators (`a + b`, `a > b`) synchronize + automatically. Inside a UDF, call `a.synchronized(b)` or `a.synchronized_all([b, c])`: they return + the series (the calling series first) on shared intervals covering only their overlap, so the + result may be shorter than the inputs, or empty. + +```python +import numpy as np + +@TimeSeriesExpression.udf +def magnitude(x, y): + xs, ys = x.synchronized(y) + return SampleSeries(xs.tstarts, xs.tends, np.hypot(xs.values, ys.values)) + +speed = magnitude(vel_x, vel_y) +``` + --- ## Virtual signals diff --git a/skills/impulse-tsal/SKILL.md b/skills/impulse-tsal/SKILL.md index 69888d25..5e469f56 100644 --- a/skills/impulse-tsal/SKILL.md +++ b/skills/impulse-tsal/SKILL.md @@ -129,6 +129,14 @@ three `SampleSeries` (min, max, average). **User-defined functions** +**Prefer native functionality.** Before writing a UDF, check whether the request can be expressed +with native building blocks: the operators and signal methods above, the `Intervals` methods below, +the event types (`impulse-events`), the aggregation classes (`impulse-aggregations`) and calculated +channels (`impulse-channels`). Write a UDF only for the part native functionality cannot express, +and keep everything else native. Native operations already handle empty inputs, gaps and unaligned +channels (see the rules below). Future Impulse versions may also optimize the execution of native +functionality, while logic inside a UDF stays opaque to the engine. + ```python from impulse_query_engine.model.series import SampleSeries from impulse_query_engine.analyze.metadata.time_series_expression import TimeSeriesExpression @@ -161,6 +169,35 @@ derated = derate(power) - Values are constant per container. The declaration **propagates** when the UDF is wrapped in an aggregation, event, or calculated channel — the wrapper does not redeclare anything. +**Rules for correct UDF results** (when an input is a `SampleSeries`): + +- **Handle empty inputs.** Every UDF runs once at planning time, with every series input empty (and + every declared container tag/metric `None`), so the engine can infer the result type. Indexing + (`series.values[0]`, `series.tstarts[-1]`) fails on empty input. Guard with `if len(series) == 0:` + and return an empty result of the UDF's return type: `SampleSeries.empty()` for a series, `np.nan` + for a number. The built-in reducers (`sum()`, `min()`, `max()`, `mean()`) already return NaN. +- **Return the same type on every path.** The type returned on the planning call fixes the output + column type; a return type that depends on the data (a `SampleSeries` on one branch, a float on + another) produces values that do not fit that column. +- **Respect gaps.** A series may contain gaps; two samples are contiguous only if + `tstarts[i+1] == tends[i]` exactly. Compute neighbour-based logic (differences, gradients, + cumulative sums) per segment using `series.continuous_interval_indices` (inclusive `(start, end)` + index pairs), as `diff()`, `cumtrapz()` and `resample()` do. +- **Synchronize before combining raw arrays of several series.** Channels have different timestamps, + so element-wise numpy on `a.values` and `b.values` pairs unrelated samples (or raises on a shape + mismatch). Operators (`a + b`, `a > b`) synchronize automatically; inside a UDF call + `a.synchronized(b)` or `a.synchronized_all([b, c])`, which return the series (self first) on shared + intervals covering only their overlap, so the result may be shorter or empty. + +```python +import numpy as np + +@TimeSeriesExpression.udf +def magnitude(x, y): + xs, ys = x.synchronized(y) + return SampleSeries(xs.tstarts, xs.tends, np.hypot(xs.values, ys.values)) +``` + ## What an expression evaluates to (the core data model) Every expression resolves — per container — to one of four in-memory types. Knowing the type tells you diff --git a/skills/impulse/SKILL.md b/skills/impulse/SKILL.md index 0b5064b6..d1f03f4e 100644 --- a/skills/impulse/SKILL.md +++ b/skills/impulse/SKILL.md @@ -132,6 +132,14 @@ report.add_page(page) report.run() # compute and write the gold star schema ``` +## Prefer native functionality + +Build solutions from Impulse's native building blocks (TSAL operators and methods, event types, +aggregation classes, calculated channels) and fall back to a user-defined function only for logic +they cannot express. Native operations handle edge cases such as empty inputs, gaps and unaligned +channels, and future Impulse versions may also optimize their execution, while logic inside a UDF +stays opaque to the engine. See **`impulse-tsal`** → User-defined functions. + ## Where to go next - Building signal expressions → **`impulse-tsal`**