Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -310,6 +310,41 @@ still return a valid object of the declared type. For example, an empty `SampleS
UDF should return a valid, empty `SampleSeries`.
:::

#### Rules for correct results

When a UDF receives one or more `SampleSeries`, follow these rules:

- **Handle empty inputs.** On the planning call above, every series argument is empty. Indexing
(`series.values[0]`, `series.tstarts[-1]`) raises on empty input. Guard with
`if len(series) == 0:` and return an empty result of the UDF's return type: `SampleSeries.empty()`
for a series, `np.nan` for a number. The built-in reducers (`sum()`, `min()`, `max()`, `mean()`)
already return NaN for an empty series.
- **Return the same type on every path.** The type returned on the planning call determines the
Spark column type of the result. A UDF whose return type depends on the data (for example, a
`SampleSeries` on one branch and a float on another) produces values that do not fit that column.
- **Respect gaps.** A series may contain gaps. Two consecutive samples are contiguous only if
`tstarts[i + 1] == tends[i]` holds exactly. Compute neighbour-based logic (differences, gradients,
cumulative sums) per contiguous segment using `series.continuous_interval_indices`, a list of
inclusive `(start, end)` index pairs. The built-in `diff()`, `cumtrapz()` and `resample()` work this
way.
- **Synchronize before combining the raw arrays of several series.** Different channels have
different timestamps, so element-wise numpy operations on `a.values` and `b.values` pair unrelated
samples (or raise on a shape mismatch). TSAL operators (`a + b`, `a > b`) synchronize
automatically. Inside a UDF, call `a.synchronized(b)` or `a.synchronized_all([b, c])`: they return
the series (the calling series first) on shared intervals covering only their overlap, so the
result may be shorter than the inputs, or empty.

```python
import numpy as np

@TimeSeriesExpression.udf
def magnitude(x, y):
xs, ys = x.synchronized(y)
return SampleSeries(xs.tstarts, xs.tends, np.hypot(xs.values, ys.values))

speed = magnitude(vel_x, vel_y)
```

---

## Virtual signals
Expand Down
37 changes: 37 additions & 0 deletions skills/impulse-tsal/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,14 @@ three `SampleSeries` (min, max, average).

**User-defined functions**

**Prefer native functionality.** Before writing a UDF, check whether the request can be expressed
with native building blocks: the operators and signal methods above, the `Intervals` methods below,
the event types (`impulse-events`), the aggregation classes (`impulse-aggregations`) and calculated
channels (`impulse-channels`). Write a UDF only for the part native functionality cannot express,
and keep everything else native. Native operations already handle empty inputs, gaps and unaligned
channels (see the rules below). Future Impulse versions may also optimize the execution of native
functionality, while logic inside a UDF stays opaque to the engine.

```python
from impulse_query_engine.model.series import SampleSeries
from impulse_query_engine.analyze.metadata.time_series_expression import TimeSeriesExpression
Expand Down Expand Up @@ -161,6 +169,35 @@ derated = derate(power)
- Values are constant per container. The declaration **propagates** when the UDF is wrapped in an
aggregation, event, or calculated channel — the wrapper does not redeclare anything.

**Rules for correct UDF results** (when an input is a `SampleSeries`):

- **Handle empty inputs.** Every UDF runs once at planning time, with every series input empty (and
every declared container tag/metric `None`), so the engine can infer the result type. Indexing
(`series.values[0]`, `series.tstarts[-1]`) fails on empty input. Guard with `if len(series) == 0:`
and return an empty result of the UDF's return type: `SampleSeries.empty()` for a series, `np.nan`
for a number. The built-in reducers (`sum()`, `min()`, `max()`, `mean()`) already return NaN.
- **Return the same type on every path.** The type returned on the planning call fixes the output
column type; a return type that depends on the data (a `SampleSeries` on one branch, a float on
another) produces values that do not fit that column.
- **Respect gaps.** A series may contain gaps; two samples are contiguous only if
`tstarts[i+1] == tends[i]` exactly. Compute neighbour-based logic (differences, gradients,
cumulative sums) per segment using `series.continuous_interval_indices` (inclusive `(start, end)`
index pairs), as `diff()`, `cumtrapz()` and `resample()` do.
- **Synchronize before combining raw arrays of several series.** Channels have different timestamps,
so element-wise numpy on `a.values` and `b.values` pairs unrelated samples (or raises on a shape
mismatch). Operators (`a + b`, `a > b`) synchronize automatically; inside a UDF call
`a.synchronized(b)` or `a.synchronized_all([b, c])`, which return the series (self first) on shared
intervals covering only their overlap, so the result may be shorter or empty.

```python
import numpy as np

@TimeSeriesExpression.udf
def magnitude(x, y):
xs, ys = x.synchronized(y)
return SampleSeries(xs.tstarts, xs.tends, np.hypot(xs.values, ys.values))
```

## What an expression evaluates to (the core data model)

Every expression resolves — per container — to one of four in-memory types. Knowing the type tells you
Expand Down
8 changes: 8 additions & 0 deletions skills/impulse/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,14 @@ report.add_page(page)
report.run() # compute and write the gold star schema
```

## Prefer native functionality

Build solutions from Impulse's native building blocks (TSAL operators and methods, event types,
aggregation classes, calculated channels) and fall back to a user-defined function only for logic
they cannot express. Native operations handle edge cases such as empty inputs, gaps and unaligned
channels, and future Impulse versions may also optimize their execution, while logic inside a UDF
stays opaque to the engine. See **`impulse-tsal`** → User-defined functions.

## Where to go next

- Building signal expressions → **`impulse-tsal`**
Expand Down
Loading