Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@
The home baked data pipeline for **Western Formula Racing**.

This package handles:
1. **Data Ingestion:** Reliable fetching from InfluxDB 3.0 in wide (columnar) or narrow (legacy EAV) format.
2. **Data Writing:** `WideWriter` encodes CAN frames directly to InfluxDB wide format line protocol.
1. **Data Ingestion:** Reliable fetching from TimescaleDB 3.0 in wide (columnar) or narrow (legacy EAV) format.
2. **Data Writing:** `WideWriter` encodes CAN frames directly to TimescaleDB wide format line protocol.
3. **Movement Detection:** Smart filtering of "Moving" vs "Idle" car states.
4. **Sensor Discovery:** Tools to explore available sensors on any given race day.

Expand All @@ -34,7 +34,7 @@ import slicks
from datetime import datetime

# 1. Connect (auto-configured from env vars or explicit)
slicks.connect_influxdb3(db="WFR26")
slicks.connect_timescaledb(table="WFR26")

# 2. Fetch Data — wide format (columnar, preferred)
df = slicks.fetch_telemetry(
Expand Down
14 changes: 7 additions & 7 deletions docs/advanced_usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,17 +42,17 @@ You often need to switch between `Development`, `Testing`, and `Production` data
### Option A: Environment Variables (Best for CI/CD)
Set these in your shell or `.env` file before running python:
```bash
export INFLUX_URL="http://production-server:8086"
export INFLUX_DB="Season2026_Final"
export POSTGRES_DSN="http://production-server:8086"
export POSTGRES_TABLE="Season2026_Final"
```

### Option B: Runtime Configuration (Best for Scripts/Notebooks)
```python
import slicks

slicks.connect_influxdb3(
url="http://192.168.1.50:9000",
db="DynoTest_Day1"
slicks.connect_timescaledb(
dsn="http://192.168.1.50:9000",
table="DynoTest_Day1"
)
```

Expand All @@ -74,8 +74,8 @@ If you're ingesting raw CAN bus data (e.g., from a replay script or live logger)
from slicks import WideWriter

writer = WideWriter(
url="http://localhost:8086",
token="my-token",
dsn="http://localhost:8086",
"my-token",
bucket="WFR26",
measurement="WFR26",
dbc_path="path/to/WFR26.dbc",
Expand Down
16 changes: 8 additions & 8 deletions docs/api_reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@ This document details the functions available in the `slicks` package.

## Core Functions

### `slicks.connect_influxdb3`
### `slicks.connect_timescaledb`

Updates the global InfluxDB connection settings dynamically.
Updates the global TimescaleDB connection settings dynamically.

```python
slicks.connect_influxdb3(url=None, token=None, org=None, db=None)
slicks.connect_timescaledb(dsn=None, None, org=None, table=None)
```
- **url** *(str)*: The InfluxDB host URL (e.g., `"http://localhost:8086"`).
- **url** *(str)*: The TimescaleDB host URL (e.g., `"http://localhost:8086"`).
- **token** *(str)*: Authentication token.
- **org** *(str)*: Organization name (default: `"Docs"`).
- **db** *(str)*: Database/Bucket name (default: `"WFR25"`).
Expand All @@ -30,7 +30,7 @@ slicks.fetch_telemetry(start_time, end_time, signals=None, client=None,
- **start_time** *(datetime)*: Start of the query range.
- **end_time** *(datetime)*: End of the query range.
- **signals** *(str or list[str])*: A single sensor name or a list of sensor names to fetch. Defaults to standard configuration if `None`.
- **client** *(InfluxDBClient3, optional)*: An existing client instance (advanced use).
- **client** *(TimescaleDBClient3, optional)*: An existing client instance (advanced use).
- **filter_movement** *(bool)*: If `True` (default), strips out rows where the car is stationary.
- **resample** *(str or None)*: Pandas frequency string for resampling (e.g. `"1s"`, `"100ms"`). Pass `None` for raw data.
- **schema** *(str)*: `"wide"` (default, columnar — each signal is a column) or `"narrow"` (legacy EAV — requires pivot).
Expand Down Expand Up @@ -105,13 +105,13 @@ slicks.detect_movement_ratio(df, speed_column="INV_Motor_Speed")

### `slicks.WideWriter`

Encodes CAN frames to InfluxDB wide format line protocol and writes them in batches.
Encodes CAN frames to TimescaleDB wide format line protocol and writes them in batches.

```python
from slicks import WideWriter

writer = WideWriter(
url, # InfluxDB URL
url, # TimescaleDB URL
token, # Auth token
bucket, # Bucket/database name (e.g. "WFR26")
measurement, # Measurement name (e.g. "WFR26")
Expand Down Expand Up @@ -154,7 +154,7 @@ frame = decode_frame(db, can_id, raw_bytes) # → DecodedFrame or None

### `slicks.frame_to_line_protocol`

Converts a `DecodedFrame` to an InfluxDB line protocol string.
Converts a `DecodedFrame` to an TimescaleDB line protocol string.

```python
from slicks import frame_to_line_protocol
Expand Down
10 changes: 5 additions & 5 deletions docs/getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,18 +27,18 @@ pip install -e .
Here is the minimal code needed to connect to the database and download data for a specific sensor.

### Step 1: Import and Configure
The package connects to the InfluxDB database automatically using defaults, but you can configure it explicitly.
The package connects to the TimescaleDB database automatically using defaults, but you can configure it explicitly.

```python
import slicks
from datetime import datetime

# Optional: Configure manually (or use .env file / defaults)
slicks.connect_influxdb3(
url="http://your-influx-server:8086",
token="your-token-here", # Ask Data Lead for your token
slicks.connect_timescaledb(
dsn="http://your-postgres-server:8086",
"your-token-here", # Ask Data Lead for your token
org="Docs",
db="WFR25"
table="WFR25"
)
```

Expand Down
4 changes: 2 additions & 2 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ The home baked data pipeline for **Western Formula Racing**.

This package handles:

1. **Data Ingestion:** Reliable fetching from InfluxDB 3.0.
1. **Data Ingestion:** Reliable fetching from TimescaleDB 3.0.

2. **Movement Detection:** Smart filtering of "Moving" vs "Idle" car states.

Expand All @@ -29,7 +29,7 @@ import slicks
from datetime import datetime

# 1. Connect (Auto-configured or custom)
slicks.connect_influxdb3(db="WFR25", influx_url="http://influxdb:9000", influx_token="apiv3_your_token")
slicks.connect_timescaledb(table="WFR25", dsn="http://postgresdb:9000", "apiv3_your_token")

# 2. Fetch Data (One-liner)
df = slicks.fetch_telemetry(
Expand Down
10 changes: 5 additions & 5 deletions docs/scanner.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ import slicks
from datetime import datetime

# Configure connection first
slicks.connect_influxdb3(
url="http://your-server:9000",
token="your-token",
db="WFR25"
slicks.connect_timescaledb(
dsn="http://your-server:9000",
"your-token",
table="WFR25"
)

# Scan for data availability
Expand Down Expand Up @@ -73,7 +73,7 @@ slicks.scan_data_availability(
| `start` | `datetime` | *required* | Start of scan range (UTC or timezone-aware) |
| `end` | `datetime` | *required* | End of scan range |
| `timezone` | `str` | `"UTC"` | Timezone for display (e.g., `"America/Toronto"`) |
| `table` | `str` | `None` | Table to scan (defaults to `"iox.{INFLUX_DB}"`) |
| `table` | `str` | `None` | Table to scan (defaults to `"iox.{POSTGRES_TABLE}"`) |
| `bin_size` | `str` | `"hour"` | Granularity: `"hour"` or `"day"` |
| `include_counts` | `bool` | `True` | Include row counts (slightly slower if `True`) |
| `show_progress` | `bool` | `True` | Show progress bar during scan |
Expand Down
8 changes: 4 additions & 4 deletions examples/end_to_end.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@ def main():
# Configure connection.
# For CI, these are pulled from environment variables (GitHub Secrets).
# For local use, you can set these in a .env file or call configure() directly.
slicks.connect_influxdb3(
url=os.getenv("INFLUX_URL"),
token=os.getenv("INFLUX_TOKEN"),
slicks.connect_timescaledb(
dsn=os.getenv("POSTGRES_DSN"),
os.getenv(""),
org=os.getenv("INFLUX_ORG"),
db=os.getenv("INFLUX_DB")
table=os.getenv("POSTGRES_TABLE")
)

# ---------------------------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "slicks"
version = "0.2.3"
version = "0.3.0"
description = "The home baked data pipeline for Western Formula Racing"
readme = "README.md"
authors = [
Expand Down
Loading