Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,14 @@ This changelog covers these gems, which release together on the same version:
A `Tooling` section records changes to the checks and scripts in `bin/` and `examples/`, which ship in no
gem but are what an operator runs against their own image.

## next / unreleased

### Yabeda::HotCell

#### Fixed

* `perform` records nothing for a call whose response has no `perform_ms`: a call that the cell refused with `capacity`, or that the client failed itself with `unavailable` or `timeout`. Previously, it recorded 0 seconds, which pulled down the low percentiles. (#112)

## v1.1.0 / 2026-10-08

### HotCell::Server
Expand Down
2 changes: 1 addition & 1 deletion docs/observability.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ The metrics are in the `hotcell` group:
| Metric | Type | Tags | Description |
| --- | --- | --- | --- |
| `requests` | counter | `cell`, `operation`, `code`, `cause` | Each call. `code` is `ok` on success, and `cause` is empty when there's none. |
| `perform` | histogram | `cell`, `operation` | Seconds that the cell spent in `perform`. |
| `perform` | histogram | `cell`, `operation` | Seconds that the cell spent in `perform`. A call whose response has no `perform_ms`, such as `capacity`, `unavailable` or `timeout`, records nothing. |
| `up` | gauge | `cell` | 1 when the local cell answers its control socket, otherwise 0. |
| `running` | gauge | `cell` | Workers busy right now. |
| `queued` | gauge | `cell` | Connections waiting for a worker. |
Expand Down
6 changes: 5 additions & 1 deletion yabeda-hotcell/lib/yabeda/hot_cell.rb
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,11 @@ def self.record_perform(event)
# and a query by code would silently split.
Yabeda.hotcell.requests.increment(labels.merge(code: event.payload[:code] || "ok",
cause: event.payload[:cause].to_s))
Yabeda.hotcell.perform.measure(labels, (event.payload[:perform_ms] || 0) / 1000.0)

# The response has no perform_ms when the cell refuses the call with `capacity`, or when the client fails
# the call itself with `unavailable` or `timeout`. Recording 0 would pull down the low percentiles.
perform_ms = event.payload[:perform_ms]
Yabeda.hotcell.perform.measure(labels, perform_ms / 1000.0) if perform_ms
end

private_class_method def self.set_counters(cell, counters)
Expand Down
8 changes: 8 additions & 0 deletions yabeda-hotcell/test/perform_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,14 @@ def test_counts_a_failure_under_its_own_code
assert_equal 1, requests(code: "capacity", cause: "")
end

def test_records_no_perform_time_when_the_cell_refuses_a_call_with_capacity
answer HotCell::Response.failed(HotCell::Failure.new(code: "capacity", message: "no"), timing: { queued_ms: 0 })

assert_raises(TemporarilyUnavailable) { Anything.perform_in_hotcell [], [], {} }

assert_nil perform_seconds
end

# `killed` is one code and several verdicts: which limit the worker hit decides whether the file did it.
# Every other code carries an empty cause, because a label that is sometimes absent is a separate series in
# Prometheus, and a query by code would silently split.
Expand Down
Loading