Skip to content

Add -n/--tail support to lstk logs#359

Open
gtsiolis wants to merge 3 commits into
mainfrom
pro-358-support-n-tail-n-in-lstk-logs
Open

Add -n/--tail support to lstk logs#359
gtsiolis wants to merge 3 commits into
mainfrom
pro-358-support-n-tail-n-in-lstk-logs

Conversation

@gtsiolis

@gtsiolis gtsiolis commented Jul 2, 2026

Copy link
Copy Markdown
Member

Motivation

DevRel parity report: the Python CLI supports localstack logs -n 20 to inspect the last N log lines, while lstk logs only offered all-or-follow — cumbersome on a busy emulator and a migration papercut. -n, --tail (default all) is also the exact convention of docker logs, docker compose logs, and nerdctl logs.

Changes

  • Add -n, --tail to lstk logs (string, default "all", wording identical to docker logs); validated at the command boundary — non-negative integer or all
  • Thread the value through container.Logs / ui.RunLogs into runtime.StreamLogs, which previously hardcoded Tail: "all"
  • Regenerate MockRuntime for the StreamLogs signature change

--tail N counts the lines lstk prints

lstk logs filters noisy loggers (request logs, internal handlers) before printing. Handing the limit straight to the runtime would count lines that the filter then drops, so --tail 1 printed nothing whenever the newest raw line happened to be a filtered one — a burst of request logs was enough to swallow the whole limit.

So the limit is now applied after filtering:

  • Non-verbose: read a raw suffix, filter it, keep the last N survivors. shouldFilter is a pure, order-preserving predicate, so the last N survivors of a long-enough suffix are the last N survivors overall.
  • Rather than reading the container's whole history to show a few lines, the suffix starts at N × 8 raw lines and grows geometrically, only when it turned out to be too heavily filtered (falling back to the full history past 100k).
  • Verbose (-v): every line is printed, so the runtime's own tail is already exact — delegated to it, which is also far cheaper.
  • --follow: the filtered backlog is emitted first, then the live stream starts at tail=0.

One trade-off worth a look: the follow path now issues two runtime calls, so lines written in the sub-millisecond gap between them aren't displayed. Making that exact needs Timestamps: true plus Since, which means parsing and stripping a timestamp prefix off every line — happy to add it if we'd rather not have the gap. It's called out in a comment at the call site.

Tests

  • Unit tests for the tail semantics, verified red-green against the pre-fix path (both regression tests fail with 0 lines emitted, reproducing the reported symptom): limit vs. filtered burst, suffix growth past a long filtered run, ordering and capping at the visible count, --tail 0, and verbose delegating to the runtime
  • Integration tests: --tail/-n limit output, --tail 1 survives a burst of filtered lines, default shows all lines, invalid values are rejected with a helpful error, and --follow --tail starts from the tail
  • Manually verified against a real container in both plain and TUI (PTY) modes, including --tail 0, --tail all, over-count values, and combined -fn 3

Closes PRO-358

@gtsiolis gtsiolis added semver: minor docs: needed Pull request requires documentation updates labels Jul 2, 2026
@gtsiolis gtsiolis self-assigned this Jul 2, 2026
@gtsiolis
gtsiolis force-pushed the pro-358-support-n-tail-n-in-lstk-logs branch 4 times, most recently from cff248a to b3e7ad8 Compare July 9, 2026 07:11
@gtsiolis
gtsiolis force-pushed the pro-358-support-n-tail-n-in-lstk-logs branch 4 times, most recently from 80c342e to 52b13c3 Compare July 17, 2026 13:47
@gtsiolis
gtsiolis marked this pull request as ready for review July 17, 2026 13:48
@gtsiolis
gtsiolis requested a review from a team as a code owner July 17, 2026 13:48
@gtsiolis
gtsiolis force-pushed the pro-358-support-n-tail-n-in-lstk-logs branch 2 times, most recently from a8809b2 to a3a8697 Compare July 21, 2026 07:08

@skyrpex skyrpex left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tail value is computed before filtering, so the output is confusing in some scenarios. For example, when the latest log entries are of type INFO:

$ ./bin/lstk logs --verbose
emulator |
emulator | LocalStack version: 2026.6.3
emulator | LocalStack build date: 2026-07-15
emulator | LocalStack build git hash: 7bf63b7c4
emulator |
emulator | 2026-07-21T12:19:57.291  INFO --- [  MainThread] l.p.c.b.licensingv2        : Successfully activated cached license cc9b6903-7df5-4afb-9d50-f27d7c
           9a4cf5:enterprise from /var/lib/localstack/cache/license.json 🔑✅
emulator | 2026-07-21T12:20:00.115  INFO --- [  MainThread] l.p.c.extensions.plugins   : loaded 0 extensions
emulator | Ready.
emulator | 2026-07-21T12:20:02.016  INFO --- [et.reactor-2] localstack.request.http    : GET /_localstack/pods/state/metamodel => 200
emulator | 2026-07-21T12:20:02.017  INFO --- [et.reactor-1] localstack.request.http    : GET /_localstack/pods/state/metamodel => 200
emulator | 2026-07-21T12:20:02.017  INFO --- [et.reactor-0] localstack.request.http    : GET /_localstack/pods/state/metamodel => 200
emulator | 2026-07-21T12:20:02.024  INFO --- [et.reactor-1] localstack.request.http    : GET /_localstack/pods/state/metamodel => 200
emulator | 2026-07-21T12:20:02.025  INFO --- [et.reactor-2] localstack.request.http    : GET /_localstack/pods/state/metamodel => 200
emulator | 2026-07-21T12:20:02.026  INFO --- [et.reactor-0] localstack.request.http    : GET /_localstack/pods/state/metamodel => 200

For reference, this is the $ ./bin/lstk logs output:

emulator |
emulator | LocalStack version: 2026.6.3
emulator | LocalStack build date: 2026-07-15
emulator | LocalStack build git hash: 7bf63b7c4
emulator |
emulator | 2026-07-21T12:19:57.291  INFO --- [  MainThread] l.p.c.b.licensingv2        : Successfully activated cached license cc9b6903-7df5-4afb-9d50-f27d7c
           9a4cf5:enterprise from /var/lib/localstack/cache/license.json 🔑✅
emulator | 2026-07-21T12:20:00.115  INFO --- [  MainThread] l.p.c.extensions.plugins   : loaded 0 extensions
emulator | Ready.

Now, if I run ./bin/lstk logs --tail 1 the output is empty as it's tailed on the full set first, and then filtered afterwards. What I expect, as a user, is the following:

$ ./bin/lstk logs --tail 1
emulator | Ready.

@gtsiolis
gtsiolis force-pushed the pro-358-support-n-tail-n-in-lstk-logs branch from a3a8697 to e535afa Compare July 21, 2026 12:52
@gtsiolis

gtsiolis commented Jul 21, 2026

Copy link
Copy Markdown
Member Author

Thanks for taking a closer look, @skyrpex! Added e535afa to fix tail filtering. Updated also PR description.

@gtsiolis
gtsiolis requested a review from skyrpex July 21, 2026 12:52
@gtsiolis

Copy link
Copy Markdown
Member Author

250f218 is unrelated to the tail fix, so a quick note on why it's in here.

While verifying the tail change, a test in this package wedged for the full 10 minute timeout instead of failing. The log stream goroutine called pw.CloseWithError as a plain statement, so when it dies without returning normally, a panic or a test double calling t.Fatal, the read loop blocks on a writer that never writes.

No behavior change, and I couldn't reproduce it against the real Docker path, so it isn't a user-facing bug.

I'd still keep it here because this PR made the StreamLogs call count variable, one call, two with --follow, more when the over-fetch grows. That makes a mock mismatch easier to write, and it used to cost 10 minutes of CI. The same scenario now fails in 0.00s with a clear error.

Happy to split it out if you'd rather keep this PR focused. What do you think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs: needed Pull request requires documentation updates semver: minor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants