[fix] keep one-row metric data when command output is short - #4308
Open
orangeCatDeveloper wants to merge 1 commit into
Open
[fix] keep one-row metric data when command output is short#4308orangeCatDeveloper wants to merge 1 commit into
orangeCatDeveloper wants to merge 1 commit into
Conversation
A trailing existence check like 'grep -o mount-name' emits nothing when the target is gone, so the row was dropped (partial output) or the whole collection failed (empty output) and the metric vanished exactly when it mattered (apache#1560). Pad missing trailing fields with null, accept blank output as empty data when stderr is empty and exit status <= 1, and carry stderr/exit code into failure messages. Ssh and script collectors shared the copy-pasted logic, now merged into OneRowResponseSupport; ssh also gains an optional output charset.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What's changed?
Fixes #1560.
Scenario: an SSH/script monitor uses
oneRowparsing — each output line fills one field. The reporter's script ends with an existence check:When the mount disappears, the script outputs 3 lines instead of 4 — and the user expects the last field to show empty so they can alert on it.
What actually happened — two code paths, both losing the data:
parseResponseDataByOnerejected the whole row when output lines were fewer than alias fields, discarding the cpu/memory/disk values that were collected. The metric page went blank and every cycle loggedssh response data not enough. Ironically, the loop right below that length check already pads missing columns withNULL_VALUE— the check just never let short output reach it.response data is null, treating "grep found nothing" (a meaningful answer) the same as "the command blew up".Fix:
NULL_VALUE, collected values survive.NULL_VALUEand succeed. Real failures still fail — and the message now carries the captured stderr or exit code instead of the fixedresponse data is null.System.err; script never read it — it is drained on a daemon thread to avoid the classic stderr-pipe deadlock).OneRowResponseSupport. The script collector shares the same defect and is fixed together; no other collector has one-row command semantics.SshProtocolgains an optionalcharset(default UTF-8) for non-UTF-8 remote output, matching whatScriptProtocolalready had.Known tradeoff (pinned by an explicit test): a command that exits 1 silently with no output and no stderr is indistinguishable from a grep no-match and is accepted as empty data. Requiring exit 0 instead would break the reported use case itself, since grep exits 1 on no match.
With
NULL_VALUEflowing through, the alert calculator sees the field asnull, so rules like!exists(nfs_mount)can fire on "the mount is gone" — exactly the issue's ask.Before / after
Verified against a real sshd container, using the reporter's script shape (
echo 52; echo 35.8033; echo 5%; grep -o 'missing-mount' /proc/mounts) with 5 alias fields (cpu, memory, disk, nfs_mount, responseTime):Before — no metric row is stored; the collector logs the same error as the issue's screenshot, every collect cycle:
After — the row is stored with the gap visible as null, and no errors are logged:
Checklist
Add or update API