Skip to content

fix(objectstore): Do not log errors for client aborts#6064

Open
jjbayer wants to merge 3 commits into
masterfrom
fix/error-verbose
Open

fix(objectstore): Do not log errors for client aborts#6064
jjbayer wants to merge 3 commits into
masterfrom
fix/error-verbose

Conversation

@jjbayer

@jjbayer jjbayer commented Jun 9, 2026

Copy link
Copy Markdown
Member

Most of the "upload failed" errors we see in the objectstore service are broken client streams (unexpected EOF).

  1. If they are broken because of an error in an upstream request handler that we control, the error should be instrumented there.
  2. If they are broken because the end user loses the connection or cancels the upload, we should not consider it an error on our end.

In both cases we do not need to log an error in the objectstore service.

Fixes INGEST-884

@linear-code

linear-code Bot commented Jun 9, 2026

Copy link
Copy Markdown

INGEST-884

@jjbayer jjbayer marked this pull request as ready for review June 9, 2026 12:28
@jjbayer jjbayer requested a review from a team as a code owner June 9, 2026 12:28
Comment on lines +180 to +182
if self.kind.is_client_error() {
return;
}

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the code change that suppresses error logs for client errors.

Comment thread relay-server/src/services/objectstore.rs Outdated
Comment on lines +235 to +241
ErrorKind::LoadShed => false,
ErrorKind::UploadFailed(objectstore_client::Error::Reqwest(error)) => {
find_error_source(error, is_hyper_user_error).is_some()
}
ErrorKind::UploadFailed(_) => false,
ErrorKind::Uuid(_) => false,
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a rule on when to do this vs?

  ErrorKind::InvalidScoping
  | ErrorKind::Timeout(_)
  | ErrorKind::LoadShed
  | ErrorKind::Uuid(_) => false,

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simplified the expression now.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 6324cba. Configure here.

find_error_source(error, is_user_error).is_some()
}
_ => false,
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Client abort detection too narrow

Medium Severity

is_client_error only treats UploadFailed reqwest errors as client faults when the source chain contains an io::Error with kind UnexpectedEof or FileTooLarge. The upload endpoint already classifies the same objectstore UploadFailed reqwest failures using is_hyper_user_error, and PATCH bodies map inbound failures to io::Error::other, so many disconnects never match and still emit relay_log::error!.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 6324cba. Configure here.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's OK, I want to start conservatively and relax the condition if necessary.

@jjbayer jjbayer enabled auto-merge June 9, 2026 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants