Skip to content

fix: remove noisy per-PID trace logs from fd usage collection#1246

Merged
duncanista merged 1 commit into
mainfrom
rey.abolofia/fd-logs
Jun 18, 2026
Merged

fix: remove noisy per-PID trace logs from fd usage collection#1246
duncanista merged 1 commit into
mainfrom
rey.abolofia/fd-logs

Conversation

@purple4reina

Copy link
Copy Markdown
Contributor

What does this PR do?

The usage-metrics monitoring loop polls /proc every 10ms and reads /proc/{pid}/fd for every PID each tick. When read_dir fails (e.g. /proc/1/fd is not readable by the extension), the existing code logged a trace! message per unreadable PID per tick. At ~40 ticks per invocation (tested with a Hello World golang function), a single persistently-unreadable PID produced ~40 trace messages per invocation (~4000 per 100 invocations).

These failures are benign — a PID's fd directory may be unreadable while the process is otherwise alive (typically a permission issue). The other /proc/{pid} readers (fd_max, threads_max, threads_use) already fail silently with continue. This change makes get_fd_use_data_from_path consistent with that behavior by removing the trace logging and dropping the now-unused trace import.

Motivation

Observed ~4000 noisy log messages per 100 invocations when trace logging was enabled, making trace output unusable for debugging.

Testing

  • cargo build — clean compile, no unused-import warnings
  • cargo test proc — all 8 proc tests pass, including test_get_fd_use_data
  • cargo clippy — clean

Checklist

  • PR description is filled out
  • Tests pass

Copilot AI review requested due to automatic review settings June 8, 2026 19:39
@purple4reina purple4reina requested a review from a team as a code owner June 8, 2026 19:39
@purple4reina purple4reina requested a review from jchrostek-dd June 8, 2026 19:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces log noise in the /proc-based usage-metrics monitoring loop by removing per-PID trace! logging when /proc/{pid}/fd is unreadable, aligning fd collection behavior with other /proc/{pid} readers that already fail silently.

Changes:

  • Removed trace! logging on fs::read_dir("/proc/{pid}/fd") failures (benign permission/readability cases).
  • Dropped the unused trace import from tracing.
  • Minor simplification: inline files.count() into the accumulator update.

@purple4reina

Copy link
Copy Markdown
Contributor Author

Related to #535

@datadog-prod-us1-3

datadog-prod-us1-3 Bot commented Jun 8, 2026

Copy link
Copy Markdown

Pipelines

Fix all issues with BitsAI

⚠️ Warnings

🚦 3 Pipeline jobs failed

DataDog/datadog-lambda-extension | integration-suite: [lmi]   View in Datadog   GitLab

DataDog/datadog-lambda-extension | integration-suite: [snapstart]   View in Datadog   GitLab

DataDog/datadog-lambda-extension | integration-suite: [on-demand]   View in Datadog   GitLab

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 09e0f63 | Docs | Datadog PR Page | Give us feedback!

@duncanista duncanista left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't think the solution is removing it, but rather reducing how often this is logged, cc @shreyamalpani since you added this code, any opinions?

@shreyamalpani

Copy link
Copy Markdown
Contributor

I believe PID 1 is the Lambda init process and we are probably seeing this failure log because of a permissions issue. I think this is safe to remove for now, but if we do start seeing file descriptor metrics drop we should add this back, ignoring any failures on PID 1 as that is a known failure.

@purple4reina purple4reina force-pushed the rey.abolofia/fd-logs branch 4 times, most recently from 838a8ec to da30d04 Compare June 12, 2026 17:51
The usage-metrics monitoring loop polls /proc every 10ms (100Hz) and reads
/proc/{pid}/fd for every PID. When a read_dir fails (e.g. /proc/1/fd is not
readable), it logged a trace line per PID per tick — ~40 messages/invocation
(~4000 per 100 invocations) when trace logging is enabled.

These failures are benign (a PID's fd dir may be unreadable while it is
otherwise alive), so drop the logging and fail silently with `continue`,
matching the other /proc/{pid} readers (fd_max, threads_max, threads_use).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@purple4reina purple4reina force-pushed the rey.abolofia/fd-logs branch from da30d04 to 09e0f63 Compare June 17, 2026 21:23
@duncanista duncanista merged commit a61ba77 into main Jun 18, 2026
58 of 61 checks passed
@duncanista duncanista deleted the rey.abolofia/fd-logs branch June 18, 2026 19:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants