Skip to content

Bug Report: no scoped NSF fetch forces a full sync-down per lookup, and the async poll timeout doesn't scale with account size #2388

Description

@kevtgenmab

Component: keepercommander (no scoped NSF fetch); keeper-security/terraform-provider-commander (fixed poll timeout)
Files:

  • keepercommander/commands/nested_share_folder/display_commands.py — NestedShareGetCommand.execute()
  • keepercommander/sync_down.py — sync_down()
  • terraform-provider-commander/internal/provider/api/api_manager.go — PollRequestResult
    Severity: Medium — as an account's NSF folder/record count grows, per-lookup cost and async-poll duration both increase without bound, eventually exceeding fixed client-side timeouts and rate limits

Summary

nsf-get <uid> resolves purely from local in-memory caches (params.nested_share_folders/nested_share_records) populated exclusively by a full, unscoped sync_down() call — there is no server-side "fetch just this one NSF folder/record" command. Every caller that needs to look up a single NSF object must first run a full-account sync-down. As an account's total NSF data volume grows, each sync_down() call takes proportionally longer (decrypting and reconstructing the full NSF folder tree every time), and callers that perform many sequential lookups (e.g. one per managed object) end up issuing many redundant full-account syncs in a row.

Separately, the async request-polling client used against Service Mode enforces a fixed poll timeout (60 seconds by default) with no retry/backoff on expiry:

pollTimeout := a.RequestTimeout
if pollTimeout <= 0 {
    pollTimeout = 60 * time.Second
}
...
case <-pollCtx.Done():
    return nil, fmt.Errorf("timeout waiting for request %s to complete (timeout: %v)", requestId, pollTimeout)

As the account's NSF dataset grows, sync_down()'s per-call processing time can exceed this fixed ceiling, causing Unable to perform sync down: timeout waiting for request <uuid> to complete even with zero rate-limiting or connectivity issues — a pure function of dataset size vs. a static timeout.

Root Cause

  1. No server-side scoped fetch exists for a single NSF folder/record — sync_down() is the only way to populate the caches nsf-get/nsf-list depend on, and it always processes the full account.
  2. The async result-poll timeout is a static value with no adjustment for account size and no retry/backoff when it expires — a single slow sync_down fails outright rather than being retried or given more time.

Reproduction

  1. Grow an account's NSF folder/record count substantially (dozens+).
  2. Call nsf-get <uid> for an existing object from a fresh process (forcing a real sync_down first).
  3. Observe increasing latency proportional to NSF folder/record count; eventually, callers with a fixed poll timeout (e.g. 60s) see timeout waiting for request <uuid> to complete even though the request would have succeeded given more time.

Proposed Fix

  • Add a server-side scoped fetch for a single NSF folder/record by UID, avoiding the need for a full sync_down() per lookup.
  • Make the async poll timeout scale with (or be informed by) the account's data volume, or support incremental/paginated sync_down so a single lookup's cost doesn't grow linearly with total account size.

References

  • keepercommander/commands/nested_share_folder/display_commands.py — _resolve_as_folder()/_resolve_as_record(), purely local-cache lookups
  • keepercommander/sync_down.py — full-account NSF ingestion, no per-object scoping parameter
  • internal/provider/api/api_manager.go (terraform-provider-commander) — PollRequestResult, fixed timeout, no retry on expiry

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions