Skip to content

Document cold start scenario #542

Description

@krzysztof-kukla1974

We should have documented special considerations related to Cold Start scenario.

  1. How to tweak Collect Data Source parameters?
  2. How to avoid throttling issues?
  3. How to monitor the system and adjust?
  4. How to use "merge data batches during cold start" parameter?
  5. How to configure Trace -> Setup -> Data Retrieval -> Run Interval - to not produce too many Data Batches on start and to not kill the 10 collection jobs in parallel queue.

Example:

  1. Stop Teams connector.
  2. Configure Trace -> Setup -> Data Retrieval -> Run Interval to 1 hour - so, only 1 Data Batch (each type) will be created per 1 hour.
  3. Abandon all Data Batches which were previously run (for instance with incorrect setup) - that would cancel all underlying Collect jobs.
  4. Reconfigure Data Source.
  5. Set Start Date to XYX.
  6. Set Frequency in Minutes to 1440.
  7. Set Merge Batches During Cold Start to Yes.
  8. Set Max Number of Batches To Merge to 30 <<< this means we'll create Data Batches each 30 day long.
  9. Reset Data Source.
  10. Enable Data Source.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions