Skip to content

Update: Make remaining periodic Celery tasks configurable - #471

Draft
MoralCode wants to merge 1 commit into
mainfrom
PredictiveManish/periodic-config
Draft

MoralCode wants to merge 1 commit into
mainfrom
PredictiveManish/periodic-config

Conversation

@MoralCode

Copy link
Copy Markdown
Contributor

Note

This PR was ported from augurlabs/augur#3411 filed by @PredictiveManish because the contribution was deemed to still be useful.

Description

This PR fixes #136

Notes for Reviewers
Changes:

  • Added four new configuration options in the Celery section:

    • non_repo_domain_tasks_interval_in_days (default: 30 days)
    • retry_errored_repos_interval_in_days (default: 1 day)
    • process_contributors_interval_in_seconds (default: 3600 seconds / 1 hour)
    • create_collection_status_records_interval_in_days (default: 1 day)
  • Updated setup_periodic_tasks() in celery_app.py to read intervals from config instead of using hardcoded values

  • Converted retry_errored_repos from crontab-based scheduling to interval-based for consistency

  • Added enable/disable functionality for all tasks (setting interval to 0 disables the task)

  • Improved logging messages to show the configured intervals

All tasks now follow the same pattern as refresh_materialized_views_interval_in_days, allowing users to customize scheduling intervals or disable tasks as needed through the configuration system.

Signed commits

  • Yes, I signed my commits.

Signed-off-by: PredictiveManish <manish.tiwari.09@zohomail.in>
@MoralCode

Copy link
Copy Markdown
Contributor Author
  • needs check for any legacy variable/project names

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

make remaining periodic tasks configurable

2 participants