Repository navigation
test: caldav's test servers from caldav.testing, run in parallel - #38
Merged
Merged
Conversation
caldav now ships its test-server registry in caldav.testing, so the conftest imports it instead of loading tests/test_servers by path from a caldav checkout, and CALDAV_SOURCE is gone. The registry also merges all its config sources, finds the checkout's yaml whatever the working directory, and probes external servers itself, so the conftest's workarounds for those go too. python-caldav/caldav#817 python-caldav/caldav#824 Prompt: Now that we have #817 done on the local caldav master (please confirm), please fix the calendaring library to use this, and run the integration tests towards all the docker servers plus all the external servers defined under [a caldav checkout holding the test servers' passwords] [#817: python-caldav/caldav#817] Assisted-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
caldav's Zimbra test server is configured with ssl_verify_cert: false, and urllib3's "Unverified HTTPS request" warning then failed every Zimbra test under filterwarnings = error. The filter matches on the message, so it needs no urllib3 import. Prompt: Now that we have #817 done on the local caldav master (please confirm), please fix the calendaring library to use this, and run the integration tests towards all the docker servers plus all the external servers defined under [a caldav checkout holding the test servers' passwords] [#817: python-caldav/caldav#817; this fixes what the Zimbra run turned up] Assisted-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
A run on every caldav test server took hours, most of them on the slowest server. Under pytest -n N --dist loadgroup each server's tests now stay on one worker, so the servers run in parallel and keep one connection and one session lock each; pytest-xdist joins the dev extra. --caldav-servers-config adds the servers of a test-server yaml outside the caldav checkout, its entries winning on a name clash. Prompt: The biggest problem is the purelymail runs [due to test sleeps due to purelymail having an async search cache], a single run can easily take several hoursl If we cannot do that with pytest-xdist, then it's needed to do aome similar lgoic as in [a personal script that runs caldav's tests in parallel, one tmux window per slow server] Followup-Prompt: Do the needful ensuring all tests are run towardes all servers - if ecloud failed on the last run, then it's a good reason to retry, [(sic)] Assisted-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
A server whose time-range REPORT answers from a lagging index (search-cache delay, filtered-only: Purelymail) now gets its ranges filtered on the client, as plain listings are current there. A calendar no longer claims the component kinds the server is hinted to refuse (robur lists VJOURNAL, then answers a journal with 400), which had failed every search scenario there. Purelymail's 20 failing tests now pass, and robur's search scenarios fall from 26 failures to a transient 500. Prompt: please check the handover document. Start creating that sweep-script for ecloud, and then ... the primary mission is to figure out why tests are failing. Fork out subagents with opus when needed. [handover: notes on the test results from calendaring tests when run on all of caldav's test servers] Followup-Prompt: [start the fixes, beginning with the search-related ones] Assisted-By: Claude Sonnet 5.5, Claude Opus 5.5 Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
RDATE and EXDATE compare as sets of values, so a server that splits RDATE:a,b into one line per date no longer fails verification (Bedework, Cyrus). A TZID compares by how the zone behaves from 1970 on, so Europe/Oslo stored as Europe/Berlin (gmx) is accepted while Europe/Sofia stored as Europe/Berlin is still a difference. In the conformance suite, time-entry users compare in any order, and a backend whose delete-collection level is unknown is tried, as open_collection already does (Cyrus). Prompt: please check the handover document. Start creating that sweep-script for ecloud, and then ... the primary mission is to figure out why tests are failing. Fork out subagents with opus when needed. [handover: notes on the test results from calendaring tests when run on all of caldav's test servers] Followup-Prompt: "gmx rewrites Oslo to Berlin". That's harmless. Does it rewrite Sofia to Berlin too? That's not harmless. Please check. Followup-Prompt: [start the fixes, beginning with the search-related ones] Assisted-By: Claude Sonnet 5.5, Claude Opus 5.5 Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
Infomaniak and CCS refuse any MKCALENDAR whose component set names VJOURNAL, so every collection made for the scenario stories failed in setup. Their hints already say save-load.journal is unsupported: create_collection() now leaves such components out of the request, and refuses before any I/O when nothing is left; the capability says LOSSY. Prompt: Also, I note that there are tons of failures and errors, please look into it Followup-Prompt: go ahead [with separate calendar, task list and journal fixtures in the scenario tests] Assisted-By: Claude Opus 5.5 <noreply@anthropic.com> Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
Time entries are events in the task's own collection, so a tasks-only collection must declare time-log unsupported, as the caldav and files backends already did; the memory backend claimed it and then refused the entry. The three copies of that narrowing are now one helper, held_only(), which leaves alone a backend keeping its own time log. Prompt: quite many systems, including many caldav servers have separated task lists and calendars. Looking into the scenario tests, I see only a "calendar"-fixture, I think it's needed with a separate "tasklist"-fixture. Also, have a look at my TODO-notes there. Followup-Prompt: go ahead [with the fixtures above; the new tasks-only fixture exposed this] Assisted-By: Claude Opus 5.5 <noreply@anthropic.com> Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
Many servers keep task lists apart from calendars, and CCS makes a calendar for one component only, so the stories now get a calendar, a task list and a journal each. Where a backend makes none for a kind, a story marked checks_refusal gets one for another kind and checks the refusal there; the others are skipped, as before. The time log goes into the calendar where the task list holds no events. The dates are next week's rather than November 2026, which would one day be past every server's time range. The task story checks that the list holds tasks. Also the reworded docstring and comments. Prompt: quite many systems, including many caldav servers have separated task lists and calendars. Looking into the scenario tests, I see only a "calendar"-fixture, I think it's needed with a separate "tasklist"-fixture. Also, have a look at my TODO-notes there. Followup-Prompt: go ahead [with the proposed fixtures, relative dates and the TODO notes] Assisted-By: Claude Opus 5.5 <noreply@anthropic.com> Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
The pull request's changelog check wants one; the change is to the test setup only, hence internal. Prompt: [review and push the branch] Assisted-By: Claude Opus 5.5 <noreply@anthropic.com> Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
The tests take caldav's test servers from caldav.testing, which caldav 3.7.0 is the first release to ship; with 3.6 the type check fails on its imports. The library itself still works with caldav 3.6. Prompt: Why is PR38 on hold? Because it needs caldav 3.7.0 maybe? 3.7.0 is out today Assisted-By: Claude Opus 5.5 <noreply@anthropic.com> Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Disclaimer: AI-generated code and text below the line, generated by Claude Opus 5.5, based on a wish to run functional tests for the calendaring library.
caldav now ships its test-server registry in
caldav.testing(python-caldav/caldav#824). The conftest imports it from there instead of loadingtests/test_serversby path from a caldav checkout, and the workarounds for caldav's old config handling are gone.A run on every caldav test server took many hours, because the servers were tested one after another. Under
pytest -n N --dist loadgroup, each server's tests now stay on one worker, so the servers run in parallel.--caldav-servers-config=PATHadds the servers of a test-server yaml kept outside the caldav checkout. A warning filter lets the test servers configured without TLS verification (caldav's Zimbra) run underfilterwarnings = error.Tests only: the default suite passes serially and under
-n 8 --dist loadgroup(2817 passed, 433 skipped, 4 xfailed). The first run on all servers turned up many divergences, which are left for roadmap 3.2.🤖 Generated with Claude Code