Skip to content

Full adaptation of CI/CD to parametrized FIND_PAGE_SIZE#425

Open
sl-at-ibm wants to merge 3 commits into
mainfrom
SL-parametrized-pagesize
Open

Full adaptation of CI/CD to parametrized FIND_PAGE_SIZE#425
sl-at-ibm wants to merge 3 commits into
mainfrom
SL-parametrized-pagesize

Conversation

@sl-at-ibm

Copy link
Copy Markdown
Collaborator

In preparation to deploy of Data API v1.0.48, with (deploy-time) configurable page size, this PR completes the adaptation of integration tests to cope with this change.

The changes here are inert unless the tests are run after e.g. export FIND_PAGE_SIZE=50. So this can be merged any time now. Once Astra prod gets the new page size (of 50 by default), setting a new repo env variable will make the tests run smoothly.
At that point, it might be even simpler to bump the HCD compose data api version to 1.0.48 and hardcode 50 again.

Note that right now the workflow yamls transmit the repo secret (for FIND_PAGE_SIZE all the way to the Astra integration test only, because this repo has full control over the HCD case (and will manage the version used there)).

@sl-at-ibm sl-at-ibm requested a review from erichare June 18, 2026 15:03
@github-actions

Copy link
Copy Markdown

Coverage report

for commit: 11d4098e4520db97f3df5ce99d3e0f1a30269336.
download detailed report here.

                                                File   Stmts   Miss      Cover     Delta
----------------------------------------------------------------------------------------
                      astrapy/exceptions/__init__.py     108     19     82.41%    -0.93%
                                     astrapy/repl.py      77     77      0.00%     0.00%
                              astrapy/admin/admin.py     763    361     52.69%     0.00%
                                 astrapy/__init__.py      26     10     61.54%     0.00%
                               astrapy/utils/meta.py      42     15     64.29%     0.00%
                                  astrapy/results.py      50     16     68.00%     0.00%
              astrapy/exceptions/table_exceptions.py      27      8     70.37%     0.00%
         astrapy/exceptions/collection_exceptions.py      40     11     72.50%     0.00%
                  astrapy/data_types/data_api_set.py      90     23     74.44%     0.00%
                         astrapy/exceptions/utils.py      53     11     79.25%     0.00%
                  astrapy/data/cursors/pagination.py      21      4     80.95%     0.00%
                           astrapy/authentication.py     138     24     82.61%     0.00%
                        astrapy/utils/user_agents.py      18      3     83.33%     0.00%
                                   astrapy/client.py      68     11     83.82%     0.00%
               astrapy/data_types/data_api_vector.py      44      7     84.09%     0.00%
          astrapy/data/info/collection_descriptor.py     222     33     85.14%     0.00%
                      astrapy/data/info/reranking.py     122     18     85.25%     0.00%
                            astrapy/utils/parsing.py       7      1     85.71%     0.00%
                  astrapy/data_types/data_api_map.py      59      8     86.44%     0.00%
 astrapy/data/info/table_descriptor/table_listing.py      49      6     87.76%     0.00%
                            astrapy/data/database.py     657     75     88.58%     0.00%
 astrapy/data/info/table_descriptor/table_indexes.py     230     26     88.70%     0.00%
            astrapy/data_types/data_api_timestamp.py      98     11     88.78%     0.00%
                      astrapy/data/info/vectorize.py     131     14     89.31%     0.00%
                  astrapy/data/info/database_info.py     104     11     89.42%     0.00%
    astrapy/settings/definitions/definitions_data.py      38      4     89.47%     0.00%
  astrapy/data/info/table_descriptor/type_listing.py      39      4     89.74%     0.00%
             astrapy/data_types/data_api_dict_udt.py      10      1     90.00%     0.00%
                              astrapy/utils/unset.py      10      1     90.00%     0.00%
                        astrapy/utils/api_options.py     206     20     90.29%     0.00%
astrapy/data/info/table_descriptor/table_altering.py     114     11     90.35%     0.00%
             astrapy/data_types/data_api_duration.py      55      5     90.91%     0.00%
                      astrapy/utils/api_commander.py     248     22     91.13%     0.00%
                               astrapy/data/table.py     738     65     91.19%     0.00%
                          astrapy/admin/endpoints.py      36      3     91.67%     0.00%
 astrapy/data/info/table_descriptor/table_columns.py     204     17     91.67%     0.00%
 astrapy/data/info/table_descriptor/type_altering.py      85      7     91.76%     0.00%
                 astrapy/utils/duration_std_utils.py      92      7     92.39%     0.00%
         astrapy/exceptions/devops_api_exceptions.py      79      6     92.41%     0.00%
                          astrapy/data/collection.py     752     57     92.42%     0.00%
                astrapy/data/cursors/query_engine.py     214     16     92.52%     0.00%
         astrapy/event_observers/context_managers.py      27      2     92.59%     0.00%
                 astrapy/data_types/data_api_time.py      98      7     92.86%     0.00%
                 astrapy/data_types/data_api_date.py      89      6     93.26%     0.00%
 astrapy/data/info/table_descriptor/type_creation.py      32      2     93.75%     0.00%
             astrapy/exceptions/error_descriptors.py      53      3     94.34%     0.00%
   astrapy/settings/definitions/definitions_types.py      19      1     94.74%     0.00%
         astrapy/data/utils/collection_converters.py      80      4     95.00%     0.00%
              astrapy/data/utils/table_converters.py     411     20     95.13%     0.00%
                 astrapy/data/cursors/find_cursor.py     631     30     95.25%     0.00%
astrapy/data/info/table_descriptor/table_creation.py      49      2     95.92%     0.00%
                astrapy/event_observers/observers.py      30      1     96.67%     0.00%
                      astrapy/data/cursors/cursor.py      92      3     96.74%     0.00%
                 astrapy/data/cursors/farr_cursor.py     350     11     96.86%     0.00%
                           astrapy/utils/str_enum.py      32      1     96.88%     0.00%
                   astrapy/utils/duration_c_utils.py      66      2     96.97%     0.00%
                         astrapy/utils/date_utils.py      80      2     97.50%     0.00%
   astrapy/settings/definitions/definitions_admin.py      41      1     97.56%     0.00%
           astrapy/exceptions/data_api_exceptions.py      84      1     98.81%     0.00%
           astrapy/data/utils/distinct_extractors.py     104      1     99.04%     0.00%
                           astrapy/admin/__init__.py       3      0    100.00%     0.00%
                              astrapy/api_options.py       3      0    100.00%     0.00%
                               astrapy/collection.py       2      0    100.00%     0.00%
                                astrapy/constants.py       5      0    100.00%     0.00%
                                  astrapy/cursors.py       7      0    100.00%     0.00%
                            astrapy/data/__init__.py       0      0    100.00%     0.00%
                    astrapy/data/cursors/__init__.py       1      0    100.00%     0.00%
             astrapy/data/cursors/reranked_result.py       8      0    100.00%     0.00%
                      astrapy/data/utils/__init__.py       0      0    100.00%     0.00%
      astrapy/data/utils/extended_json_converters.py      28      0    100.00%     0.00%
                   astrapy/data/utils/table_types.py      36      0    100.00%     0.00%
               astrapy/data/utils/vector_coercion.py      13      0    100.00%     0.00%
                      astrapy/data_types/__init__.py      10      0    100.00%     0.00%
                                 astrapy/database.py       2      0    100.00%     0.00%
                 astrapy/event_observers/__init__.py       5      0    100.00%     0.00%
                   astrapy/event_observers/events.py      50      0    100.00%     0.00%
                                      astrapy/ids.py       5      0    100.00%     0.00%
                                     astrapy/info.py      15      0    100.00%     0.00%
                        astrapy/settings/__init__.py       0      0    100.00%     0.00%
                        astrapy/settings/defaults.py      44      0    100.00%     0.00%
            astrapy/settings/definitions/__init__.py       0      0    100.00%     0.00%
                  astrapy/settings/error_messages.py       2      0    100.00%     0.00%
                                    astrapy/table.py       2      0    100.00%     0.00%
                           astrapy/utils/__init__.py       0      0    100.00%     0.00%
                     astrapy/utils/document_paths.py      46      0    100.00%     0.00%
                     astrapy/utils/python_version.py       5      0    100.00%     0.00%
                      astrapy/utils/request_tools.py      32      0    100.00%     0.00%
----------------------------------------------------------------------------------------
                                              totals    8856   1149     87.03%    -0.01%

NUM_DOCS = 2 * PAGE_SIZE + 5
NUM_DOCS_PAGINATION = 2 * (2 * PAGE_SIZE) + 5
# TODO: once v1.0.48 is in production, hardcode 50 and bump Data API version in docker compose file:
FIND_PAGE_SIZE = int(os.environ.get("FIND_PAGE_SIZE") or "20")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want the fallback value to be 20 here, or should it be 50 to match the new server default?

Oh, LOL i typed that before i read your comment above. Makes sense we should wait till its in production!

Comment thread DEVELOPING.md
Prepend tests with a `ASTRAPY_TEST_LATEST_MAIN=y` for features found on `main` that are not released anywhere.
_(Tip: run a code search first to see what is currently marked as such. Chances are nothing is.)_

### Page size increase (starting in v1.0.48)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect. I need to read the whole PR before commenting in the future 😄

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to make the corresponding changes in test_table_cursor_sync as well?

required: true
HEADER_EMBEDDING_API_KEY_VOYAGEAI:
required: true
FIND_PAGE_SIZE:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we also need to include FIND_PAGE_SIZE in the release workflow? i think it runs the integration tests directly

@erichare erichare left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a couple comments @sl-at-ibm, possible places that might also need adapting (not 100% sure) hence the request changes. Let me know if im off base though and ill approve!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants