fix(jumpstart): send NextToken when Hub.list_models pages through hub contents

[evakravi]

Problem

Hub.list_models() never returns on a hub with more than one page of contents. Hub._list_and_paginate_models reads NextToken from each list_hub_contents response but never sends it, so every call requests the first page and the loop never ends. On SageMakerPublicHub (743 models, 8 pages of 100) the call ran for 5 minutes without a result. Every 2.x release with the hub module (2.224.1 through 2.257.6) has the same loop.

Solution

Pass the token as next_token on each call after the first. Session.list_hub_contents maps it to NextToken. #6263 is the same fix for v3.

Tests

A unit test serves each page once and raises KeyError on a second request for the same page. The old loop fails on its second call. On origin/master-v2 it fails with KeyError: ('ModelReference', None). On this branch tests/unit/sagemaker/jumpstart/hub/test_hub.py passes: 10 passed.

SAGEMAKER_SUPPRESS_V2_WARNING=1 PYTHONPATH=src python -m pytest tests/unit/sagemaker/jumpstart/hub/test_hub.py

Live check with this branch against SageMakerPublicHub in us-west-2: Hub.list_models(max_results=100) returned in 2.8 s after 9 ListHubContents calls (1 ModelReference, 8 Model, 7 with NextToken). It returned 743 unique Model rows, each with OriginalCreationTime.

black==24.3.0, flake8==7.1.2, and pydocstyle==6.1.1 pass on the two changed files.

Merge Checklist

• I read the contribution guide.
• The change is backward compatible.
• The commits use the repository commit-message format.
• The change creates no S3 or STS client.
• The change has regression coverage and adds no dependency.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.