Release/0.7.0

[jackiryan]

Description

This release is primarily intended to add support for the GHRSST L4 MUR product from PODAAC.

Refactored pipeline to use more concise messages between lambdas. The "Browse Image Transfer" (BIT) workflow is now more consolidated and the following lambdas from version <=0.6.0 are now part of the same lambda:

• process_harmony_results (reading result urls from Harmony API request and generating checksums)
• generate_image_metadata (producing the image metadata xml for GIBS with data start and end times)
• build_image_sets (associating sets of browse image, world file, and image metadata xml)
• save_cnm_message (saving the cnm json message with image set information for GIBS)

All of these functions occur synchronously, so they have been combined into the new "Handle BIG Result" step of the pipeline which produces and saves CNM messages to S3. This was done to reduce the size of messages passed by the bignbit pipeline between steps, since these messages are limited to 256KB in size. For datasets like GHRSST MUR where many tiled images are produced, the old system caused Step Functions to fail the workflow due to oversized messages. Rather than introduce a new database or save additional intermediate files to S3, this approach both simplifies the workflow and mitigates the message size issue.

In practice, this means that the final state output of the workflow has changed. Previously, the pobit item of the state payload contained an array of image_set objects. Now it contains an array of references to CNM messages that have been sent to GIBS over the SQS queue:

  "pobit": [
      {
        "cmr_provider": "LARC_CLOUD",
        "collection_name": "PREFIRE_SAT2_2B-FLX_EEDTEST",
        "cnm_bucket": "podaac-sit-svc-internal",
        "cnm_key": "bignbit-cnm-output/PREFIRE_SAT2_2B-FLX_EEDTEST_flx_LL/PREFIRE_SAT2_2B-FLX_S07_R00_20210721013413_03040.nc.G00.2026-02-19T17:27:01.816Z.cnm.json",
        "gibs": {
          "cnmContent": {
            "MD5OfMessageBody": "9597edb67114477b7c1133aec065062d",
            "MD5OfMessageAttributes": "f7aec4559a577e6f7a0ba62823347d93",
            "MessageId": "ffa61002-f475-48b6-b02f-fe89b408e3ea",
            "SequenceNumber": "18900253714877423616",
            "ResponseMetadata": {
              "RequestId": "e900647d-6804-5d8d-b4a2-066c2237bebc",
              "HTTPStatusCode": 200,
              "HTTPHeaders": {
                "x-amzn-requestid": "e900647d-6804-5d8d-b4a2-066c2237bebc",
                "date": "Thu, 19 Feb 2026 17:27:15 GMT",
                "content-type": "text/xml",
                "content-length": "512",
                "connection": "keep-alive"
              },
              "RetryAttempts": 0
            }
          }
        }
      },
      ...
  ]

Additionally, warnings are now issued when a Harmony API call reports success but produces no data. This situation is occasionally encountered with some data sets, though its root cause is unknown. For now, we want to capture the occurrences without failing the overall workflow since other variables or projections in the browse image workflow may still succeed.

Added

• issues/108: Handle case when no data is returned from a Harmony job by throwing a warning that can be tracked in CloudWatch logs.

Changed

• issues/148: Refactored message passing system after browse image generation to handle large tiled outputs (100s of output files).

Removed

• issues/148: Removed "Generate Image Metadata", "Build Image Sets", "Process Harmony Results", and "Save CNM Message" lambdas in favor of a consolidated "Process BIG Result" lambda that generates and save CNM messages for the entire result of a browse image generation workflow.

Overview of verification done

• Tested the following data sets in the SIT venue:
  • OPERA HLS
  • PREFIRE COG
  • TEMPO NO2
  • GHRSST L4 MUR (new!!)

Overview of integration done

Integration testing in UAT is TBD.

PR checklist:

• Linted
• Updated unit tests
• Updated changelog
• Integration testing

See Pull Request Review Checklist for pointers on reviewing this pull request