Skip to content

Updating tutorials to newest release of pydap #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Mikejmnez wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Updating tutorials to newest release of pydap #19

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
bbcc692
add DAYMET stream example
Mikejmnez Mar 16, 2026
b9579c0
update environment
Mikejmnez Mar 16, 2026
6b94633
add DAYMET example (pure pydap)
Mikejmnez Mar 16, 2026
a3a634f
reorganization
Mikejmnez Mar 27, 2026
def89ab
update README to describe tutorial organization
Mikejmnez Mar 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 8 additions & 15 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,22 +30,15 @@ running in Jupyter should open. That's it for the setup.

## The notebooks

* **Get Started.ipynb**
_All about Earthdata Login_ Run this to create a local token object. Must have a
an active EDL account. The token object will be used in other notebooks.
The overall organization is:

* **ECCOv4.ipynb**
_Uses pydap an xarray_ to aggregate an an entire collection of
the ECCOv4 simulation. It uses the _Common Metadata Repository_ (CMR) to find cloud OPeNDAP URLs
associated with the DOI of the data collection.
* **binder/*.ipynb**: These demonstrate basic and most performant access to OPeNDAP data via PyDAP. These cover
* How to find OPeNDAP URLS
* How to Authenticate
* How to subset by variable names, time range, and coordinate values.
* How to best stream OPeNDAP dap4 responses into local NetCDF4 files.

* **earthaccess.ipynb**
_Uses earthaccess to create a VirtualZarr from OPeNDAP DMR++ for Cloud
data_. earthaccess uses the _Common Metadata Repository_ (CMR) to query all dataset by their
short name.

* **on-premOPeNDAP.ipynb** _Using pydap an xarray_ it creates a virtually aggregation of 100s of
OPeNDAP URLs hosted by NASA on premises on the OB.DAAC.
* **binder/Xarray/*.ipynb **: These tutorials demonstrate access to OPeNDAP data using Xarray and "PyDAP" as the backend engine. To get the most performant access requires extra tricks, and these tutorials cover what is needed to get close to performance access.

----
## **Optional**: Running the notebooks locally
Expand Down Expand Up @@ -77,7 +70,7 @@ conda env create -f binder/environment.yml`
After a few minutes, activate the new environment

```
conda activate Earthdata2025`
conda activate Earthdata2026`
```

* Step 3 Starting the local copy of the notebooks
Expand Down
99 changes: 99 additions & 0 deletions binder/Authenticate.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "da20b7f1-ab3a-477c-a9bb-4c3f97c99afb",
"metadata": {},
"source": [
"<span style='color:#009999'> <span style='font-family:serif'> <font size=\"15\"> <span style='color:#0066cc'> **Earthdata Login authentication with earthaccess**\n",
" \n",
" \n",
"<img src=\"img/EarthData.png\" alt=\"drawing\" width=\"750\"/> \n",
" \n",
"\n",
"\n",
" <span style='color:#ff6666'><font size=\"5\"> **Requirements**\n",
"1. <font size=\"3\">Valid EDL account. If not, got o [Login Page](https://urs.earthdata.nasa.gov/home) and set up a Username and Password.\n",
"\n",
" <span style='color:#ff6666'><font size=\"5\"> **Objectives**\n",
"- <font size=\"3\"><span style='color:Black'>To demonstrate remote access via token to Earthdata.\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "e477a20a-4586-405d-8dce-9e5cee6e0ab1",
"metadata": {},
"outputs": [],
"source": [
"import earthaccess"
]
},
{
"cell_type": "markdown",
"id": "5774c511-3c18-4741-970a-bf78682c25a6",
"metadata": {},
"source": [
"<span style='font-family:serif'> <font size=\"5.5\"><span style='color:#0066cc'> **EDL Authentication via earthaccess**\n",
"\n",
"<font size=\"3.5\"> You can authenticate via earthaccess as demonstrated below. You must have a valid EDL account. There are two strategies for authenticating with `earthaccess`:\n",
"\n",
"1. `strategy=\"interactive\"`. This will promt your edl `username-password`.\n",
"2. `strategy=\"netrc\"`. Use this if the notebook is running on an environment where a `.netrc` with your credentials is recoverable.\n",
"\n",
"<font size=\"3.5\">Below the default will be `interactive`, assuming there is not .netrc credential stored in the current machine. Once authenticated, all EDL credentials will be injected into an existing local (discoverable) `.netrc` file, or create one if there is none yet."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "42cf01d6-ebed-4028-8e77-63c8dfbdc40e",
"metadata": {},
"outputs": [],
"source": [
"auth = earthaccess.login(strategy=\"interactive\", persist=True) # you will be promted to add your EDL credentials\n",
"\n",
"# pass Token Authorization to a new Session.\n",
"my_session = auth.get_session()"
]
},
{
"cell_type": "markdown",
"id": "44bd8c9a-09a4-4187-a814-f371a02b885a",
"metadata": {},
"source": [
"<font size=\"3.5\"> The `my_session` object contains all valid credentials object and can be used by PyDAP to send data requests to the remote NASA OPeNDAP server."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a4ca32f3-eadc-4f0c-adca-dc5ef06379ca",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.12"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
85 changes: 14 additions & 71 deletions binder/CMR_queries.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,103 +14,46 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"id": "adc54031-dc9f-4858-83be-a84c6ee4eef0",
"metadata": {},
"outputs": [],
"source": [
"from pydap.net import create_session\n",
"from pydap.client import get_cmr_urls"
"from pydap.client import get_cmr_urls\n",
"import datetime as dt"
]
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"id": "f764fa2b-acc4-43ba-8762-b606dc4a96e4",
"metadata": {},
"outputs": [],
"source": [
"ecostress_ccid = \"C2076114664-LPCLOUD\""
"ECOSTRESS_ccid = \"C2076114664-LPCLOUD\"\n",
"bounding_box = [-128.847656,41.112469,-107.050781,46.679594]\n",
"time_range = [dt.datetime(2025, 3, 1), dt.datetime(2025, 3, 31)]"
]
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"id": "c4306a74-3848-42ec-9e57-e396f9f47b80",
"metadata": {},
"outputs": [],
"source": [
"urls = get_cmr_urls(ccid=ecostress_ccid, bounding_box=list((-130.8, 41, -124, 45)))"
"urls = get_cmr_urls(ccid=ECOSTRESS_ccid, bounding_box=bounding_box, time_range=time_range, limit=500)\n",
"print(\"Found \", len(urls), \"relevant opendap urls for ECOSTRESS data\")"
]
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"id": "68aa026d-51d5-4bab-a642-2a5c04258712",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00152_003_20180716T130457_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00258_001_20180723T101233_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00289_001_20180725T100502_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00346_001_20180729T014458_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00346_002_20180729T014550_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00392_002_20180801T004543_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00392_003_20180801T004635_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00408_005_20180802T013113_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00423_001_20180803T003844_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00423_002_20180803T003936_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00438_002_20180803T234626_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00438_003_20180803T234718_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00441_002_20180804T043829_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00453_003_20180804T225449_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00457_005_20180805T052214_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00457_005_20180805T052214_0712_05',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00457_006_20180805T052306_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00457_007_20180805T052358_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00469_004_20180805T233925_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00469_004_20180805T233925_0712_05',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00469_005_20180805T234017_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00472_001_20180806T043039_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00472_002_20180806T043131_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00484_004_20180806T224658_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00484_004_20180806T224658_0712_05',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00484_005_20180806T224750_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00484_006_20180806T224842_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00487_002_20180807T033902_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00499_003_20180807T215521_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00499_004_20180807T215613_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00503_007_20180808T042338_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00503_008_20180808T042430_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00518_007_20180809T033122_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00518_008_20180809T033214_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00530_004_20180809T214742_0712_05',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00530_004_20180809T214742_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00530_005_20180809T214834_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00530_006_20180809T214926_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00533_005_20180810T023944_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00545_002_20180810T205606_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00545_003_20180810T205658_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00549_011_20180811T032420_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00549_011_20180811T032420_0712_03',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00576_004_20180812T204906_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00576_005_20180812T204958_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00579_008_20180813T013925_0712_03',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00579_008_20180813T013925_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00579_009_20180813T014017_0712_03',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00591_001_20180813T195545_0712_04',\n",
" 'https://opendap.earthdata.nasa.gov/collections/C2076114664-LPCLOUD/granules/ECOv002_L2_LSTE_00591_001_20180813T195545_0712_05']"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"urls"
"urls[:10]"
]
},
{
Expand Down Expand Up @@ -138,7 +81,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.13"
"version": "3.12.12"
}
},
"nbformat": 4,
Expand Down
Loading