Merge pull request #208 from DomainTools/release-v2.10.0

DT Python Wrapper Release v2.10.0

Author: cpanares

CHANGELOG.md

@@ -1,5 +1,11 @@
 # Changelog
 
+### 2.10.0
+- [NEW] Support HMAC signature authentication for feed endpoints and CLI (`--no-sign-api-key`).
+- [UPDATE] Officially support all available feed parameters and CLI options across real-time threat feed endpoints (including `--frombeginning`, risk filters, and IP-specific filters).
+- [FIX] Prevent authentication credentials from being passed as plain-text query parameters when header or HMAC authentication is enabled.
+- [FIX] Remove incorrect requirement for `sessionID`, `after`, or `before` parameters when using the feed download endpoint.
+
 ### 2.9.0
 - [NEW] Add support for querying real-time IP feeds (`iphotlist` and `iprisk`).
 - [FIX] Type hint used for `top` parameter in Feeds API from str to int.

PYTHON_SUPPORT.md

@@ -2,28 +2,20 @@
 
 ## Policy
 
-The DomainTools API library will support all versions of Python that are actively maintained by the Python 
+The DomainTools API library will support all versions of Python that are actively maintained by the Python
 Software Foundation. When a version of Python enters End of Life (EOL), the API library will also end support
-for that version of Python. 
+for that version of Python.
 
-When a version's End of Life date is reached, DomainTools will ensure that a release of the API library that 
+When a version's End of Life date is reached, DomainTools will ensure that a release of the API library that
 contains all changes up to that point in time is available. If a release already exists that has all
-changes at the point of a version's EOL date, no new one will be made. Any changes (features, bugfixes, etc) 
+changes at the point of a version's EOL date, no new one will be made. Any changes (features, bugfixes, etc)
 released after an EOL date will not be tested on the now-unsupported version.
 
-Versions of Python from other organizations (e.g. cython, pypy, jython) will not be actively supported. DomainTools 
-will not develop specifically for those versions of Python, but we welcome community assistance (such as pull 
+Versions of Python from other organizations (e.g. cython, pypy, jython) will not be actively supported. DomainTools
+will not develop specifically for those versions of Python, but we welcome community assistance (such as pull
 requests) to support them.
 
-### Python 2
 
-DomainTools API library support for Python 2.7 (and all Python 2) will end on November 30, 2020. 
+### Python >=3.9
 
-### Python 3.5
-
-DomainTools will continue to support Python 3.5 until November 30, 2020. 
-
-## Upcoming Timeline:
-
-- Support for Python 2.7 will end on Nov 30, 2020
-- Support for Python 3.5 will end on Nov 30, 2020
+DomainTools currently supports Python 3.9 and above.

README.md

@@ -253,6 +253,19 @@ Please see the [supported versions](https://github.com/DomainTools/python_api/ra
 for the DomainTools Python support policy.
 
 
+Authentication
+===================
+
+The wrapper supports two authentication modes, selected automatically based on the product:
+
+| Product | Default method | Params sent |
+|---|---|---|
+| Standard API (Iris, Whois, etc.) | HMAC-SHA256 signed | `api_username`, `timestamp`, `signature` as query params |
+| Real-Time Threat Feeds (RTTF) | Header authentication | `X-Api-Key` header |
+
+RTTF feeds also support HMAC signing as an opt-in via `always_sign_api_key=True` — see the RTTF section below.
+
+
 Real-Time Threat Feeds
 ===================
 
@@ -264,18 +277,24 @@ Custom parameters aside from the common `GET` Request parameters:
     api = API(USERNAME, KEY)
     api.nod(endpoint="feed", **kwargs)
     ```
-- `header_authentication`: by default, we're using API Header Authentication. Set this False if you want to use API Key and Secret Authentication. Apparently, you can't use API Header Authentication for `download` endpoints so this will be defaulted to `False` even without explicitly setting it.
+- `header_authentication`: by default, all RTTF endpoints (both `feed` and `download`) use API Header Authentication, sending the API key via the `X-Api-Key` header. Set this to `False` to pass the API key as a query parameter instead.
     ```python
     api = API(USERNAME, KEY, header_authentication=False)
     api.nod(**kwargs)
     ```
+- `always_sign_api_key`: set to `True` to use HMAC-SHA256 signed authentication instead of header auth. When set, `header_authentication` automatically defaults to `False` — both methods do not fire simultaneously. The signing algorithm is identical to the standard API: `HMAC-SHA256(key, username + timestamp + path)`, with `timestamp` and `signature` sent as query parameters.
+    ```python
+    api = API(USERNAME, KEY, always_sign_api_key=True)
+    api.nod(after="-60")
+    # sends: api_username, timestamp, signature — no X-Api-Key header
+    ```
 - `output_format`: (choose either `csv` or `jsonl` - default is `jsonl`). Cannot be used in `domainrdap` feeds. Additionally, `csv` is not available for `download` endpoints.
     ```python
     api = API(USERNAME, KEY)
     api.nod(output_format="csv", **kwargs)
     ```
 
-The Feed API standard access pattern is to periodically request the most recent feed data, as often as every 60 seconds. Specify the range of data you receive in one of two ways:
+The `feed` endpoint streams live NDJSON data. The standard access pattern is to poll as often as every 60 seconds. Specify the range of data you receive in one of two ways:
 
 1. With `sessionID`: Make a call and provide a new `sessionID` parameter of your choosing. The API will return the last hour of data by default.
     - Each subsequent call to the API using your `sessionID` will return all data since the last.
@@ -284,9 +303,75 @@ The Feed API standard access pattern is to periodically request the most recent
     - Either an `after=-60` query parameter, where (in this example) -60 indicates the previous 60 seconds.
     - Or `after` and `before` query parameters for a time range, with each parameter accepting an ISO-8601 UTC formatted timestamp (a UTC date and time of the format YYYY-MM-DDThh:mm:ssZ)
 
-## Handling iterative response from RTUF endpoints:
+The `download` endpoint returns a standard JSON response (not a stream) listing available S3 batch files. Time parameters (`sessionID`, `after`, `before`) are **not** required for download calls.
+
+```python
+api = API(USERNAME, KEY)
+result = api.nod(endpoint="download", limit=5)
+print(result["download_name"])
+for f in result["files"]:
+    print(f["name"], f["url"])
+```
+
+### Feed parameters
+
+The feed methods accept the following parameters, grouped by purpose. Availability depends on the feed (see the notes below the table).
+
+#### Session Management Parameters
+
+- `sessionID`: A custom string used to distinguish between different sessions. Required when using `fromBeginning`.
+- `after`: Start of the query window. Either an integer offset relative to now in seconds (e.g. `-60`), or an absolute ISO 8601 UTC datetime (`YYYY-MM-DDTHH:MM:SSZ`).
+- `before`: End of the query window (inclusive). Either an integer from `-1` to `-432000` (seconds before now), or an absolute ISO 8601 UTC datetime. The query window covers at most the most recent 5 days; a value older than 5 days returns no records.
+- `fromBeginning`: Boolean (`true`/`false`/`1`/`0`, default `false`). Requires a valid `sessionID`. When `true` on the first request of a new session, returns the first hour of data in the time window instead of the last. Using it with an existing `sessionID` returns an HTTP 406; using it without a `sessionID` or with a non-boolean value returns an HTTP 422.
+
+    ```python
+    api = API(USERNAME, KEY)
+    api.nod(sessionID="my-new-session-id", after=-3600, fromBeginning=True)
+    ```
+
+#### Filter Parameters
+
+- `domain`: Filter for an exact domain or a substring contained within a domain by prefixing or suffixing your substring with `*`.
+- `overall_min`, `malware_min`, `phishing_min`, `spam_min`, `proximity_min`: Integer risk score thresholds (range `1` to `99`, optional). Available on the `realtime_domain_risk` and `domainhotlist` feeds only. When multiple are supplied they act as a logical AND — a domain must meet ALL specified thresholds to be returned.
+
+    ```python
+    api = API(USERNAME, KEY)
+    api.domainhotlist(after=-3600, overall_min=70, phishing_min=50)
+    ```
+
+- IP feed filters (available on the `iprisk` and `iphotlist` feeds only). All are optional integers/strings and combine as a logical AND:
+    - Domain activity & volume: `pdns_resolutions_min`, `bad_pdns_resolutions_min` (positive integers, distinct/bad domains resolving to the IP in the last 24 hours) and `total_domains_max` (positive integer; caps total hosted domains to filter out superhosters like CDNs).
+    - Threat intelligence & combined risk percentages: `third_party_threats_min` (positive integer), plus `all_threats_combined_percent_min`, `combined_phishing_percent_min`, `combined_malware_percent_min`, `combined_spam_percent_min` (percentages `0` to `100` of hosted domains confirmed or predicted malicious).
+    - Confirmed threat percentages: `all_threats_percent_min`, `percent_phishing_min`, `percent_malware_min`, `percent_spam_min` (percentages `0` to `100` of hosted domains actively confirmed).
+    - Infrastructure & geolocation: `asn` (integer, digits only — no `AS` prefix or wildcards), `organization` (exact name, no wildcards) and `country_code` (case-sensitive two-letter code, e.g. `CN`, `US`, `NL`).
+
+    ```python
+    api = API(USERNAME, KEY)
+    api.iprisk(after=-3600, bad_pdns_resolutions_min=5, total_domains_max=1000, country_code="US")
+    ```
+
+#### Result formatting parameters
+
+- `output_format`: `csv` or `jsonl` (default `jsonl`). Not available on the `domainrdap` feed. `csv` is not available for `download` endpoints.
+- `headers`: When `csv` output is used, adds a header row to the first line of the response.
+- `top`: Positive integer from `1` to `1,000,000,000` limiting the number of results in the response payload. Ignored for the `download` endpoint.
+
+#### Download-only parameters
+
+These parameters are only accepted when `endpoint="download"`. They are ignored for the `feed` endpoint.
+
+- `limit`: Maximum number of files to return in the response.
+- `page`: Zero-indexed page of results to return. Available on `realtime_domain_risk`, `domainhotlist`, `iphotlist`, and `iprisk`.
+- `prefix`: Filter files by date prefix (e.g. `"2026-08-"`). Available on `realtime_domain_risk`, `domainhotlist`, `iphotlist`, and `iprisk`.
+
+```python
+api = API(USERNAME, KEY)
+api.iphotlist(endpoint="download", limit=10, page=0, prefix="2026-08-")
+```
+
+## Handling iterative response from RTTF endpoints:
 
-Since we may dealing with large feeds datasets, the python wrapper uses `generator` for efficient memory handling. Therefore, we need to iterate through the `generator` if we're accessing the partial results of the feeds data.
+Since we may be dealing with large feeds datasets, the python wrapper uses `generator` for efficient memory handling. Therefore, we need to iterate through the `generator` if we're accessing the partial results of the feeds data.
 
 ### Single request because the requested data is within the maximum result:
 ```python

VERSION

@@ -1 +1 @@
-2.9.0
+2.10.0

domaintools/_version.py

@@ -20,4 +20,4 @@
 
 """
 
-current = "2.9.0"
+current = "2.10.0"