from googlenewsdecoder import gnewsdecoder fails on a fresh install with:
ImportError: Modest backend is deprecated since selectolax 1.0. It's outdated, not maintained,
contains bugs and does not follow modern HTML5 standards. Please use lexbor backend instead
The free fix: keep selectolax below 1.0.
pip install googlenewsdecoder "selectolax<1.0"Why: googlenewsdecoder 0.2.1 asks for selectolax>=0.4.12 with no upper bound, and selectolax 1.0.0
(PyPI, 2026-10-03) removed the Modest backend that googlenewsdecoder imports. We re-checked it on
2026-10-17 in a clean Python 3.13 virtualenv: pip install googlenewsdecoder pulls selectolax 1.0.0 and the
import fails; after pip install "selectolax<1.0" (0.4.13) it imports. The upstream report is
SSujitX/google-news-url-decoder#20.
This repo turns news.google.com/rss/articles/... links into the publisher's own article URL, using only
Python's standard library. It calls
hiver's Google News Scraper
on Apify with your own Apify token. The scripts are free; the Actor charges $0.80 per 1,000 decoded links
on Apify's free plan (less on paid plans). Links it cannot decode come back as rows with the reason and
are not charged.
Why not decode locally? googlenewsdecoder works once installed, but each link needs requests to Google,
and after a burst Google answers HTTP 429. The Actor paces its requests and returns links Google refused
as free not_decoded: Google rate-limited, retry later rows instead of failing the run, so a batch never
dies halfway through; re-send those links later. It does not rotate IPs or otherwise get around a refusal.
Two ways to use it: decode_links.py (links or a feed in, CSV out) or hiver_gnewsdecoder, a
function-level replacement for gnewsdecoder() (see "Drop-in replacement" below).
export APIFY_TOKEN=... # Apify Console > Settings > API & Integrations
python decode_links.py links.txt # one link per line; blank lines and # comments are ignored
python decode_links.py --feed "https://news.google.com/rss/search?q=federal+reserve&hl=en-US&gl=US&ceid=US:en"Accepted: /rss/articles/<id>, /articles/<id> and /read/<id> links (with or without ?oc=5), and
Google News RSS feed URLs (/rss/search, /rss/topics, /rss/headlines), whose items are decoded in
feed order. A link given twice is returned once. --max-items caps the rows (default: the number of links, at least 100, at most 5,000; error rows
count) and
--max-charge stops the run at a dollar amount (default $0.50).
Output decoded_links.csv:
| column | |
|---|---|
inputUrl |
the link you gave (or the feed item's link) |
url |
the publisher's article URL; empty when not decoded |
publisherDomain |
e.g. cnbc.com |
status |
decoded, not_google_news_link or not_decoded: <reason> |
title, publisher, publishedAt |
filled for feed items; empty for pasted links |
Our test run of the Actor (build 0.1.16, 2026-10-12, run OWYXDZh9msX4hICmK) got 8 links:
5 Google News article links, the first one again without ?oc=5, a made-up id and a non-Google URL.
It returned 7 rows in 15 seconds and Apify charged 5 decoded-link events:
inputUrl (shortened) |
url |
status |
|---|---|---|
| news.google.com/rss/articles/CBMiowFB...?oc=5 | https://www.scotusblog.com/2026/10/trump-announces-committee-investigation-of-fed-governor-lisa-cook/ | decoded |
| news.google.com/rss/articles/CBMipwFB...?oc=5 | https://www.aljazeera.com/economy/2026/10/9/trump-launches-probe-into-federal-reserve-governor-lisa-cook | decoded |
| news.google.com/articles/CBMiiAFB... | https://www.nbcnews.com/business/economy/trump-probe-fed-gov-lisa-cook-rcna602500 | decoded |
| news.google.com/rss/articles/CBMib0FV...?oc=5 | https://www.cnbc.com/2026/10/09/american-debt-delinquency.html | decoded |
| news.google.com/rss/articles/CBMi1wFB...?oc=5 | https://www.deutschlandfunk.de/bundesbank-vorstandsmitglied-theurer-sieht-noch-keine-systemische-staatsschuldenkrise-aber-warnsigna-100.html | decoded |
| news.google.com/rss/articles/CBMiNOTAREALID... | (empty) | not_decoded: Google did not return a publisher URL |
| example.com/news/story | (empty) | not_google_news_link |
Google decides how fast it answers: in our run of the previous build 25 minutes earlier, with the same 8 links, Google refused that worker from the start, so all 7 rows came back free and undecoded and the run still succeeded. That is what the free rows are for.
For Python 3.10+, install this function-level replacement (standard library at runtime):
python -m venv .venv
. .venv/bin/activate
python -m pip install "git+https://github.com/hiver-data/google-news-decoder.git"
export APIFY_TOKEN=... # your own Apify account; calls below use paid Actor runsChange only your import; existing function calls can stay:
from hiver_gnewsdecoder import gnewsdecoder # replaces: from googlenewsdecoder import gnewsdecoder
result = gnewsdecoder(source_url) # str -> one dict
results = gnewsdecoder([source_url, source_url]) # list -> ordered list, duplicates retained
# {"success": True, "decoded_url": "https://publisher.example/article?..."}
# or {"success": False, "message": "Google rate-limited, retry later"}from hiver_gnewsdecoder import gnews_decoder_async provides the same shapes with await.
One nonempty batch containing article links starts one Actor run, not one run per link;
empty lists and invalid/feed-only inputs start none. Unlike the CSV CLI, the shim does not
expand feeds. Lists are capped at 5,000 inputs; split larger lists explicitly.
interval, proxy and async concurrency are accepted but ignored: pacing is Actor-side,
no client proxy is used, and refusals are never automatically retried.
timeout (default 10.0, or None) is per Apify HTTP operation, not total run duration.
Missing tokens and invalid argument types/limits raise ValueError/TypeError; HTTP/run
failures return failure dicts without retrying. Check Apify Console before resubmitting:
a lost response or cancelled async await does not stop an already-started run.
Optional keyword-only apify_token= overrides APIFY_TOKEN; max_charge=0.50 sets the
run's USD cap. The import swap changes where execution happens: your account pays the
Actor's Store rate and any applicable Apify charges. It is not a free local decoder or
a replacement for GoogleDecoder/GoogleDecoderAsync classes. No dependency on
googlenewsdecoder, selectolax or httpx, even if those packages cannot import.
From a clone, python -m pip install ., then run the Tests command below.
Tests use the verbatim input and rows of the Actor's test run 737RxMYPBd1ccXsKk (five decoded, two
errors, duplicate input restored). Those five publisher URLs have no query strings;
three separate real Google responses test query preservation through explicitly
reconstructed Apify envelopes. They are not additional decoded rows from that run. All tests
are offline; there is no live paid-run verification of this shim.
python -m unittest discover -s tests -vThe fixtures are real rows from two runs of the Actor's code made on 2026-10-11 (one RSS search feed, 12 links decoded; and five pasted links: two decoded, one duplicate collapsed, one bad id and one non-Google URL as free rows). The tests use a fake Apify API built from those rows; they never call Apify or Google.
Want Google News alerts in Slack or Discord instead? See hiver-data/google-news-alerts.
MIT licensed. Unofficial. Not affiliated with Google or with the googlenewsdecoder project.