fix(ci): repair dev live-data test failures

[Elarwei001]

Summary

This draft consolidates the CI/test-stability fixes for the current dev hatch-test failures into one branch so the GitHub test matrix can be evaluated in a single PR.

Included fixes:

• OpenTargets drugs: add GraphQL sub-selections for synonyms and tradeNames so the live drugs query no longer returns HTTP 400, then flatten those fields back to lists of labels to preserve the existing gget output shape.
• ARCHS4 tissue expression: tolerate the upstream CSV omitting optional color; keep required expression columns and sorting.
• OpenTargets live-data tests: replace exact snapshots for drifting live resources with structural/semantic invariants for diseases, DepMap, interactions, pharmacogenetics, drugs, and ARCHS4.
• ELM setup in tests: retry transient live ELM database download failures instead of immediately failing collection.

Not included:

• OpenTargets resource="expression" is not migrated to baselineExpression in this PR. That would change user-visible output semantics for the same call signature, so it remains a separate design question tracked in gget opentargets -r expression returns empty: OpenTargets retired target.expressions (data moved to baselineExpression) #247.
• The two existing expression live fixtures are now explicitly skipped because OpenTargets target.expressions currently returns no rows; the skip message points to gget opentargets -r expression returns empty: OpenTargets retired target.expressions (data moved to baselineExpression) #247 rather than silently changing gget semantics inside this CI-repair PR.
• The existing mypy/pre-commit baseline is not addressed here.

Live contracts being tested

• Drugs: live results must expose valid CHEMBL ids, names, synonym lists, indication rows, and the expected GraphQL columns without HTTP 400.
• ARCHS4 tissue: live tissue results must return id/min/q1/median/q3/max, numeric quartile ordering, median-descending sort, and no dependency on optional color.
• OpenTargets drift tests: diseases, DepMap, interactions, and pharmacogenetics assert stable schema/id/score/filter invariants instead of pinning release-specific values.
• ELM setup: transient download failures get a small number of retries; persistent upstream failure still fails the test rather than being hidden.

Local verification

Passing locally:

~/.local/bin/hatch test -py 3.12,3.13,3.14 tests/test_opentargets.py tests/test_archs4.py tests/test_elm.py
# py3.12: 31 passed, 2 skipped
# py3.13: 31 passed, 2 skipped
# py3.14: 31 passed, 2 skipped

uvx pre-commit run ruff-check --files gget/gget_opentargets.py tests/test_opentargets.py tests/fixtures/test_opentargets.json docs/src/en/opentargets.md docs/src/en/updates.md
# Passed

uvx pre-commit run ruff-format --files gget/gget_opentargets.py tests/test_opentargets.py
# Passed

The two skipped tests are the known OpenTargets expression upstream break tracked in #247.

Related

• Addresses CI on dev is failing (unrelated to incoming PRs): OpenTargets synonyms HTTP 400 + opentargets fixture drift + flaky archs4 — masks/blocks all PRs #243
• Addresses OpenTargets tests pin exact live values that drift across data releases (diseases/depmap/interactions/pharmacogenetics) #249
• Leaves gget opentargets -r expression returns empty: OpenTargets retired target.expressions (data moved to baselineExpression) #247 open for an explicit maintainer decision about whether/how to expose OpenTargets baselineExpression.
• Replaces the non-expression parts of draft PRs fix(opentargets): give drug synonyms/tradeNames a GraphQL sub-selection (HTTP 400 fix) #244, fix(archs4): tolerate missing color column from ARCHS4 (no crash) #246, and test(opentargets): loosen live-data assertions to structural/invariant (data drifts across releases) #250. The earlier expression PR fix(opentargets): repoint expression to baselineExpression (upstream retired target.expressions) #248 was closed; its semantic migration is intentionally not included here.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 90.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 56.40%. Comparing base (5cf607f) to head (2a8a2b7).
⚠️ Report is 1 commits behind head on dev.

Files with missing lines	Patch %	Lines
gget/gget_opentargets.py	88.88%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##              dev     #252      +/-   ##
==========================================
+ Coverage   56.14%   56.40%   +0.25%     
==========================================
  Files          29       29              
  Lines        9244     9253       +9     
==========================================
+ Hits         5190     5219      +29     
+ Misses       4054     4034      -20     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Elarwei001]

currently opentarget change the format of its resource="drugs" response from "string" to a "DrugLabelAndSource" object. the response example looks like this:

`query: { drug(chemblId: "CHEMBL1201631") { name synonyms { label source } tradeNames { label source } } }

HTTP 200
name: INSULIN HUMAN

synonyms (first 8, now WITH source):
{'label': 'Biosulin r', 'source': 'ChEMBL'}
{'label': 'Capsulin', 'source': 'ChEMBL'}
{'label': 'Exubera (inhaled insulin human)', 'source': 'ChEMBL'}
{'label': 'High molecular weight insulin human', 'source': 'ChEMBL'}
{'label': 'Human insulin', 'source': 'ChEMBL'}
{'label': 'Humulin', 'source': 'ChEMBL'}
{'label': 'Insulin', 'source': 'ChEMBL'}
{'label': 'Insulin (human)', 'source': 'ChEMBL'}

tradeNames (first 8, now WITH source):
{'label': 'Actrapid', 'source': 'ChEMBL'}
{'label': 'Afrezza', 'source': 'ChEMBL'}
{'label': 'Exubera', 'source': 'ChEMBL'}
{'label': 'Humulin br', 'source': 'ChEMBL'}
{'label': 'Humulin r', 'source': 'ChEMBL'}
{'label': 'Inpremzia', 'source': 'ChEMBL'}
{'label': 'Ins humulin r', 'source': 'ChEMBL'}
{'label': 'Insuman', 'source': 'ChEMBL'}

-> Each entry now shows {'label': ..., 'source': ...}. So source was
always available; the earlier output omitted it only because the query
selected { label } (and gget intentionally keeps just label to
preserve its historical plain-string output).`

so, will need to change the decoding structure to capture current response labels.

test code:

#!/usr/bin/env python3
"""
Show that DrugLabelAndSource really has BOTH `label` and `source` —
we only saw `label` before because our query only selected `label`.
GraphQL returns exactly the fields you request, nothing more.

Read-only OpenTargets API call. No GitHub, no writes.
Run: /Users/elar/gget/.venv/bin/python /Users/elar/cc_output/gget/verify-opentargets-source.py
"""
import json

import requests

URL = "https://api.platform.opentargets.org/api/v4/graphql"
DRUG = "CHEMBL1201631"

# Note: now we ALSO request `source` in the sub-selection.
q = f'{{ drug(chemblId: "{DRUG}") {{ name synonyms {{ label source }} tradeNames {{ label source }} }} }}'
print("query:", q, "\n")

r = requests.post(URL, json={"query": q}, timeout=60)
print("HTTP", r.status_code)
drug = (r.json().get("data") or {}).get("drug") or {}
print("name:", drug.get("name"), "\n")

print("synonyms (first 8, now WITH source):")
for item in (drug.get("synonyms") or [])[:8]:
    print("   ", item)

print("\ntradeNames (first 8, now WITH source):")
for item in (drug.get("tradeNames") or [])[:8]:
    print("   ", item)

print(
    "\n-> Each entry now shows {'label': ..., 'source': ...}. So `source` was"
    "\n   always available; the earlier output omitted it only because the query"
    "\n   selected `{ label }` (and gget intentionally keeps just `label` to"
    "\n   preserve its historical plain-string output)."
)

[Elarwei001]

try the GENE's associate diseases, the result may change (which means opentarget's response won't be stable).

for example, if compare with the previous test case in dev, current value is update both on ID and Score:

test code:

#!/usr/bin/env python3
"""
Measure the ACTUAL drift of `gget opentargets -r diseases` for IL13
(ENSG00000169194, the gene used in test_opentargets), so we can design a
RIGOROUS test (known-ID-set + score tolerance), not a weak one.

Compares the live OpenTargets top-2 associated diseases against the OLD
fixture values that PR #250 deleted.

Read-only OpenTargets API. No GitHub, no writes.
Run: /Users/elar/gget/.venv/bin/python /Users/elar/cc_output/gget/verify-opentargets-diseases-drift.py
"""
import json

import requests

URL = "https://api.platform.opentargets.org/api/v4/graphql"
GENE = "ENSG00000169194"  # IL13

# What the OLD fixture asserted (the 171 lines PR #250 removed), top 2:
OLD = [
    {"score": 0.7297489019498119, "id": "EFO_0000274", "name": "atopic eczema"},
    {"score": 0.6642728577751653, "id": "MONDO_0004979", "name": "asthma"},
]

q = f"""
{{ target(ensemblId: "{GENE}") {{
     associatedDiseases(page: {{index: 0, size: 2}}) {{
       count
       rows {{ score disease {{ id name }} }}
     }}
}} }}
"""

r = requests.post(URL, json={"query": q}, timeout=60)
print("HTTP", r.status_code)
body = r.json()
rows = (((body.get("data") or {}).get("target") or {}).get("associatedDiseases") or {}).get("rows")
if not rows:
    print(json.dumps(body, indent=2)[:800])
    raise SystemExit("no rows — inspect raw response above")

print(f"\n{'rank':<5}{'LIVE id':<16}{'LIVE name':<22}{'LIVE score':<14}"
      f"{'OLD id':<16}{'OLD score':<14}{'Δscore':<12}{'id changed?'}")
print("-" * 110)
for i, row in enumerate(rows):
    live_id = row["disease"]["id"]
    live_name = row["disease"]["name"]
    live_score = row["score"]
    old = OLD[i] if i < len(OLD) else {"id": "-", "score": float("nan"), "name": "-"}
    dscore = live_score - old["score"]
    rel = abs(dscore) / old["score"] if old["score"] else float("nan")
    idchg = "SAME" if live_id == old["id"] else f"{old['id']} -> {live_id}"
    print(f"{i:<5}{live_id:<16}{live_name:<22}{live_score:<14.6f}"
          f"{old['id']:<16}{old['score']:<14.6f}{dscore:<+12.6f}{idchg}")
    print(f"     (relative score drift this row: {rel*100:.3f}%)")

print("\nUse this to decide: (a) the acceptable ID SET per disease (did EFO<->MONDO"
      "\nflip, or is the name stable while id changed?), and (b) a score TOLERANCE"
      "\nthat survives normal release drift but still catches wrong-data. Run on a"
      "\nfew genes / a few days to see the real drift magnitude before fixing the test.")