An open protocol for making websites agent-native
AI agents browsing the web today have to scrape HTML, guess which button to click, and fight CAPTCHAs. AgentWebGate adds a structured agent interface to any FastAPI or Django app in ~10 lines of code — a typed, authenticated endpoint that agents call directly, no browser interface required.
from fastapi import FastAPI
from agent_web_gate import AgentWebGate
app = FastAPI()
gate = AgentWebGate(app, name="My Store", auth="api_key", admin_key="secret")
gate.add_role("reader", actions=["search", "browse"], auto_approve=True)
gate.add_role("buyer", actions=["search", "checkout"], auto_approve=False)
@gate.action("search", description="Search the product catalog.")
def search(query: str, limit: int = 10) -> list:
return db.search(query, limit)Agents can now discover and call your site's capabilities without touching the DOM.
AgentWebGate registers two things on your app:
GET /.well-known/agent — a public manifest agents check first:
{
"name": "My Store",
"actions": [
{ "id": "search", "description": "Search the product catalog.", "input_schema": { ... } }
],
"authentication": { "type": "api_key", "key_request_endpoint": "POST /.well-known/agent/auth/request" }
}POST /.well-known/agent/{action_id} — authenticated action endpoints:
# Agent requests a key (auto-approved for low-risk roles)
curl -X POST https://yoursite.com/.well-known/agent/auth/request \
-H "Content-Type: application/json" \
-d '{"agent_id": "openai-shopper", "description": "Shopping assistant", "role": "reader"}'
# → {"status": "approved", "api_key": "agk_..."}
# Agent calls an action
curl -X POST https://yoursite.com/.well-known/agent/search \
-H "X-Agent-Key: agk_..." \
-H "Content-Type: application/json" \
-d '{"query": "wireless keyboard"}'
# → {"result": [...]}pip install gentle-webRequires Python 3.10+, FastAPI 0.100+, Pydantic v2.
| Manifest | Auto-generated /.well-known/agent — agents discover what your site supports |
| Self-service keys | Agents request keys for named roles; no manual provisioning |
| Role-based access | Define roles with action scopes, like GitHub App permissions |
| Human approval gate | Sensitive roles (buyer, admin) sit pending until you approve |
| Rate limiting | Per-key sliding window, configured per site |
| Monitoring dashboard | /_agentwebgate/ — live logs, key management, approve/deny/revoke |
| Async support | Decorated functions can be sync or async |
| Django adapter | from agent_web_gate import AgentWebGateDjango |
You define what each role can do. Agents request a role when they ask for a key. You decide whether that role auto-approves or requires sign-off.
# Agents get these keys immediately
gate.add_role("reader", actions=["search", "get_product", "check_inventory"], auto_approve=True)
# These sit in the dashboard until a human approves
gate.add_role("buyer", actions=["search", "checkout", "track_order"], auto_approve=False)The admin dashboard at /_agentwebgate/ shows all pending requests. One click to approve or deny.
Every agent call is logged to SQLite. The dashboard at /_agentwebgate/?admin_key=... shows:
- Total calls, error rate, average latency
- Per-action breakdown
- Full request/response log per call
- All API keys with status (approved / pending / denied / revoked)
# urls.py
from agent_web_gate import AgentWebGateDjango
gate = AgentWebGateDjango(name="My Site", db_path="agentwebgate.db", admin_key="secret")
gate.add_role("reader", actions=["search"], auto_approve=True)
@gate.action("search")
def search(query: str) -> list:
...
urlpatterns = [
*gate.get_urls(),
# ... your other urls
]Browser automation (Playwright, OpenAI Operator, Anthropic Computer Use) works on any site but:
- Breaks when UI changes
- Triggers CAPTCHAs and bot detection
- Gives you zero visibility into what the agent did
- Slow and expensive per-interaction
AgentWebGate gives agents a stable typed interface. You control exactly what they can do and see everything they do.
The web has robots.txt for crawlers and sitemap.xml for search engines. It doesn't have a standard interface for AI agents. AgentWebGate is a lightweight attempt at filling that gap — a /.well-known/agent endpoint that any site can expose, and any agent can discover.
Instead of every AI company building better HTML scrapers, site owners declare what agents are allowed to do. Same shift as robots.txt for search, DKIM for email, and Stripe for payments — move the protocol down to the infrastructure layer.
MIT