feat(tools): add SkimReaderTool (x402-paid clean web reader)#6266
Open
JessieJanie wants to merge 2 commits into
Open
feat(tools): add SkimReaderTool (x402-paid clean web reader)#6266JessieJanie wants to merge 2 commits into
JessieJanie wants to merge 2 commits into
Conversation
Adds SkimReaderTool, which fetches any URL and returns clean, agent-ready Markdown plus structured metadata via Skim (skim402.com). Each call is paid automatically over the x402 protocol ($0.002 in USDC on Base) using a wallet the user controls; no API keys or signup. The private key signs USDC payment authorizations locally and is never transmitted. - lib/crewai-tools/src/crewai_tools/tools/skim_reader_tool/ (tool + README) - wires exports in tools/__init__.py and crewai_tools/__init__.py - adds the optional 'x402' extra in lib/crewai-tools/pyproject.toml - adds tests/tools/skim_reader_tool_test.py (fully mocked, offline)
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdds a new ChangesSkimReaderTool
Sequence DiagramsequenceDiagram
participant Agent
participant SkimReaderTool
participant _get_session
participant x402Session
participant SkimAPI
Agent->>SkimReaderTool: _run(url)
SkimReaderTool->>SkimReaderTool: validate_url(url)
SkimReaderTool->>_get_session: request cached session
_get_session->>_get_session: import x402/EVM deps, validate private key (hex-64)
_get_session->>x402Session: wrap requests.Session with EVM payment policy (USDC cap)
_get_session-->>SkimReaderTool: payment-aware Session
SkimReaderTool->>SkimAPI: POST /api/v1/read {url, mode: "basic"}
SkimAPI-->>SkimReaderTool: JSON {markdown, metadata}
SkimReaderTool->>SkimReaderTool: extract markdown/text, build YAML frontmatter (if include_metadata)
SkimReaderTool-->>Agent: Markdown string
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![corridor-security[bot]](https://avatars.githubusercontent.com/in/1177339?s=60&v=4){kind=small}
There was a problem hiding this comment.
Security Issues
- Software Supply Chain Failure
The newx402extra declares direct dependencies with open-ended lower-bound-only specifiers and no corresponding lockfile update in this diff. Installingcrewai-tools[x402]can resolve arbitrary future upstream releases, including compromised or malicious packages.
Summary: This PR adds a paid Skim web reader tool and introduces new optional third-party dependencies for x402/EVM payment support.
Risk: Medium risk. The runtime target URL is validated before use, but the new user-installable extra weakens the repository’s supply-chain guardrails by allowing unbounded dependency resolution.
Recommendations:
- Replace the open-ended dependency specifiers with bounded constraints and regenerate/commit the root
uv.lockso installs remain reproducible.
| "eth-account>=0.13.0", | ||
| "requests>=2.31.0", | ||
| ] | ||
|
|
There was a problem hiding this comment.
The new optional extra uses lower-bound-only dependency specifiers:
x402 = [
"x402[evm]>=2.0.0",
"eth-account>=0.13.0",
"requests>=2.31.0",
]These are direct dependencies resolved when users install crewai-tools[x402]; without upper bounds and an updated root uv.lock, a future compromised or malicious release could be pulled into builds automatically. The project supply-chain guardrail requires bounded (~= or >=,<) constraints and committed lockfile updates.
Remediation: Use bounded version constraints for each new direct dependency and regenerate/commit the repository uv.lock.
For more details, see the finding in Corridor.
Provide feedback: Reply with whether this is a valid vulnerability or false positive to help improve Corridor's accuracy.
…ation tests Adds an offline (mocked) test that _get_session builds and caches the x402-wrapped requests session, and a test asserting _run validates the URL before issuing the request. Addresses review feedback.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
lib/crewai-tools/tests/tools/skim_reader_tool_test.py (1)
77-81: ⚡ Quick winAssert request timeout to lock the call contract.
_run()passestimeout=self.timeout; addingassert call["timeout"] == 60.0here prevents silent regressions in request behavior.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@lib/crewai-tools/tests/tools/skim_reader_tool_test.py` around lines 77 - 81, The test is not verifying that the timeout parameter is passed correctly to the request call. After the existing assertions for endpoint and json in the test block (following the assertion on the json parameter), add a new assertion to verify that the timeout value equals the expected value of 60.0 by checking call["timeout"]. This ensures that the timeout parameter passed by _run() is correctly captured and prevents regressions in request behavior.lib/crewai-tools/src/crewai_tools/tools/skim_reader_tool/skim_reader_tool.py (1)
151-158: 💤 Low valueConsider case-insensitive handling of
0xprefix.The prefix check only handles lowercase
0x. Users occasionally provide keys with uppercase0X, which would fail validation even though it's a valid hex prefix in Ethereum tooling.Suggested fix
- normalized = key[2:] if key.startswith("0x") else key + normalized = key[2:] if key.lower().startswith("0x") else key🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@lib/crewai-tools/src/crewai_tools/tools/skim_reader_tool/skim_reader_tool.py` around lines 151 - 158, The prefix validation in the skim_reader_tool.py file only checks for lowercase "0x" prefix using key.startswith("0x"), which means uppercase "0X" prefixes will not be recognized and will fail validation. Modify the prefix check to be case-insensitive by either converting the key to lowercase before checking (key.lower().startswith("0x")) or explicitly checking for both variations using key.startswith(("0x", "0X")). This ensures that both "0x" and "0X" prefixes are properly stripped before validation, allowing users to provide private keys with either case format.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@lib/crewai-tools/tests/tools/skim_reader_tool_test.py`:
- Around line 109-122: The tests test_get_session_requires_a_key and
test_get_session_rejects_malformed_key fail non-deterministically because
_get_session() performs dynamic imports of x402 and eth-account modules before
validating the key parameters, causing ImportError in environments without these
optional extras rather than the expected ValueError. Monkeypatch the
importlib.import_module function (or whatever dynamic import mechanism is used
within _get_session) to return mock objects so that the key validation logic is
tested in isolation without being blocked by missing optional dependencies. Add
the monkeypatch fixture parameter to both test functions and configure it to
intercept the import calls within _get_session.
---
Nitpick comments:
In
`@lib/crewai-tools/src/crewai_tools/tools/skim_reader_tool/skim_reader_tool.py`:
- Around line 151-158: The prefix validation in the skim_reader_tool.py file
only checks for lowercase "0x" prefix using key.startswith("0x"), which means
uppercase "0X" prefixes will not be recognized and will fail validation. Modify
the prefix check to be case-insensitive by either converting the key to
lowercase before checking (key.lower().startswith("0x")) or explicitly checking
for both variations using key.startswith(("0x", "0X")). This ensures that both
"0x" and "0X" prefixes are properly stripped before validation, allowing users
to provide private keys with either case format.
In `@lib/crewai-tools/tests/tools/skim_reader_tool_test.py`:
- Around line 77-81: The test is not verifying that the timeout parameter is
passed correctly to the request call. After the existing assertions for endpoint
and json in the test block (following the assertion on the json parameter), add
a new assertion to verify that the timeout value equals the expected value of
60.0 by checking call["timeout"]. This ensures that the timeout parameter passed
by _run() is correctly captured and prevents regressions in request behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 523bd38d-9f30-4a1b-b196-0868c5f4cb95
📒 Files selected for processing (7)
lib/crewai-tools/pyproject.tomllib/crewai-tools/src/crewai_tools/__init__.pylib/crewai-tools/src/crewai_tools/tools/__init__.pylib/crewai-tools/src/crewai_tools/tools/skim_reader_tool/README.mdlib/crewai-tools/src/crewai_tools/tools/skim_reader_tool/__init__.pylib/crewai-tools/src/crewai_tools/tools/skim_reader_tool/skim_reader_tool.pylib/crewai-tools/tests/tools/skim_reader_tool_test.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
SkimReaderTool— a tool that fetches any URL and returns clean, agent-ready Markdown plus structured metadata (title, byline, published date, language, excerpt) via Skim, the x402-native clean reader API for AI agents.Reads are paid per call over the x402 protocol ($0.002 in USDC on Base) using a wallet the user controls — no API keys, no signup. The wallet private key is used only to sign USDC payment authorizations locally and is never transmitted. This mirrors the existing reader/scraper tools (e.g.
FirecrawlScrapeWebsiteTool) but with x402 micropayments instead of API keys.What's included
lib/crewai-tools/src/crewai_tools/tools/skim_reader_tool/— the tool, package__init__.py, andREADME.mdtools/__init__.pyandcrewai_tools/__init__.py(alphabetical)x402extra inlib/crewai-tools/pyproject.toml(x402[evm],eth-account,requests)tests/tools/skim_reader_tool_test.py— fully mocked and offline (no network, no real payments), compatible with the--block-networktest policyUsage
The x402 client dependencies are imported lazily, so importing
crewai_toolswithout thex402extra installed is unaffected; the tool raises a clear error only when actually run without the extra or a wallet key.Notes
ruff checkandruff format --checkpass locally against the repo config (ruff 0.15.1).tool.specs.jsonsince the spec-generation workflow runs from the base repo on merge; happy to add it if maintainers prefer it in the PR.Summary by CodeRabbit
x402optional dependency group.