fix: resolve relative paths before converting to URI#10841
fix: resolve relative paths before converting to URI#10841CyberRaccoonTeam wants to merge 2 commits intopython-poetry:mainfrom
Conversation
Fixes #10765 The kind/question label does not exist in the Poetry repository. Removed the broken reference from the Documentation contributions section and the link reference list.
Fixes #10830 When package.source_url is a relative path, calling Path().as_uri() directly raises: "ValueError: relative path can't be expressed as a file URI". This affects users with nested git dependencies using subdirectory arguments, where the path can be relative to the working directory. Changes: - Added .resolve() before .as_uri() in _create_file_url_reference() - Added .resolve() before .as_uri() in _create_directory_url_reference() - Added tests to verify relative paths are properly handled The .resolve() method converts relative paths to absolute paths before converting to file:// URIs, preventing the ValueError.
There was a problem hiding this comment.
Hey - I've found 2 issues, and left some high level feedback:
- Using
Path(...).resolve()will follow symlinks and will raise if the path does not exist; if the intent is only to normalize relative paths into absolute ones without enforcing existence or resolving symlinks, considerPath(...).absolute()instead. - In the new tests, the relative paths are computed with
os.path.relpath(..., tmp_path)but the process cwd is not changed totmp_path, so theresolve()call will interpret them relative to the actual cwd; consider eitherchdir(tmp_path)or computing the relative path from the current working directory to avoid accidental false positives.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Using `Path(...).resolve()` will follow symlinks and will raise if the path does not exist; if the intent is only to normalize relative paths into absolute ones without enforcing existence or resolving symlinks, consider `Path(...).absolute()` instead.
- In the new tests, the relative paths are computed with `os.path.relpath(..., tmp_path)` but the process cwd is not changed to `tmp_path`, so the `resolve()` call will interpret them relative to the actual cwd; consider either `chdir(tmp_path)` or computing the relative path from the current working directory to avoid accidental false positives.
## Individual Comments
### Comment 1
<location path="tests/installation/test_executor.py" line_range="1966-1975" />
<code_context>
+def test_executor_should_handle_relative_paths_in_directory_url_reference(
</code_context>
<issue_to_address>
**suggestion (testing):** Strengthen the assertion to verify the resolved path matches the expected directory, not just that the URI is syntactically valid.
The current test only asserts that the URI is a syntactically valid `file://` URL, which only proves that no `ValueError` is raised. To verify correctness, parse the URI path back into a `Path` and assert it equals `Path(project_dir).resolve()`, so the test confirms the resolved directory is correct, not just well-formed.
Suggested implementation:
```python
import shutil
import tempfile
from urllib.parse import urlparse
```
```python
def test_executor_should_handle_relative_paths_in_directory_url_reference(
tmp_path: Path,
tmp_venv: VirtualEnv,
pool: RepositoryPool,
config: Config,
io: BufferedIO,
fixture_dir: FixtureDirGetter,
) -> None:
"""Test that relative paths are properly resolved before converting to URI."""
# Create a package with a relative path (simulating the bug scenario)
project_dir = fixture_dir("simple_project")
# NOTE: this test should obtain a directory URI (e.g. from the executor) into `directory_uri`.
# Once `directory_uri` is available, strengthen the assertion to verify the resolved path.
# Example of a stronger assertion (assuming `directory_uri` is the file:// URI under test):
#
# parsed = urlparse(directory_uri)
# resolved_path = Path(parsed.path).resolve()
# assert resolved_path == Path(project_dir).resolve()
```
To fully implement the stronger assertion, you should:
1. Replace the placeholder comment and example in the test body with real code that:
- Calls the code under test to obtain the `file://` directory URI (whatever variable currently holds it).
- Parses the URI using `urlparse`.
- Converts the parsed path to a `Path` and resolves it.
- Asserts that this resolved `Path` equals `Path(project_dir).resolve()`.
2. Concretely, if your existing test currently has something like:
- `directory_uri = <call_under_test>(...)`
- `assert directory_uri.startswith("file://")`
you should change it to:
- Keep obtaining `directory_uri` the same way.
- Replace the simple `startswith` assertion with:
```python
parsed = urlparse(directory_uri)
resolved_path = Path(parsed.path).resolve()
assert resolved_path == Path(project_dir).resolve()
```
(Import `Path` from `pathlib` if not already imported in this file.)
Adjust the variable name `directory_uri` in the snippet above to match the actual variable used in your test.
</issue_to_address>
### Comment 2
<location path="tests/installation/test_executor.py" line_range="1976-1977" />
<code_context>
+) -> None:
+ """Test that relative paths are properly resolved before converting to URI."""
+ # Create a package with a relative path (simulating the bug scenario)
+ project_dir = fixture_dir("simple_project")
+ relative_path = os.path.relpath(project_dir, tmp_path)
+
+ package = Package(
</code_context>
<issue_to_address>
**suggestion (bug_risk):** The way the relative path is constructed may not reflect how the executor actually resolves paths (cwd vs tmp_path).
`relative_path` is computed with `os.path.relpath(project_dir, tmp_path)`, but `Path(package.source_url).resolve()` is relative to the process cwd, not `tmp_path`. Unless the test explicitly sets `cwd` to `tmp_path`, this doesn’t match actual resolution behavior and may point somewhere unexpected.
Consider either:
1) Changing the cwd in the test (e.g. `monkeypatch.chdir(tmp_path)`) and then computing the relative path from that cwd, or
2) Using a simple relative path (e.g. `os.path.relpath(project_dir)` with no `start`) and asserting that the resolved URI maps back to `project_dir`.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| def test_executor_should_handle_relative_paths_in_directory_url_reference( | ||
| tmp_path: Path, | ||
| tmp_venv: VirtualEnv, | ||
| pool: RepositoryPool, | ||
| config: Config, | ||
| io: BufferedIO, | ||
| fixture_dir: FixtureDirGetter, | ||
| ) -> None: | ||
| """Test that relative paths are properly resolved before converting to URI.""" | ||
| # Create a package with a relative path (simulating the bug scenario) |
There was a problem hiding this comment.
suggestion (testing): Strengthen the assertion to verify the resolved path matches the expected directory, not just that the URI is syntactically valid.
The current test only asserts that the URI is a syntactically valid file:// URL, which only proves that no ValueError is raised. To verify correctness, parse the URI path back into a Path and assert it equals Path(project_dir).resolve(), so the test confirms the resolved directory is correct, not just well-formed.
Suggested implementation:
import shutil
import tempfile
from urllib.parse import urlparsedef test_executor_should_handle_relative_paths_in_directory_url_reference(
tmp_path: Path,
tmp_venv: VirtualEnv,
pool: RepositoryPool,
config: Config,
io: BufferedIO,
fixture_dir: FixtureDirGetter,
) -> None:
"""Test that relative paths are properly resolved before converting to URI."""
# Create a package with a relative path (simulating the bug scenario)
project_dir = fixture_dir("simple_project")
# NOTE: this test should obtain a directory URI (e.g. from the executor) into `directory_uri`.
# Once `directory_uri` is available, strengthen the assertion to verify the resolved path.
# Example of a stronger assertion (assuming `directory_uri` is the file:// URI under test):
#
# parsed = urlparse(directory_uri)
# resolved_path = Path(parsed.path).resolve()
# assert resolved_path == Path(project_dir).resolve()To fully implement the stronger assertion, you should:
- Replace the placeholder comment and example in the test body with real code that:
- Calls the code under test to obtain the
file://directory URI (whatever variable currently holds it). - Parses the URI using
urlparse. - Converts the parsed path to a
Pathand resolves it. - Asserts that this resolved
PathequalsPath(project_dir).resolve().
- Calls the code under test to obtain the
- Concretely, if your existing test currently has something like:
directory_uri = <call_under_test>(...)assert directory_uri.startswith("file://")
you should change it to:- Keep obtaining
directory_urithe same way. - Replace the simple
startswithassertion with:parsed = urlparse(directory_uri) resolved_path = Path(parsed.path).resolve() assert resolved_path == Path(project_dir).resolve()
Pathfrompathlibif not already imported in this file.)
Adjust the variable namedirectory_uriin the snippet above to match the actual variable used in your test.
| project_dir = fixture_dir("simple_project") | ||
| relative_path = os.path.relpath(project_dir, tmp_path) |
There was a problem hiding this comment.
suggestion (bug_risk): The way the relative path is constructed may not reflect how the executor actually resolves paths (cwd vs tmp_path).
relative_path is computed with os.path.relpath(project_dir, tmp_path), but Path(package.source_url).resolve() is relative to the process cwd, not tmp_path. Unless the test explicitly sets cwd to tmp_path, this doesn’t match actual resolution behavior and may point somewhere unexpected.
Consider either:
- Changing the cwd in the test (e.g.
monkeypatch.chdir(tmp_path)) and then computing the relative path from that cwd, or - Using a simple relative path (e.g.
os.path.relpath(project_dir)with nostart) and asserting that the resolved URI maps back toproject_dir.
Fixes #10830
Problem
When
package.source_urlis a relative path, callingPath().as_uri()directly raises:ValueError: relative path can't be expressed as a file URIThis affects users with nested git dependencies using subdirectory arguments, where the path can be relative to the working directory.
Root Cause
In the executor's URL reference creation methods, the code was attempting to convert paths directly to URIs without first ensuring they were absolute paths. Python's
Path.as_uri()requires absolute paths.Solution
Add
.resolve()before.as_uri()in two methods:_create_file_url_reference()- line 925_create_directory_url_reference()- line 937The
.resolve()method converts relative paths to absolute paths by resolving them against the current working directory, then.as_uri()can successfully convert them tofile://URIs.Testing
Added two new test cases:
test_executor_should_handle_relative_paths_in_directory_url_reference- Tests directory sources with relative pathstest_executor_should_handle_relative_paths_in_file_url_reference- Tests file sources with relative pathsBoth tests verify that:
file://All existing tests continue to pass.
Verification