Skip to content

[C++] S3FileSystem::MakeUri drops bucket from path #50204

Description

@raulcd

Describe the bug, including details regarding any error messages, version, and platform.

The following new ad-hoc test:

TEST_F(S3OptionsTest, MakeUri) {
  S3Options options;
  options.ConfigureAccessKey("minio", "miniopass");
  options.region = "us-east-1";
  ASSERT_OK_AND_ASSIGN(auto fs, S3FileSystem::Make(options));
  ASSERT_OK_AND_ASSIGN(auto uri, fs->MakeUri("/bucket/somedir/subdir/subfile"));
  EXPECT_EQ(uri,
            "s3://minio:miniopass@bucket/somedir/subdir/subfile"
            "?region=us-east-1&scheme=https&endpoint_override="
            "&allow_bucket_creation=0&allow_bucket_deletion=0");
}

fails on Windows MinGW because the bucket is dropped from the URL. I tested it in isolation on this PR:

And the failure

 [ RUN      ] S3OptionsTest.MakeUri
D:/a/arrow/arrow/cpp/src/arrow/filesystem/s3fs_test.cc:1747: Failure
Expected equality of these values:
  uri
    Which is: "***somedir/subdir/subfile?region=us-east-1&scheme=https&endpoint_override=&allow_bucket_creation=0&allow_bucket_deletion=0"
  "***bucket/somedir/subdir/subfile" "?region=us-east-1&scheme=https&endpoint_override=" "&allow_bucket_creation=0&allow_bucket_deletion=0"
    Which is: "***bucket/somedir/subdir/subfile?region=us-east-1&scheme=https&endpoint_override=&allow_bucket_creation=0&allow_bucket_deletion=0"

[  FAILED  ] S3OptionsTest.MakeUri (0 ms)

This is because the manual path parsing doesn't take Windows paths into account:

Result<std::string> S3FileSystem::MakeUri(std::string path) const {
if (path.length() <= 1 || path[0] != '/') {
return Status::Invalid("MakeUri requires an absolute, non-root path, got ", path);
}
ARROW_ASSIGN_OR_RAISE(auto uri, util::UriFromAbsolutePath(path));
if (!options().GetAccessKey().empty()) {
uri = "s3://" + options().GetAccessKey() + ":" + options().GetSecretKey() + "@" +
uri.substr("file:///"s.size());
} else {
uri = "s3" + uri.substr("file"s.size());
}

I found out when testing our arrow-s3fs-module-test on all platforms not only on Linux where we have a similar test. This is a pre-existing bug.
Seems to be related to util::UriFromAbsolutePath but haven't been able to pin down the problem due to difficulty testing MinGW.

Component(s)

C++

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions