Skip to content

[common] Resolve the atomic rename under the Kerberos FileSystem wrapper - #9652

Open
LuciferYang wants to merge 1 commit into
apache:masterfrom
LuciferYang:fix/hadoop-kerberos-atomic-rename
Open

[common] Resolve the atomic rename under the Kerberos FileSystem wrapper#9652
LuciferYang wants to merge 1 commit into
apache:masterfrom
LuciferYang:fix/hadoop-kerberos-atomic-rename

Conversation

@LuciferYang

Copy link
Copy Markdown
Contributor

Purpose

close #9651

HadoopFileIO.tryAtomicOverwriteViaRename looks up FileSystem's three-argument atomic rename reflectively:

method = ReflectionUtils.getMethod(fs.getClass(), "rename", 3);

ReflectionUtils.getMethod goes through clz.getMethods(), so it only sees public methods. With Kerberos configured, fs is a HadoopSecuredFileSystem, which overrides the two-argument rename and leaves the three-argument one protected on the base class. The lookup fails, the cached method stays null, and the method returns false for good, so every caller silently falls back to an in-place newOutputStream(path, true): the snapshot hint files, consumer resets, TagManager.createOrReplaceTag, _SUCCESS and the service files all stop being written atomically on a secured cluster, with nothing logged.

The lookup now resolves against the file system underneath the wrapper, since the wrapper genuinely cannot override that method: it would have to call a protected member on a different FileSystem instance, which does not compile.

Reaching past the wrapper means taking its one responsibility with you. Every delegating method in HadoopSecuredFileSystem runs inside ugi.doAs, and invoking the rename directly on the unwrapped file system would run it as whatever the calling thread happens to be, while the temporary file being renamed was created as the login user. The invocation therefore goes through a new callAsLoginUser on the wrapper, which is the same runSecuredWithIOException the other methods use.

Tests

HadoopSecuredFileSystemTest.testAtomicRenameRunsOnTheDelegateAsTheLoginUser wraps a RawLocalFileSystem subclass that exposes the three-argument rename as public and counts calls, secures it through trySecureFileSystem, installs it with HadoopFileIO.setFileSystem, and then asserts tryAtomicOverwriteViaRename returns true, the delegate's atomic rename ran exactly once, it ran as the login user, and the content landed.

Against the unfixed code that test fails on the first assertion: the method returns false, which is the silent fallback.

testUnwrapAndCallAsLoginUser covers the two new methods directly, including that callAsLoginUser propagates an IOException rather than wrapping it.

mvn -pl paimon-common -Dtest=HadoopSecuredFileSystemTest test on JDK 8: 6 tests, 0 failures. spotless:check and checkstyle:check on paimon-common are clean.

What this does not cover is a real Kerberos cluster: the delegate here is a local file system with a public three-argument rename, so the test pins the lookup and the identity, not HDFS's rename semantics.

tryAtomicOverwriteViaRename looks up FileSystem's 3-arg rename with
ReflectionUtils.getMethod, which only sees public methods.
HadoopSecuredFileSystem overrides just the 2-arg rename, so with Kerberos
configured the lookup failed and the method returned false for good:
hint files, consumer resets, tags and _SUCCESS were all written by
in-place overwrite instead, silently.

Resolve and invoke on the file system under the wrapper, with the
invocation inside the wrapper's doAs so the rename still runs as the
login user like the temp file it renames.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Kerberos FileSystem wrapper silently disables atomic rename for metadata writes

1 participant