Skip to content

out_opensearch: lock the client cache against flush threads - #195

Open
kenhys wants to merge 1 commit into
mainfrom
fix-flush
Open

kenhys wants to merge 1 commit into
mainfrom
fix-flush

Conversation

@kenhys

@kenhys kenhys commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Before: client() cleared the shared @_os slot and then rebuilt it with ||=, without a lock. With flush_thread_count of 2 or more and a host placeholder, a thread could pass the "is this the same host" check and then pick up the client another thread had just built for a different host, so its chunk went to that cluster instead.

After: the whole check-and-build runs under a mutex, so a thread always gets a client for the host it asked for. The mutex is created in initialize because configure builds a client before it returns.

Before: client() cleared the shared @_os slot and then rebuilt it with
`||=`, without a lock. With flush_thread_count of 2 or more and a host
placeholder, a thread could pass the "is this the same host" check and
then pick up the client another thread had just built for a different
host, so its chunk went to that cluster instead.

After: the whole check-and-build runs under a mutex, so a thread always
gets a client for the host it asked for. The mutex is created in
initialize because configure builds a client before it returns.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Kentaro Hayashi <hayashi@clear-code.com>
@cosmo0920

Copy link
Copy Markdown
Contributor

@kenhys This type of fix also ought to apply for https://github.com/uken/fluent-plugin-elasticsearch?
Once the fix is applied there, we need to release a new version of ES plugin as well.

@kenhys

kenhys commented Sep 17, 2026

Copy link
Copy Markdown
Contributor Author

@kenhys This type of fix also ought to apply for https://github.com/uken/fluent-plugin-elasticsearch? Once the fix is applied there, we need to release a new version of ES plugin as well.

I've overlooked it, maybe similar fix is needed.

@kenhys
kenhys marked this pull request as ready for review September 17, 2026 01:35
@kenhys
kenhys requested a review from Watson1978 September 17, 2026 01:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants