Skip to content

Support Secret Manager in JdbcIO for Java, Python and YAML - #39834

Open
shunping wants to merge 3 commits into
apache:masterfrom
shunping:secret-manager-3
Open

Support Secret Manager in JdbcIO for Java, Python and YAML#39834
shunping wants to merge 3 commits into
apache:masterfrom
shunping:secret-manager-3

Conversation

@shunping

@shunping shunping commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Description

Adds support for resolving database passwords securely using Secret Manager across Java, Python (cross-language), and Beam YAML for JdbcIO.

Follow-up to #39636 and #39806.

Changes

  • Leveraged Secret Manager classes in Java JDBCIO
  • Surfaced secret manager parameter in Python (xlang) and Yaml.
  • Added unit and integration tests.

Comment thread sdks/python/apache_beam/yaml/integration_tests.py Fixed
Comment thread sdks/python/apache_beam/yaml/integration_tests.py Dismissed
Comment thread sdks/python/apache_beam/yaml/integration_tests.py Dismissed
Comment thread sdks/python/apache_beam/yaml/integration_tests.py Dismissed
@shunping
shunping marked this pull request as ready for review August 21, 2026 01:40
@shunping

Copy link
Copy Markdown
Collaborator Author

r: @damccorm for java and @derrickaw for xlang and yaml.

@shunping
shunping requested review from damccorm and derrickaw August 21, 2026 01:41
@github-actions

Copy link
Copy Markdown
Contributor

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

@damccorm damccorm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I think there are some improvements we can make to the documentation, but the code itself looks great

public abstract String getReadQuery();

@SchemaFieldDescription(
"Secret Manager to use for fetching secret values. For example, GoogleCloudSecretManager.")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need a few things to improve the docs here:

  1. An example of the full secret manager string and how it is used (this would go in password, not here I guess).
  2. A set of allowed secret managers for the default expansion service.

It might make sense to abstract this out to a small doc since it will probably show up in several places.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, the same feedback applies to the yaml doc below.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants