Skip to content

feat(vertex_ai): support GenerateContentConfig labels for Dify app_id propagation#3168

Draft
mas-sakai wants to merge 7 commits into
langgenius:mainfrom
mas-sakai:feat/vertex-ai-app-id-labels
Draft

feat(vertex_ai): support GenerateContentConfig labels for Dify app_id propagation#3168
mas-sakai wants to merge 7 commits into
langgenius:mainfrom
mas-sakai:feat/vertex-ai-app-id-labels

Conversation

@mas-sakai
Copy link
Copy Markdown

Note

This is a Draft PR. It depends on
langgenius/dify-plugin-sdks#313
being merged and a new SDK released. pyproject.toml / uv.lock currently
pin dify_plugin to the #313 branch so the Gemini route can import
get_current_session(). The pin will be reverted to a released version
constraint (dify_plugin>=0.3.0,<0.6.0) before this PR is marked Ready
for review. The CI "Validate Dependencies" / "Check Plugin Install"
steps may fail until then.

Summary

Enable Cloud Billing visibility per Dify app for the Vertex AI plugin (Gemini
route). When the new enable_request_metadata credential is set to enabled,
the plugin attaches dify_app_id and dify_source to
GenerateContentConfig.labels, so operators can break Vertex AI cost down by
labels.dify_app_id in GCP Cloud Billing without any ETL.

Related issues / PRs:

Scope (intentionally narrow)

  • Gemini route only (models/llm/llm.py::_generate). The Anthropic-on-Vertex
    route (_generate_anthropic) and other providers are out of scope.
  • Label keys: dify_app_id and dify_source only. dify_tenant_id /
    dify_user_id are deferred to a follow-up to keep the surface minimal.
  • Default-off: enable_request_metadata defaults to disabled, so existing
    users who do not touch their credentials are completely unaffected. With
    disabled, the control flow is identical to the previous version.

Implementation notes

  • New helper models/llm/_labels.py with normalize_label_value and
    build_dify_labels. Output conforms to the GCP labels spec (lowercase,
    [a-z0-9_-] only, 63-char cap). UUID app_ids pass through unchanged.
  • get_current_session() is imported locally inside _generate and wrapped in
    try/except ImportError, so the plugin keeps working against older
    dify_plugin releases (no hard runtime dependency on fix getnumtoken retrun list[int] #313 once it ships).
  • 9 unit tests added in tests/test_labels.py (passthrough, character
    replacement, truncation, casing, empty input, None / empty short-circuits
    in build_dify_labels).

Change Type

  • Documentation / non-plugin change
  • Non-LLM plugin (tools, extensions, datasource, etc.)
  • LLM plugin

Screenshots / Videos

🚧 Cloud Billing breakdown screenshots (by labels.dify_app_id) will be
added once E2E verification completes
(currently testing on GCP project
dify-vertex-labels-poc). All 9 unit tests pass (uv run pytest tests/).

Before After

LLM Plugin Checklist

Areas affected by this change (check all that apply)
  • Message flow (system messages, user ↔ assistant turn-taking)
  • Tool interaction flow (multi-round usage, Agent App and Agent Node)
  • Multimodal input (images, PDFs, audio, video, etc.)
  • Multimodal output (images, audio, video, etc.)
  • Structured output (JSON, XML, etc.)
  • Token consumption metrics
  • Other LLM functionality (reasoning, grounding, prompt caching, etc.) — request metadata / Cloud Billing labels
  • New models / model parameter fixes

Version

  • Bumped top-level version in manifest.yaml (not the one under meta) — 0.0.530.0.54
  • dify_plugin>=0.3.0,<0.6.0 is declared in pyproject.toml and locked in uv.lock (or kept in requirements.txt for legacy plugins without uv.lock) — SDK docs

⚠️ The dify_plugin constraint is currently a temporary git-branch pin
targeting #313.
It will be reverted to dify_plugin>=0.3.0,<0.6.0 before this PR leaves
Draft.

Testing

  • Local deployment — Dify version: 1.14.x (E2E Cloud Billing breakdown verification in progress on GCP project dify-vertex-labels-poc)
  • SaaS (cloud.dify.ai)

Unit tests: uv run pytest tests/ — 9 passed (label normalization helper).

mas-sakai and others added 5 commits May 19, 2026 14:28
Add `_labels.py` with `normalize_label_value` and `build_dify_labels`
helpers that produce GCP-compliant Vertex AI label dicts. UUID app_ids
pass through unchanged; other strings are lowercased, restricted to
`[a-z0-9_-]`, and truncated to 63 characters. Unit tests cover passthrough,
character replacement, truncation, casing, empty input, and the `None`/
empty short-circuits in `build_dify_labels`.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When `enable_request_metadata` is `enabled` in credentials, attach
`dify_app_id` and `dify_source` to `GenerateContentConfig.labels` for the
Gemini route only. The `get_current_session` import is local to the call
site and wrapped in `try/except ImportError` so the plugin keeps working
against older `dify_plugin` releases. With the default `disabled` setting,
control flow is identical to the previous version.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add an opt-in `enable_request_metadata` credential (select, defaults to
`disabled`) so operators can choose to attach Dify metadata to Vertex AI
requests for Cloud Billing breakdown. Existing users who do not touch
their credentials are unaffected.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Point `dify_plugin` at the `feat/pass-session-to-model-plugins` branch of
`ryuta-kobayashi-ug/dify-plugin-sdks` (PR langgenius#313) so the Gemini route can
import `get_current_session()`. Refresh `uv.lock` accordingly. This pin
is temporary for the review period; revert to a release-version
constraint once langgenius#313 is merged and a new SDK is published.

Also add `PR_BODY.md` as a draft of the upstream PR body so it is
available alongside the branch.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@mas-sakai mas-sakai temporarily deployed to models/vertex_ai May 20, 2026 05:45 — with GitHub Actions Inactive
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the capability to track Cloud Billing costs on a per-application basis by attaching Dify metadata as labels to Vertex AI requests. The implementation includes a utility for normalizing label values according to GCP constraints, logic in the Gemini model to extract session metadata, and a new configuration setting to enable this feature. Feedback suggested optimizing the label normalization helper by truncating long input strings prior to processing to enhance performance and reduce memory usage.

Comment on lines +34 to +38
if not s:
return ""
lowered = s.lower()
sanitized = _INVALID_CHAR_RE.sub("_", lowered)
return sanitized[:_MAX_LABEL_LENGTH]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

入力文字列 s が非常に長い場合、lower() や正規表現による置換処理を行う前に文字列を切り詰めることで、メモリ使用量と処理時間を節約できます。特に app_id が外部から渡される可能性がある場合、事前に制限文字数(63文字)でスライスしておくことが効率的です。

Suggested change
if not s:
return ""
lowered = s.lower()
sanitized = _INVALID_CHAR_RE.sub("_", lowered)
return sanitized[:_MAX_LABEL_LENGTH]
if not s:
return ""
# 処理負荷を軽減するため、正規化の前に切り詰めを行います
s = s[:_MAX_LABEL_LENGTH]
lowered = s.lower()
sanitized = _INVALID_CHAR_RE.sub("_", lowered)
return sanitized[:_MAX_LABEL_LENGTH]

mas-sakai and others added 2 commits May 22, 2026 09:56
Remove `PR_BODY.md` (and the local `PR_BODY_EN.md` working copy) from
the repository. The content lives in the PR description on GitHub; the
checked-in copies are working notes that would otherwise land in main
with the merge.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The descriptive text for the `enable_request_metadata` credential was
placed in `placeholder`, but `type: select` is rendered as a dropdown
and does not surface placeholder text. Move the description to `help`
so it actually shows in the credentials UI, matching the convention
used by the openai provider's select credentials.

Refs: langgenius/dify#35772, langgenius/dify-plugin-sdks#311

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@mas-sakai mas-sakai deployed to models/vertex_ai May 22, 2026 00:57 — with GitHub Actions Active
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant