Skip to content

Add AI Client connector (XWPENG-20) - #1967

Open
shadyvb wants to merge 2 commits into
developfrom
feature/add-ai-client-connector
Open

Add AI Client connector (XWPENG-20)#1967
shadyvb wants to merge 2 commits into
developfrom
feature/add-ai-client-connector

Conversation

@shadyvb

@shadyvb shadyvb commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Fixes XWPENG-20.

Adds a bundled AI Client connector to Stream that listens to WordPress AI Client before/after_generate_result hooks and writes audit records for each generation. Records include acting user, timestamp, provider, model, input/output/thinking token counts, duration, finish reason, and extended metadata. Full prompt and response text capture is supported via a single opt-in setting (log_prompt_and_response_text), disabled by default with a PII warning. Developer filters wp_stream_ai_client_log_prompt and wp_stream_ai_client_log_response allow redaction or omission of stored text. The Stream list table shows only the first line of multiline summaries so token/metadata rows stay scannable when prompt/response logging is enabled.

Checklist

  • Project documentation has been updated to reflect the changes in this pull request, if applicable.
  • I have tested the changes in the local development environment (see contributing.md).
  • I have added phpunit tests.

Release Changelog

  • New: AI Client connector logs WordPress AI Client generations (tokens, model, provider, duration) with optional full prompt/response capture behind a single opt-in toggle.

Release Checklist

  • This pull request is to the develop branch.
  • Release version follows semantic versioning. Does it include breaking changes?
  • Update changelog in readme.txt.
  • Bump version in stream.php.
  • Bump Stable tag in readme.txt.
  • Bump version in classes/class-plugin.php.
  • Draft a release on GitHub.

Test plan

  • On WP 7.0+ with AI Client configured, trigger a generation and confirm a Stream record appears under AI Client β†’ Prompts β†’ Generated with user, model, provider, and token counts.
  • With Log Prompt and Response text off (default), confirm prompt/response body are not stored in the record summary or meta.
  • Enable Log Prompt and Response text, confirm the PII warning is visible, prompt/response appear in the record, and the list table shows only the first summary line.
  • Hook wp_stream_ai_client_log_prompt / wp_stream_ai_client_log_response to redact text; confirm stored values reflect the filter output.
  • On a site without WP_AI_Client_Event_Dispatcher, confirm Stream loads with no fatals and no AI records are written.
  • Run PHPUnit: vendor/bin/phpunit --filter Connector_AI_Client and vendor/bin/phpunit --filter Test_List_Table.

@shadyvb
shadyvb force-pushed the feature/add-ai-client-connector branch from 51dfd34 to 496a047 Compare August 21, 2026 11:54
Register a bundled Stream connector that listens to wp_ai_client_before/
after_generate_result and writes activity records with provider, model,
token counts (input/output/thought), duration, and extended metadata.

- Single opt-in toggle (log_prompt_and_response_text) for full prompt/
  response capture, disabled by default with PII warning
- wp_stream_ai_client_log_prompt / log_response filters for redaction
- List table first-line summary preview for multiline entries
- PHPUnit coverage for logging, toggles, filters, and graceful no-op
- Docs, changelog, connectors.md regen; gitignore .ai/ local tooling
@shadyvb
shadyvb force-pushed the feature/add-ai-client-connector branch from 496a047 to f4132b0 Compare August 21, 2026 11:54
@shadyvb
shadyvb marked this pull request as ready for review August 21, 2026 11:55

@bartoszgadomski bartoszgadomski left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work @shadyvb! I left inline comments, please take a look.

Comment thread readme.txt Outdated

* Show only the first line of multiline summaries in the Stream list table.

[View the full release notes on GitHub.](https://github.com/xwp/stream/blob/master/changelog.md#unreleased)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this entry shouldn't be included in readme.txt but only in β€Žchangelog.mdβ€Ž (which you already added).


$log_args = $this->append_result_context_to_log_args( $log_args, $event, $model, $result );
/* translators: 1: AI operation (e.g. "chat"), 2: provider slug, 3: model ID */
$message = esc_html__( '%1$s via %2$s/%3$s (tokens: %4$d/%6$d/%5$d) in %7$dms', 'stream' );

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Token order in the summary looks swapped.

Log::log() renders the summary with vsprintf( $message, $args ), so the placeholders map to $log_args by position: %4$d = input_tokens, %5$d = output_tokens, %6$d = thought_tokens. This template is %4$d/%6$d/%5$d, so the feed prints input / thought / output, while the args array, the meta keys, and the PR description all read input/output/thought.

If the order is deliberate, please document it; otherwise use %4$d/%5$d/%6$d. Nothing currently asserts the rendered summary, so a test on the formatted string would lock this down.

Two smaller notes on the same string:

  • The translators comment only describes placeholders 1-3, but the template uses up to %7$d (and %8$s/%9$s below). Translators can't safely reorder what they can't identify.
  • Other connectors use __() / _x() for log templates (e.g. Connector_Menus, Connector_Posts, Connector_Jetpack); esc_html__() here means any entity-worthy character in a translation gets stored HTML-encoded in the summary column. The value is already run through wp_strip_all_tags() on insert and wp_kses() on output, so the escaping is redundant.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The order is deliberate, yes. Fixed the escaping and translator comment as well.

* @param array<string, mixed> $record Record about to be inserted.
* @return array<string, mixed>
*/
public function filter_wp_stream_record_array( array $record ) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The array type hint can turn a filtered-out record into a fatal.

wp_stream_record_array is not guaranteed to carry an array: Connector_Woocommerce::callback_wp_stream_record_array() returns false to drop a record (connectors/class-connector-woocommerce.php:697), and that callback runs on the same hook at the same priority 10.

Today this is safe only because load_connectors() instantiates ai-client before woocommerce, so this filter runs first. That ordering is not contractual β€” the wp_stream_connectors filter lets third parties reorder or inject connectors, and any other callback returning false/null before this one produces a TypeError on record insert rather than a skipped record.

Suggest dropping the hint and guarding instead:

public function filter_wp_stream_record_array( $record ) {
	if ( ! is_array( $record ) ) {
		return $record;
	}
	// ...
}

Same reasoning applies to add_settings_fields( array $fields ).


// Add the prompt and response text to the message if enabled.
if ( $this->is_prompt_and_response_logging_enabled() ) {
$message .= sprintf( "\n\n[%s]\n%s\n\n[%s]\n%s", esc_html__( 'Prompt', 'stream' ), '%8$s', esc_html__( 'Response', 'stream' ), '%9$s' );

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth calling out in the docs: prompt/response text reaches alert destinations.

With the opt-in enabled, prompt and response bodies become part of summary, and Stream alerts forward summary verbatim to third parties: Slack (alerts/class-alert-type-slack.php:146,150), IFTTT (alerts/class-alert-type-ifttt.php:282), and the admin menu alert (alerts/class-alert-type-menu-alert.php:51).

So an admin who enables this setting also starts shipping AI prompt text to any configured webhook. The readme.txt PII warning covers storage but not egress. Either mention it in the warning copy, or note that wp_stream_ai_client_log_prompt / wp_stream_ai_client_log_response are the redaction point for that path too.

*/
private function normalize_capability( $capability ) {
if ( null === $capability ) {
return 'unknown';

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docblock contradicts the code, and there are two "unknown" slugs.

The docblock above says null or unresolvable capabilities "fall back to 'chat'", but this returns 'unknown' for null and 'unknown_operation' for an unresolvable value.

Since operation is user-facing in the summary and filterable in the list table, two distinct slugs for what is effectively the same "couldn't determine it" case will fragment filtering. Suggest a single value and a docblock that matches.

}

if ( $value instanceof \WordPress\AiClient\Common\AbstractEnum ) {
return isset( $value->value ) ? (string) $value->value : (string) $value->name;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This branch always resolves to ->name, by the method's own reasoning.

The docblock states that AbstractEnum has __get() but no __isset(), so isset( $value->value ) is always false. That makes this line equivalent to return (string) $value->name;, which yields the enum name (e.g. TEXT_GENERATION) rather than its value (e.g. text_generation), and skips the __toString() path below that the docblock describes as returning the value.

It happens to come out right for capabilities because normalize_capability() lowercases, and for roles because prompt_section_key_for_scalar_role() lowercases β€” but any enum whose value isn't just the lowercased name (hyphens, abbreviations) would produce a wrong slug. Using __toString() for AbstractEnum, or property_exists() / a try around ->value, would express the intent directly.

}
if ( method_exists( $tu, 'getThoughtTokens' ) ) {
$thought_tokens = $tu->getThoughtTokens();
$log_args['thought_tokens'] = null === $thought_tokens ? null : (int) $thought_tokens;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: setting thought_tokens to null here makes the two outputs disagree. Log::log() strips null values before writing meta, so the meta key disappears, while vsprintf() still receives null for %6$d and prints 0. The test doubles return null from getThoughtTokens(), so this is the default path, not an edge case.

Also, thought_tokens is already computed at line 314 with (int); this recomputes it. Keeping the (int) cast (0 when unsupported) or omitting the key entirely would make summary and meta consistent.

}

// Already has section headings β€” prepend system block directly.
if ( false !== strpos( $message_prompt, '--- ' ) ) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two cosmetic issues in the assembled prompt text:

  • Heading styles don't match. extract_prompt_text() emits --- User ---, while this method emits [System] and [User]. A single reader sees both conventions in one record.
  • This heuristic keys off the literal '--- ', so a prompt whose body contains a Markdown horizontal rule, a diff hunk, or any --- sequence is treated as already-sectioned and loses its [User] heading.

Passing the section list (or a boolean) down from extract_prompt_text() instead of re-detecting via strpos() would remove the guesswork. Display-only, no data loss.

// Store the pending data in the pending array, so we can match the after-hook.
$pending = $this->get_pending_storage();
$this->evict_oldest_pending_if_full( $pending );
$pending[ $model ] = array(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low priority / possibly by design: keying on the model instance pairs before/after correctly for sequential calls and for concurrent calls on distinct instances (both covered by tests), but two overlapping generations on the same instance overwrite this entry.

The realistic trigger is nesting β€” a generation started from inside an after_generate_result listener, or a helper that reuses one cached model across a nested call. The result is one lost record plus a wrong duration and mismatched prompt on the other.

If WP AI Client can't nest on one instance, a note here saying so would close the question; otherwise a small stack per model would cover it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WP AI Client runs a strict before β†’ execute β†’ after sandwich. Same-instance reuse is sequential, not nested.

Comment thread classes/class-list-table.php Outdated
* @param string $summary Full summary stored in the database.
* @return string Summary preview text.
*/
public static function format_summary_preview_html( $summary ) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changes the summary column for every connector, not just AI Client.

column_default() routes all records through this, so any multiline summary from any connector (including third-party ones) is now truncated in the activity feed. The changelog frames it as a general enhancement, which is fair, but it's a behavior change beyond the scope suggested by the PR title β€” is that intended?

Two smaller points:

  • There is no affordance signalling that content was hidden. A reader sees a complete-looking line with no hint to open the record. A trailing … would help.
  • The name says _html while the return value is plain text (the caller still passes it through wp_kses()). For a new public static method, format_summary_preview() would describe it more accurately.

… and PII warnings match Stream's existing patterns.

Truncated summaries now show an ellipsis, log templates stay translatable without early escaping, and operators are warned that stored prompt text may be forwarded to alerts or webhooks.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants