Add AI Client connector (XWPENG-20) - #1967
Conversation
51dfd34 to
496a047
Compare
Register a bundled Stream connector that listens to wp_ai_client_before/ after_generate_result and writes activity records with provider, model, token counts (input/output/thought), duration, and extended metadata. - Single opt-in toggle (log_prompt_and_response_text) for full prompt/ response capture, disabled by default with PII warning - wp_stream_ai_client_log_prompt / log_response filters for redaction - List table first-line summary preview for multiline entries - PHPUnit coverage for logging, toggles, filters, and graceful no-op - Docs, changelog, connectors.md regen; gitignore .ai/ local tooling
496a047 to
f4132b0
Compare
bartoszgadomski
left a comment
There was a problem hiding this comment.
Nice work @shadyvb! I left inline comments, please take a look.
|
|
||
| * Show only the first line of multiline summaries in the Stream list table. | ||
|
|
||
| [View the full release notes on GitHub.](https://github.com/xwp/stream/blob/master/changelog.md#unreleased) |
There was a problem hiding this comment.
I think this entry shouldn't be included in readme.txt but only in βchangelog.mdβ (which you already added).
|
|
||
| $log_args = $this->append_result_context_to_log_args( $log_args, $event, $model, $result ); | ||
| /* translators: 1: AI operation (e.g. "chat"), 2: provider slug, 3: model ID */ | ||
| $message = esc_html__( '%1$s via %2$s/%3$s (tokens: %4$d/%6$d/%5$d) in %7$dms', 'stream' ); |
There was a problem hiding this comment.
Token order in the summary looks swapped.
Log::log() renders the summary with vsprintf( $message, $args ), so the placeholders map to $log_args by position: %4$d = input_tokens, %5$d = output_tokens, %6$d = thought_tokens. This template is %4$d/%6$d/%5$d, so the feed prints input / thought / output, while the args array, the meta keys, and the PR description all read input/output/thought.
If the order is deliberate, please document it; otherwise use %4$d/%5$d/%6$d. Nothing currently asserts the rendered summary, so a test on the formatted string would lock this down.
Two smaller notes on the same string:
- The
translatorscomment only describes placeholders 1-3, but the template uses up to%7$d(and%8$s/%9$sbelow). Translators can't safely reorder what they can't identify. - Other connectors use
__()/_x()for log templates (e.g.Connector_Menus,Connector_Posts,Connector_Jetpack);esc_html__()here means any entity-worthy character in a translation gets stored HTML-encoded in thesummarycolumn. The value is already run throughwp_strip_all_tags()on insert andwp_kses()on output, so the escaping is redundant.
There was a problem hiding this comment.
The order is deliberate, yes. Fixed the escaping and translator comment as well.
| * @param array<string, mixed> $record Record about to be inserted. | ||
| * @return array<string, mixed> | ||
| */ | ||
| public function filter_wp_stream_record_array( array $record ) { |
There was a problem hiding this comment.
The array type hint can turn a filtered-out record into a fatal.
wp_stream_record_array is not guaranteed to carry an array: Connector_Woocommerce::callback_wp_stream_record_array() returns false to drop a record (connectors/class-connector-woocommerce.php:697), and that callback runs on the same hook at the same priority 10.
Today this is safe only because load_connectors() instantiates ai-client before woocommerce, so this filter runs first. That ordering is not contractual β the wp_stream_connectors filter lets third parties reorder or inject connectors, and any other callback returning false/null before this one produces a TypeError on record insert rather than a skipped record.
Suggest dropping the hint and guarding instead:
public function filter_wp_stream_record_array( $record ) {
if ( ! is_array( $record ) ) {
return $record;
}
// ...
}Same reasoning applies to add_settings_fields( array $fields ).
|
|
||
| // Add the prompt and response text to the message if enabled. | ||
| if ( $this->is_prompt_and_response_logging_enabled() ) { | ||
| $message .= sprintf( "\n\n[%s]\n%s\n\n[%s]\n%s", esc_html__( 'Prompt', 'stream' ), '%8$s', esc_html__( 'Response', 'stream' ), '%9$s' ); |
There was a problem hiding this comment.
Worth calling out in the docs: prompt/response text reaches alert destinations.
With the opt-in enabled, prompt and response bodies become part of summary, and Stream alerts forward summary verbatim to third parties: Slack (alerts/class-alert-type-slack.php:146,150), IFTTT (alerts/class-alert-type-ifttt.php:282), and the admin menu alert (alerts/class-alert-type-menu-alert.php:51).
So an admin who enables this setting also starts shipping AI prompt text to any configured webhook. The readme.txt PII warning covers storage but not egress. Either mention it in the warning copy, or note that wp_stream_ai_client_log_prompt / wp_stream_ai_client_log_response are the redaction point for that path too.
| */ | ||
| private function normalize_capability( $capability ) { | ||
| if ( null === $capability ) { | ||
| return 'unknown'; |
There was a problem hiding this comment.
Docblock contradicts the code, and there are two "unknown" slugs.
The docblock above says null or unresolvable capabilities "fall back to 'chat'", but this returns 'unknown' for null and 'unknown_operation' for an unresolvable value.
Since operation is user-facing in the summary and filterable in the list table, two distinct slugs for what is effectively the same "couldn't determine it" case will fragment filtering. Suggest a single value and a docblock that matches.
| } | ||
|
|
||
| if ( $value instanceof \WordPress\AiClient\Common\AbstractEnum ) { | ||
| return isset( $value->value ) ? (string) $value->value : (string) $value->name; |
There was a problem hiding this comment.
This branch always resolves to ->name, by the method's own reasoning.
The docblock states that AbstractEnum has __get() but no __isset(), so isset( $value->value ) is always false. That makes this line equivalent to return (string) $value->name;, which yields the enum name (e.g. TEXT_GENERATION) rather than its value (e.g. text_generation), and skips the __toString() path below that the docblock describes as returning the value.
It happens to come out right for capabilities because normalize_capability() lowercases, and for roles because prompt_section_key_for_scalar_role() lowercases β but any enum whose value isn't just the lowercased name (hyphens, abbreviations) would produce a wrong slug. Using __toString() for AbstractEnum, or property_exists() / a try around ->value, would express the intent directly.
| } | ||
| if ( method_exists( $tu, 'getThoughtTokens' ) ) { | ||
| $thought_tokens = $tu->getThoughtTokens(); | ||
| $log_args['thought_tokens'] = null === $thought_tokens ? null : (int) $thought_tokens; |
There was a problem hiding this comment.
Minor: setting thought_tokens to null here makes the two outputs disagree. Log::log() strips null values before writing meta, so the meta key disappears, while vsprintf() still receives null for %6$d and prints 0. The test doubles return null from getThoughtTokens(), so this is the default path, not an edge case.
Also, thought_tokens is already computed at line 314 with (int); this recomputes it. Keeping the (int) cast (0 when unsupported) or omitting the key entirely would make summary and meta consistent.
| } | ||
|
|
||
| // Already has section headings β prepend system block directly. | ||
| if ( false !== strpos( $message_prompt, '--- ' ) ) { |
There was a problem hiding this comment.
Two cosmetic issues in the assembled prompt text:
- Heading styles don't match.
extract_prompt_text()emits--- User ---, while this method emits[System]and[User]. A single reader sees both conventions in one record. - This heuristic keys off the literal
'--- ', so a prompt whose body contains a Markdown horizontal rule, a diff hunk, or any---sequence is treated as already-sectioned and loses its[User]heading.
Passing the section list (or a boolean) down from extract_prompt_text() instead of re-detecting via strpos() would remove the guesswork. Display-only, no data loss.
| // Store the pending data in the pending array, so we can match the after-hook. | ||
| $pending = $this->get_pending_storage(); | ||
| $this->evict_oldest_pending_if_full( $pending ); | ||
| $pending[ $model ] = array( |
There was a problem hiding this comment.
Low priority / possibly by design: keying on the model instance pairs before/after correctly for sequential calls and for concurrent calls on distinct instances (both covered by tests), but two overlapping generations on the same instance overwrite this entry.
The realistic trigger is nesting β a generation started from inside an after_generate_result listener, or a helper that reuses one cached model across a nested call. The result is one lost record plus a wrong duration and mismatched prompt on the other.
If WP AI Client can't nest on one instance, a note here saying so would close the question; otherwise a small stack per model would cover it.
There was a problem hiding this comment.
WP AI Client runs a strict before β execute β after sandwich. Same-instance reuse is sequential, not nested.
| * @param string $summary Full summary stored in the database. | ||
| * @return string Summary preview text. | ||
| */ | ||
| public static function format_summary_preview_html( $summary ) { |
There was a problem hiding this comment.
This changes the summary column for every connector, not just AI Client.
column_default() routes all records through this, so any multiline summary from any connector (including third-party ones) is now truncated in the activity feed. The changelog frames it as a general enhancement, which is fair, but it's a behavior change beyond the scope suggested by the PR title β is that intended?
Two smaller points:
- There is no affordance signalling that content was hidden. A reader sees a complete-looking line with no hint to open the record. A trailing
β¦would help. - The name says
_htmlwhile the return value is plain text (the caller still passes it throughwp_kses()). For a newpublic staticmethod,format_summary_preview()would describe it more accurately.
β¦ and PII warnings match Stream's existing patterns. Truncated summaries now show an ellipsis, log templates stay translatable without early escaping, and operators are warned that stored prompt text may be forwarded to alerts or webhooks.
Fixes XWPENG-20.
Adds a bundled AI Client connector to Stream that listens to WordPress AI Client
before/after_generate_resulthooks and writes audit records for each generation. Records include acting user, timestamp, provider, model, input/output/thinking token counts, duration, finish reason, and extended metadata. Full prompt and response text capture is supported via a single opt-in setting (log_prompt_and_response_text), disabled by default with a PII warning. Developer filterswp_stream_ai_client_log_promptandwp_stream_ai_client_log_responseallow redaction or omission of stored text. The Stream list table shows only the first line of multiline summaries so token/metadata rows stay scannable when prompt/response logging is enabled.Checklist
contributing.md).Release Changelog
Release Checklist
developbranch.readme.txt.stream.php.Stable taginreadme.txt.classes/class-plugin.php.Test plan
wp_stream_ai_client_log_prompt/wp_stream_ai_client_log_responseto redact text; confirm stored values reflect the filter output.WP_AI_Client_Event_Dispatcher, confirm Stream loads with no fatals and no AI records are written.vendor/bin/phpunit --filter Connector_AI_Clientandvendor/bin/phpunit --filter Test_List_Table.