Add Exie diagnostics and user-controlled conversation sharing - #2567
Add Exie diagnostics and user-controlled conversation sharing#2567ejsmith wants to merge 18 commits into
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 515dad0acc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5ccd9e6c26
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fe5d3513da
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9f55a0b850
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4196ca56dd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
ejsmith
changed the title
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 39588e2262
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 65c5b30f41
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
ejsmith
changed the title
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c1dcc9694e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
ejsmith
changed the title
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2350fb341e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
|
Codex Review: Didn't find any major issues. Delightful! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c500262648
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| public sealed record UpdateAssistantConversationSharingSettings | ||
| { | ||
| public bool Enabled { get; init; } |
There was a problem hiding this comment.
When a client sends {} to the new admin endpoint, System.Text.Json initializes this non-nullable value type to false, and AutoValidationEndpointFilter has no annotation or required-member metadata with which to reject the missing property. The request therefore silently disables the global conversation-sharing default even though the generated OpenAPI contract marks enabled as required; declare the property required or otherwise validate its presence before persisting it.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 25e45c0c17
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| public void StartTool(string name) | ||
| { | ||
| Stage = "tool_execution"; | ||
| LastTool = GetToolName(name); |
There was a problem hiding this comment.
When multiple tool calls run and an earlier call fails but a later call succeeds, this overwrites LastTool with the successful tool while RecordToolResult leaves LastToolError set to the earlier failure. Finish then emits the two fields together, falsely attributing the stale error code to the latest tool and misleading incident investigation. Clear the error when a new tool starts or retain the name of the tool associated with the most recent error.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 021d07ff53
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| [InlineData("{\"choices\":[{\"delta\":[]}]}")] | ||
| [InlineData("{\"choices\":[{\"delta\":{\"tool_calls\":[{}]}}]}")] | ||
| [InlineData("{\"choices\":[{\"delta\":{\"tool_calls\":[{\"index\":2147483648}]}}]}")] | ||
| [InlineData("{\"usage\":{\"prompt_tokens\":\"private invalid token count\"}}")] |
There was a problem hiding this comment.
Fresh evidence after the earlier shape-classification fix is this newly added theory input, which cannot satisfy the assertions below it: TryGetProviderUsage treats the string token count as absent, the missing choices property is skipped, and after [DONE] both provider and turn diagnostics report empty_response, not invalid_provider_response. This theory therefore fails consistently for this row; either reject nonnumeric usage fields in the parser or give this case its actual expected outcome.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6ac309e9ee
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| Exceptionless.config.setUserIdentity(userId); | ||
| } | ||
| Exceptionless.config.setUserIdentity(userId, userName ?? ''); | ||
| Exceptionless.config.currentSessionIdentifier = userId; |
There was a problem hiding this comment.
The caller supplies email_address as userId, so assigning it directly as the session identifier gives every login and browser tab for that account the same value, commonly containing invalid characters such as @ and .. Session processing groups events by this identifier and discards duplicate starts; invalid reference IDs are also rewritten to invalid-reference-id, causing unrelated browser sessions to be merged, closed by another tab, or discarded. Keep the email as user identity, but generate a distinct valid identifier for each session.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 91fb92b148
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| export function trackAssistantEvent(feature: string, context: AssistantTelemetryContext, details: Record<string, unknown> = {}): void { | ||
| // A telemetry failure must not interrupt a chat or generate another telemetry event. | ||
| void submitFeatureUsage(feature, getProperties(context, details)).catch(() => {}); |
There was a problem hiding this comment.
On an authenticated page reload, the assistant panel can emit assistant.Opened while the independently fetched meQuery is still pending. This submits the usage event immediately, while configureSessions only cancels unidentified session events, so the event is queued without the signed-in user or session and is not associated retroactively when setUserIdentity later runs. Queue or suppress assistant telemetry until the identity initialization completes so the new conversation diagnostics remain session-correlated.
Useful? React with 👍 / 👎.
|
1 participant
Summary
Add diagnostics to explain failed Exie turns and understand how people use the feature. Correlated server logs, traces, and metrics record provider failures, tool execution, cancellations, and timing. Browser events use the existing Exceptionless session to record outcomes, feedback, retries, and departures while waiting. Existing application error collection remains unchanged, and Exie failure events retain the error shown to the user. Outcome counts in the Exie usage table include hover and keyboard-focus tips explaining completed, failed, and cancelled turns.
Conversation text sharing is controlled separately:
Development Aspire runs report API and background job warnings and errors, plus browser errors, usage, and sessions, to their own seeded Exceptionless → Exceptionless project by default. Server reporting uses the local API endpoint; browser reporting uses the current origin so worktrees and forwarded ports work. No local env-file setup is required. Server and browser reporting can be disabled independently; the browser destination can be overridden. New sample data uses the project name Exceptionless; existing project names are preserved.
Browser session starts now require a user identity, preventing empty startup/resume sessions. A single telemetry identity update handles profile loading and logout, and older sessions without identity display a clickable Anonymous session summary. Exie open events occur only when the chat becomes visible, including full-page reloads.
Adds admin and current-user sharing endpoints, an additive chat response header, updated API contracts, and an investigation guide. No breaking changes to existing APIs or chat stream formats.
Validation
.env.local, same-origin event submission, and session correlation.