Skip to content

Commit 5727930

Browse files
🤖 fix: send queued messages on tool-call-end (#665)
Queued messages are now sent as soon as a tool completes execution, rather than waiting for the entire stream to finish. ## Problem Previously, queued messages were only sent on `stream-end`. In tool-using flows, this meant users had to wait for: 1. Tool execution to complete 2. LLM to process the tool result 3. LLM to finish streaming its response This created unnecessary latency when queueing messages during tool execution. ## Solution Extracted a shared `sendQueuedMessages()` method called from both: - `tool-call-end`: Sends queued messages immediately when tool completes - `stream-end`: Continues to handle non-tool streams The first handler to fire clears the queue, so no duplicate sends occur. Non-tool streams continue to work exactly as before. ## Changes - Added `sendQueuedMessages()` private method - Updated `tool-call-end` handler to auto-send - Refactored `stream-end` handler to use shared method Net: +18 lines _Generated with `mux`_
1 parent bb97a71 commit 5727930

File tree

1 file changed

+20
-8
lines changed

1 file changed

+20
-8
lines changed

src/node/services/agentSession.ts

Lines changed: 20 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -408,20 +408,18 @@ export class AgentSession {
408408
forward("stream-delta", (payload) => this.emitChatEvent(payload));
409409
forward("tool-call-start", (payload) => this.emitChatEvent(payload));
410410
forward("tool-call-delta", (payload) => this.emitChatEvent(payload));
411-
forward("tool-call-end", (payload) => this.emitChatEvent(payload));
411+
forward("tool-call-end", (payload) => {
412+
this.emitChatEvent(payload);
413+
// Tool call completed: auto-send queued messages
414+
this.sendQueuedMessages();
415+
});
412416
forward("reasoning-delta", (payload) => this.emitChatEvent(payload));
413417
forward("reasoning-end", (payload) => this.emitChatEvent(payload));
414418

415419
forward("stream-end", (payload) => {
416420
this.emitChatEvent(payload);
417421
// Stream end: auto-send queued messages
418-
if (!this.messageQueue.isEmpty()) {
419-
const { message, options } = this.messageQueue.produceMessage();
420-
this.messageQueue.clear();
421-
this.emitQueuedMessageChanged();
422-
423-
void this.sendMessage(message, options);
424-
}
422+
this.sendQueuedMessages();
425423
});
426424

427425
forward("stream-abort", (payload) => {
@@ -529,6 +527,20 @@ export class AgentSession {
529527
});
530528
}
531529

530+
/**
531+
* Send queued messages if any exist.
532+
* Called when tool execution completes or stream ends.
533+
*/
534+
private sendQueuedMessages(): void {
535+
if (!this.messageQueue.isEmpty()) {
536+
const { message, options } = this.messageQueue.produceMessage();
537+
this.messageQueue.clear();
538+
this.emitQueuedMessageChanged();
539+
540+
void this.sendMessage(message, options);
541+
}
542+
}
543+
532544
private assertNotDisposed(operation: string): void {
533545
assert(!this.disposed, `AgentSession.${operation} called after dispose`);
534546
}

0 commit comments

Comments
 (0)