Skip to content

fix(resolver): response format and evaluator metrics in deactivated branch#3152

Merged
icecrasher321 merged 4 commits intostagingfrom
fix/resolver-resp-format
Feb 6, 2026
Merged

fix(resolver): response format and evaluator metrics in deactivated branch#3152
icecrasher321 merged 4 commits intostagingfrom
fix/resolver-resp-format

Conversation

@icecrasher321
Copy link
Collaborator

Summary

Response format in not executed agent block and evaluator metrics should still be treated as accurate.

Type of Change

  • Bug fix

Testing

Tested manually

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel
Copy link

vercel bot commented Feb 6, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Feb 6, 2026 8:02am

Request Review

@icecrasher321
Copy link
Collaborator Author

@cursor review

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 6, 2026

Greptile Overview

Greptile Summary

Fixed variable resolution in deactivated workflow branches by exposing output schemas for evaluator metrics and agent response formats, even when blocks haven't executed. This enables proper validation and autocomplete for variable references like <agent1.field> throughout the workflow, regardless of execution state. Also added childWorkflowId to workflow block outputs for better traceability.

Key Changes:

  • Added getEvaluatorMetricsSchema() to dynamically generate schemas from evaluator block metric configurations
  • Added getResponseFormatSchema() to parse and expose agent response format schemas
  • Both functions are called in getBlockSchema() before checking execution state, ensuring schemas are available for all blocks
  • Added childWorkflowId output field to both workflow and workflow_input blocks

Technical Context:
The collectBlockData() function builds blockOutputSchemas for ALL workflow blocks, not just executed ones. This is critical for the variable resolver to validate references to blocks in deactivated conditional branches. Previously, only inputFormat-based schemas were exposed regardless of execution state. This fix extends that behavior to evaluator metrics and agent response formats.

Confidence Score: 5/5

  • This PR is safe to merge with minimal risk
  • The changes are well-structured and follow existing patterns in the codebase. The new helper functions mirror the existing getInputFormatFields() approach, with proper validation and error handling. The evaluator metrics logic correctly filters valid metrics and normalizes names to lowercase. The response format parsing uses existing utility functions (parseResponseFormatSafely, extractFieldsFromSchema) that already handle edge cases. The addition of childWorkflowId is a straightforward schema extension. All changes are defensive with proper null checks and early returns.
  • No files require special attention

Important Files Changed

Filename Overview
apps/sim/executor/utils/block-data.ts Added getEvaluatorMetricsSchema and getResponseFormatSchema functions to expose dynamic output schemas for evaluator metrics and agent response formats, even when blocks are not executed
apps/sim/blocks/blocks/workflow.ts Added childWorkflowId to output schema for better traceability of child workflow executions
apps/sim/blocks/blocks/workflow_input.ts Added childWorkflowId to output schema for better traceability of child workflow executions

Sequence Diagram

sequenceDiagram
    participant Executor as Workflow Executor
    participant BlockData as collectBlockData()
    participant GetSchema as getBlockSchema()
    participant Evaluator as getEvaluatorMetricsSchema()
    participant ResponseFmt as getResponseFormatSchema()
    participant Resolver as Variable Resolver

    Executor->>BlockData: Collect block data and schemas
    
    loop For each block in workflow
        BlockData->>GetSchema: Get output schema for block
        
        alt Block has inputFormat outputs
            GetSchema-->>BlockData: Return merged inputFormat schema
        else Block is evaluator
            GetSchema->>Evaluator: Check evaluator metrics
            Evaluator-->>GetSchema: Return metrics schema (even if not executed)
            GetSchema-->>BlockData: Return evaluator schema
        else Block has responseFormat
            GetSchema->>ResponseFmt: Parse responseFormat param
            ResponseFmt-->>GetSchema: Return parsed schema (even if not executed)
            GetSchema-->>BlockData: Return response format schema
        else Block is trigger or has outputs
            GetSchema-->>BlockData: Return block outputs
        end
    end
    
    BlockData-->>Executor: blockOutputSchemas for all blocks
    
    Note over Resolver: Variable references can now be validated<br/>even for non-executed blocks in deactivated branches
Loading

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.

@icecrasher321 icecrasher321 merged commit 65de273 into staging Feb 6, 2026
12 checks passed
@icecrasher321 icecrasher321 deleted the fix/resolver-resp-format branch February 6, 2026 08:14
waleedlatif1 pushed a commit that referenced this pull request Feb 6, 2026
…ranch (#3152)

* fix(resolver): response format in deactivated branch

* add evaluator metrics too

* add child workflow id to the workflow block outputs

* cleanup typing
waleedlatif1 added a commit that referenced this pull request Feb 6, 2026
* feat(skills): added skills to agent block

* improvement(skills): audit fixes, docs, icon, and UX polish

* fix(skills): consolidate redundant permission checks in POST and DELETE

* more friendly error for duplicate skills in the same workspace

* fix(executor):  loop sentinel-end wrongly queued (#3148)

* fix(executor):  loop sentinel-end wrongly queued

* fix nested subflow error highlighting

* fix(linear): align tool outputs, queries, and pagination with API (#3150)

* fix(linear): align tool outputs, queries, and pagination with API

* fix(linear): coerce first param to number, remove duplicate conditions, add null guard

* fix(resolver): response format and evaluator metrics in deactivated branch (#3152)

* fix(resolver): response format in deactivated branch

* add evaluator metrics too

* add child workflow id to the workflow block outputs

* cleanup typing

* feat(slack): add file attachment support to slack webhook trigger (#3151)

* feat(slack): add file attachment support to slack webhook trigger

* additional file handling

* lint

* ack comment

* fix(skills): hide skill selection when disabled, remove dead code

---------

Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant