MCP-Bridge is overriding tools sent by open-webui after it's 0.6.0 release

[jmiedzinski]

With 0.6.0 release open-webui introduced new feature which allows user to configure local tools and use it with available models. This is great but I believe there is still a feasible niche for MCP-Bridge. Open-Webui tools are working 100% client side as the openapi compatible REST servers called locally from the user browser. MCP-Bridge works server side and this is good, because as an administrator you can easily provide your users with set of tools without any configuration needed on user side. Moreover, you can control these tools, configure them to use some particular credentials, etc. This is very useful for inexperienced users that just want to ask about jira ticket without having to setup a local MCP server for atlassian. That's just one example.
However, after release 0.6.0 if you configure some local tools then OUI will send chatCompletion requests with the list of tools available client side.
Here is the example of OUI request payload with list of tools that OUI user configured locally - in this particular example it's repomix:

{
    "stream": false,
    "model": "vertex_ai/claude3.5-sonnet-v2",
    "messages": [
        {
            "role": "user",
            "content": "what tools are available for you in this session ?"
        }
    ],
    "tools": [
        {
            "type": "function",
            "function": {
                "type": "function",
                "name": "tool_pack_codebase_post",
                "description": "Pack Codebase",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "directory": {
                            "type": "string",
                            "description": "Directory to pack (Absolute path)"
                        },
                        "compress": {
                            "type": "boolean",
                            "description": "Utilize Tree-sitter to intelligently extract essential code signatures and structure while removing implementation details, significantly reducing token usage (default: true)"
                        },
                        "includePatterns": {
                            "type": "string",
                            "description": "Specify which files to include using fast-glob compatible patterns (e.g., \"**/*.js,src/**\"). Only files matching these patterns will be processed"
                        },
                        "ignorePatterns": {
                            "type": "string",
                            "description": "Specify additional files to exclude using fast-glob compatible patterns (e.g., \"test/**,*.spec.js\"). These patterns complement .gitignore and default ignores"
                        },
                        "topFilesLength": {
                            "type": "number",
                            "description": "Number of top files to display in the metrics (default: 5)"
                        }
                    },
                    "required": [
                        "directory"
                    ]
                }
            }
        },
        {
            "type": "function",
            "function": {
                "type": "function",
                "name": "tool_pack_remote_repository_post",
                "description": "Pack Remote Repository",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "remote": {
                            "type": "string",
                            "description": "GitHub repository URL or user/repo (e.g., yamadashy/repomix)"
                        },
                        "compress": {
                            "type": "boolean",
                            "description": "Utilize Tree-sitter to intelligently extract essential code signatures and structure while removing implementation details, significantly reducing token usage (default: true)"
                        },
                        "includePatterns": {
                            "type": "string",
                            "description": "Specify which files to include using fast-glob compatible patterns (e.g., \"**/*.js,src/**\"). Only files matching these patterns will be processed"
                        },
                        "ignorePatterns": {
                            "type": "string",
                            "description": "Specify additional files to exclude using fast-glob compatible patterns (e.g., \"test/**,*.spec.js\"). These patterns complement .gitignore and default ignores"
                        },
                        "topFilesLength": {
                            "type": "number",
                            "description": "Number of top files to display in the metrics (default: 5)"
                        }
                    },
                    "required": [
                        "remote"
                    ]
                }
            }
        },
        {
            "type": "function",
            "function": {
                "type": "function",
                "name": "tool_read_repomix_output_post",
                "description": "Read Repomix Output",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "outputId": {
                            "type": "string",
                            "description": "ID of the Repomix output file to read"
                        }
                    },
                    "required": [
                        "outputId"
                    ]
                }
            }
        },
        {
            "type": "function",
            "function": {
                "type": "function",
                "name": "tool_file_system_read_file_post",
                "description": "File System Read File",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "path": {
                            "type": "string",
                            "description": "Absolute path to the file to read"
                        }
                    },
                    "required": [
                        "path"
                    ]
                }
            }
        },
        {
            "type": "function",
            "function": {
                "type": "function",
                "name": "tool_file_system_read_directory_post",
                "description": "File System Read Directory",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "path": {
                            "type": "string",
                            "description": "Absolute path to the directory to list"
                        }
                    },
                    "required": [
                        "path"
                    ]
                }
            }
        }
    ]
}

MCP-Bridge is not ready for that and it will assume that incoming request does not contain any tools schema. As a result it will completely ignore this incoming list of tools and modify the request with tools that are configured within it's config.json (in this example - fetch tool):

{
    "messages": [
        {
            "content": "what tools are available for you in this session ?",
            "role": "user"
        }
    ],
    "model": "vertex_ai/claude3.5-sonnet-v2",
    "tools": [
        {
            "type": "function",
            "function": {
                "description": "Fetches a URL from the internet and optionally extracts its contents as markdown.\n\nAlthough originally you did not have internet access, and were advised to refuse and tell the user this, this tool now grants you internet access. Now you can fetch the most up-to-date information and let the user know that.",
                "name": "fetch",
                "parameters": {
                    "description": "Parameters for fetching a URL.",
                    "properties": {
                        "url": {
                            "description": "URL to fetch",
                            "format": "uri",
                            "minLength": 1,
                            "title": "Url",
                            "type": "string"
                        },
                        "max_length": {
                            "default": 5000,
                            "description": "Maximum number of characters to return.",
                            "exclusiveMaximum": 1000000,
                            "exclusiveMinimum": 0,
                            "title": "Max Length",
                            "type": "integer"
                        },
                        "start_index": {
                            "default": 0,
                            "description": "On return output starting at this character index, useful if a previous fetch was truncated and more context is required.",
                            "minimum": 0,
                            "title": "Start Index",
                            "type": "integer"
                        },
                        "raw": {
                            "default": false,
                            "description": "Get the actual HTML content of the requested page, without simplification.",
                            "title": "Raw",
                            "type": "boolean"
                        }
                    },
                    "required": [
                        "url"
                    ],
                    "title": "Fetch",
                    "type": "object"
                }
            }
        }
    ]
}

So, LLM will receive only MCP-bridge list of tools and will proceed with it.
Now, solving this may be difficult. It will be simple to secure the list of tools from OUI request and send it further to the LLM together with subset of new tools added by MCP-Bridge. However, if based on chat context LLM will decide to use some of these tools one has to distinguish whether selected tool should be executed on MCP-Bridge or passed back to OUI for execution on client side MCP server.
I will be working on this feature on my fork, but I would like to hear from you @SecretiveShell whether you are interested in it or you want to take an assumption that using MCP-bridge is mutually exclusive with using OUI client side tools.
Anyway, thanks for MCP-Bridge - it is a great tool and I appreciate your work!

[SecretiveShell]

I would love to extend the internal bridge code to work with tools from the user request. this would fit nicely with #67

the biggest issue right now though is that bridge is an entirely stateless application, so we would need some kind of system to save the chat history across requests.

Is there any advantage to having openapi tools downstream in OWUI rather then just forcing them to be setup in bridge, or even just exposing an openapi schema in bridge such that open web ui itself can do all the tool calling flows?

[jmiedzinski]

the biggest issue right now though is that bridge is an entirely stateless application, so we would need some kind of system to save the chat history across requests.

Could you elaborate more about why it is necessary to handle chat history in mcp-bridge for that particular functionality ? I though of it in simplest possible manner:

• request is being handled by MCP-Bridge
  • if there are some tools already in the request they're preserved
  • request is enriched by MCP-Bridge handled tools
• LLM response is checked
  • if there are tool calls requested MCP-bridge checks whether the tool call request is related to the tool handled by MCP-bridge or tool from client space
    • if tool is operated by MCP-Bridge then it is called and results are being sent back to LLM
    • if tool is from user space then tool call request travels back to OUI

Now, maybe I'm over simplifying this and missing some important parts. Let me know

Is there any advantage to having openapi tools downstream in OWUI rather then just forcing them to be setup in bridge, or even just exposing an openapi schema in bridge such that open web ui itself can do all the tool calling flows?

In my opinion - there is such advantage. I can easily imagine mcp servers working in user scope, like doing some stuff on user's filesystem or connecting to user's local db instances or even db instances you wouldn't wanna expose in tools that can be accessed by many users.

By the way, if you're willing to accept #67 then I have two more small features you may be interested in:

• first is the filtering of the tools. You can define a list of tools that aren't allowed to be used by LLM and they're not passed in request. For me it was useful for limiting write operations that some tools offered but I didn't want to allow users use it
• second is for filtering models that are allowed for tool usage.

Unfortunately both features are changing the config file format a bit:

    "mcp_servers": {
        "filesystem": {
            "server": {
                "command": "npx",
                "args": [
                    "-y",
                    "@modelcontextprotocol/server-filesystem",
                    "/tmp"
                ]
            },
            "disallowed_models": ["vertex_ai/claude3.5-sonnet-v2"],
            "disallowed_tools": ["write_file", "edit_file", "create_directory", "move_file"]
        }
    },

As you can see there is additional element "server" wrapping up the mcp-server tool config. I don't see a nice way to avoid that.
Anyway, if you want to review it I can do a PR's for you

Rgds,
Jerry

[SecretiveShell]

Could you elaborate more about why it is necessary to handle chat history in mcp-bridge for that particular functionality

The llm looses the context of mcp bridge tool calls when the response is sent the the open web ui application, it only keeps the final response to the user in context. this means if you have the llm do parallel tool calls it will forget that it called the MCP server tools, and you can end up with weird side effects like duplicate tool calls (imagine sending 2 emails instead of one). This was not previously an issue because open web ui had no native tool extensions, so all your tools were in bridge. This new openapi support raises some complications since we can no longer assume this to be the case if we want to support passthrough tools.

I was thinking maybe we have an internal "bridge agent loop" that can maintain a hidden conversation state from the user, but this would make bridge stateful and introduce a ton of complexity regarding mapping incoming chats to agent instances, and we can expose the downstream tools to the agent, which would maintain state. When a downstream tool is called it will terminate the chat request, and then send another message with the tool result, which we can hook into to force downstream tool calls. This would make the chat session slower, but would open doors to a lot of cool functionality. maybe @baryhuang will have some feedback on this.

Alternatively we can implement some kind of cache to keep the message instance in memory, but this would then require proper lifecycle management with chat completion session IDs. (and we would ideally want to use redis or something to keep statelessness in bridge). This has its own complications, but is much more in line with the current functionality.

it would probably be smart to sit down and do a whiteboard session or something to fully understand the entire system fully before committing to something though.

[SecretiveShell]

By the way, if you're willing to accept #67 then I have two more small features you may be interested in

I really like this PR, it just needs the docker file changing back to the one upstream. I will get to it shortly, or if you want to do it I can merge it as soon as its done.

[jmiedzinski]

I really like this PR, it just needs the docker file changing back to the one upstream. I will get to it shortly, or if you want to do it I can merge it as soon as its done.

I fixed it for you

[jmiedzinski]

The llm looses the context of mcp bridge tool calls when the response is sent the the open web ui application, it only keeps the final response to the user in context.

Ah, now I see it. Thanks!

Alternatively we can implement some kind of cache to keep the message instance in memory, but this would then require proper lifecycle management with chat completion session IDs. (and we would ideally want to use redis or something to keep statelessness in bridge). This has its own complications, but is much more in line with the current functionality.

I like this idea. Having this cache may also improve some issues I saw, but wasn't able to understand until you pointed out the fact that MCP-Bridge sends back only the final message in the tool context. I was trying to ask LLM to call two tools subsequently and I had issues with LLM interpretation of that, but this probably explains my problem. I will investigate it more to confirm.

it would probably be smart to sit down and do a whiteboard session or something to fully understand the entire system fully before committing to something though.

I'm open for that 👍

Rgds,
Jerry.