ToolBoundary: see what an AI tool receives

An agent sends {"q":"shoes"} to a search tool. The tool's Zod schema defines a default limit: 10. In an affected WorkflowAgent version, the handler received no limit. A parser test does not examine the handler input.

The WorkflowAgent issue reports this behavior. Its fix merged in October 2026. The MCP Python issue shows another failure: output validation replaced an explicit tool error.

I made ToolBoundary to find these differences in a test. It runs a stub handler through an SDK's public path. It records the handler input and client result. It compares those values with explicit expectations and reports the field path.

What a probe does

A probe registers a stub handler and calls it through the SDK. It records the handler input. For MCP, it also records the result or error that the client receives. The probe does not call a production handler. The WorkflowAgent probe uses a fake model, so it needs no API key.

Python: an MCP default

Install the Python package with the MCP adapter:

pip install 'tool-boundary-py[mcp-v1]'

This example compares a Pydantic default with the input that a FastMCP handler receives:

import asyncio
from pydantic import BaseModel
from tool_boundary import probe_mcp_tool

class CountInput(BaseModel):
    limit: int = 10

def count(limit: int = 10) -> dict[str, int]:
    return {"count": limit}

def register(server, handler):
    server.tool()(handler)

async def main():
    report = await probe_mcp_tool(
        case_id="python.default",
        tool_name="count",
        arguments={},
        reference=CountInput.model_validate,
        register=register,
        stub_handler=count,
        expected_structured={"count": 10},
    )
    print(report.outcome, report.observations[0].observed)
    assert report.outcome == "pass", report.to_dict()

asyncio.run(main())

The reference parser supplies the expected input. The register function adds the stub handler through the public FastMCP method. The probe also compares the client result with expected_structured. The test passes if the handler receives limit=10 and the client receives {"count":10}.

Python: keep an explicit tool error

The next stub advertises a float success result. It returns an explicit MCP error instead. On affected MCP Python versions, output validation replaces the error text.

import asyncio
from mcp.types import CallToolResult, TextContent
from tool_boundary import probe_mcp_tool

def divide() -> float:
    return CallToolResult(
        isError=True,
        content=[TextContent(type="text", text="Division by zero")],
    )

def register(server, handler):
    server.tool()(handler)

async def main():
    report = await probe_mcp_tool(
        case_id="python.error",
        tool_name="divide",
        arguments={},
        reference=lambda raw: {},
        register=register,
        stub_handler=divide,
        expected_error_text="Division by zero",
    )
    print(report.outcome)
    print([diff.path for diff in report.observations[-1].differences])

asyncio.run(main())

On affected versions, the report shows fail and a difference at /text. The full Python example shows the complete report.

Node.js: a WorkflowAgent input

Install the Node.js package with the WorkflowAgent peer packages:

npm install @esmirnov.ai/tool-boundary 'ai@^7' '@ai-sdk/workflow@^2' 'zod@^4'

This example uses a Zod default and a transform. The probe compares Zod's result with the tool executor input:

import assert from 'node:assert/strict';
import { z } from 'zod';
import { probeWorkflowTool } from '@esmirnov.ai/tool-boundary/workflow';

const report = await probeWorkflowTool({
  caseId: 'node.workflow-input',
  toolName: 'search',
  arguments: { q: 'shoes', tag: 'sale' },
  inputSchema: z.object({
    q: z.string(),
    limit: z.number().default(10),
    tag: z.string().transform((value) => value.toUpperCase()),
  }),
});

console.log(report.outcome, report.observations[0].observed);
assert.equal(report.outcome, 'pass', JSON.stringify(report, null, 2));

A fixed WorkflowAgent sends limit: 10 and tag: "SALE" to the executor. On affected versions, the report names /arguments/limit and /arguments/tag. The assertion fails if those values change.

Node.js: an MCP result

Install the MCP TypeScript SDK with ToolBoundary:

npm install @esmirnov.ai/tool-boundary '@modelcontextprotocol/sdk@^1.29' 'zod@^4'

This example compares an output default with the value that the MCP client receives:

import { z } from 'zod';
import { probeMcpTool } from '@esmirnov.ai/tool-boundary/mcp-v1';

const input = z.object({ name: z.string() });
const output = z.object({
  name: z.string(),
  counted: z.number().default(0),
});

const report = await probeMcpTool({
  caseId: 'node.mcp-output',
  toolName: 'echo',
  arguments: { name: 'x' },
  reference: (raw) => input.parse(raw),
  register: (server, spy) => {
    server.registerTool('echo', { inputSchema: input, outputSchema: output }, spy);
  },
  stubHandler: async ({ name }) => ({
    content: [{ type: 'text', text: name }],
    structuredContent: { name },
  }),
  expectedStructured: { name: 'x', counted: 0 },
});

console.log(report.outcome);
console.log(report.observations.at(-1)?.differences);

The output schema defines counted with a default of 0. The stub returns only name. On affected versions, the client rejects the raw result because counted is absent. The report names /clientAccepted and /structuredContent/counted. The MCP TypeScript issue describes this output problem. The Node.js examples show both adapters.

Read the report

The report format is the same in Python and Node.js. Each report records the SDK versions. Each observation names an input, result, or error boundary. pass means that the values match. fail means that the report has differences. unsupported means that the probe cannot compare the case. harness_error means that the test setup or SDK call failed.

The Python error and Node.js MCP examples show failures on affected SDK versions. For a regression test, assert pass on the behavior that your application needs. The probes examine SDK dispatch and local client responses. They do not examine HTTP or stdio transport.