serving assets

[mgathena]

Hi, do i always have to serve the files from a different server? I am working on a python example. I took some code from a medium post.

here is the complete code.

and a relevant snippet

def create_calculator_widget():
{SOME_HTML_SNIPPET}

@mcp.resource("ui://widget/calculator-result.html")
async def calculator_widget():
    """Serve the calculator widget HTML."""
    return {
        "uri": "ui://widget/calculator-result.html",
        "mimeType": "text/html+skybridge",
        "text": create_calculator_widget(),
        "_meta": {
            "openai/widgetDescription": "Displays the result of addition calculations in a beautiful gradient card format.",
            "openai/widgetPrefersBorder": False,
        },
    }

@mcp.tool()
def add(a: float, b: float) -> dict:
    """Add two numbers together and return structured result."""
    result = a + b
    return {
        "content": [{"type": "text", "text": f"Added {a} + {b} = {result}"}],
        "structuredContent": {
            "a": a,
            "b": b,
            "result": result,
            "operation": "addition",
            "timestamp": "2024–01–15T10:30:00Z",
        },
        "_meta": {
            "openai/outputTemplate": "ui://widget/calculator-result.html",
            "openai/toolInvocation/invoking": "Calculating…",
            "openai/toolInvocation/invoked": "Calculation complete!",
            "openai/widgetAccessible": True,
            "openai/resultCanProduceWidget": True,
        },
    }

i run the server and then expose via ngrok. here is the response shown on chatgpt

text:
'{ "content": [ { "type": "text", "text": "Added 3.0 + 7.0 = 10.0" } ], "structuredContent": { "a": 3.0, "b": 7.0, "result": 10.0, "operation": "addition", "timestamp": "2024–01–15T10:30:00Z" }, "_meta": { "openai/outputTemplate": "ui://widget/calculator-result.html", "openai/toolInvocation/invoking": "Calculating…", "openai/toolInvocation/invoked": "Calculation complete!", "openai/widgetAccessible": true, "openai/resultCanProduceWidget": true } }'

but no UI (or even placeholder appears)

[desi-voice]

Same issue

[frozenabe]

you should be defining the meta in the tool not as a response.

mcp.tool(meta=….)

[baxang]

I was utterly confused about this too. While the Kanban board example on https://developers.openai.com/apps-sdk/build/mcp-server#structure-the-data-your-tool-returns shows:

• tool's meta has "openai/outputTemplate"
• tool response does not!

however the current version of this repo has some examples that return widget in tool response's meta:

  server.setRequestHandler(
    CallToolRequestSchema,
    async (request: CallToolRequest) => {
      const widget = widgetsById.get(request.params.name);


      if (!widget) {
        throw new Error(`Unknown tool: ${request.params.name}`);
      }


      const args = toolInputParser.parse(request.params.arguments ?? {});


      return {
        content: [
          {
            type: "text",
            text: widget.responseText,
          },
        ],
        structuredContent: {
          pizzaTopping: args.pizzaTopping,
        },
        _meta: widgetInvocationMeta(widget), // <-- this line!
      };
    }
  );

And the tool response includes all widget meta data:

{
  "_meta": {
    "openai/outputTemplate": "ui://widget/pizza-list.html",
    "openai/toolInvocation/invoking": "Hand-tossing a list",
    "openai/toolInvocation/invoked": "Served a fresh list",
    "openai/widgetAccessible": true,
    "openai/resultCanProduceWidget": true
  },
  "content": [
    {
      "type": "text",
      "text": "Rendered a pizza list!"
    }
  ],
  "structuredContent": {
    "pizzaTopping": ""
  }
}

I am going to follow the code example in the doc to have component defined in the tool only.

Another confusing thing about the example app is, the use of meta.openai.com/widget in Python server apps.

    meta: Dict[str, Any] = {
        "openai.com/widget": widget_resource.model_dump(mode="json"),
        "openai/outputTemplate": widget.template_uri,
        "openai/toolInvocation/invoking": widget.invoking,
        "openai/toolInvocation/invoked": widget.invoked,
        "openai/widgetAccessible": True,
        "openai/resultCanProduceWidget": True,
    }

This is returned in a call tool response. It didn't work in our testing.

With limited information given, I am going to stick to these

• openai/outputTemplate in tool definition
• assume embedded resource works in neither content nor meta in call tool response

[mgathena]

@frozenabe Do you have a complete working example? the one in the docs uses list_tools and list_resources. Makes the code messy (IMO). Having a separate tool/resource(?) makes it cleaner.

[frozenabe]

When chatgpt connects the MCP, (or refresh)
it will call list tools and if the tool contains openai meta data in its signature, it will wire up its resource in the system so that when the tool is called it automatically read resource to render the widget (simplifying). the tools output is whats feeds into the widget.

Also “other server” probably you are referring to the string ui:// .. this is a contracted signature (string) (pub doc has more info).