Official

visuals

HTML pages (html_preview, html_render) and native inline charts, positioned diagrams and sortable tables in agent replies.

The app opens the listing; nothing installs until an agent in your Plugins workspace has read the files and you enable the plugin. In a terminal: dvg install convergence/visuals@0.2.0

Visuals — official plugin

convergence/visuals provides publish_visual for interactive, native visual replies. It ships enabled on a fresh install; an existing install keeps the choice it has, and users can turn it off or on in Settings → Plugins. It starts only when a reply first contains one of its fences and loads Pele only on the first Mermaid fence, so a chat without fences pays nothing. Its only requested permission is tools.provide.

HTML pages (T3 Code's html_preview and html_render)

html.ts holds T3's contract (pingdotgg/t3code b24f0fbb): the tool names, descriptions, limits (HTML 1–512000 UTF-16 units, title 1–200, preview width 240–1600 with 728 by default, frame height 80–2000), the result shapes and messages, the frame-height rule, and the theme bootstrap with Divergence's colours. Both tools are direct, so in code mode they stay tools of their own and the preview's PNG reaches the agent as an MCP image part.

  • html_preview sends the themed page to host/web.preview. With a confined renderer it answers T3's metadata as text and structured content plus the PNG. Without one it is an error, HtmlBackendUnavailable: <raw reason>, with no image: nothing renders unconfined and no screenshot is made up.
  • html_render saves the agent's page unchanged as a html v1 transcript artifact (the host stamps the owner) and answers T3's htmlRender reference and message. Measured heights are not supplied.
  • Inline (html-view.ts) the page is live (ui.webDocument): borderless in the reply, as wide as the column and as tall as the page up to the agent's height, with one small glass Open in panel control. The panel is a detail-pane tab (the convergence/tabs service): the page live again, its Source, Copy and Save (kernel/copy_text, kernel/save_text, both after a click).

The renderer is the host's: Debian's Chromium in a disposable container of the local Docker engine, admitted through chrome://sandbox (crates/host/src/web). Differences from T3 and from a system web view: pages are offline (remote URLs and local image paths never load; missingImages lists the paths a page names), fonts and text metrics are Linux's (DejaVu, Liberation, Noto Color Emoji), there is no GPU, and native browser menus, pickers, IME composition, accessibility, clipboard and drag from the page are not available. Without Docker or the built renderer image the page says why and its source, copy and save still work.

Version 2 supports responsive multi-panel compositions: bar (grouped/stacked/ percent; horizontal/vertical), line, area, scatter/bubble, pie/donut, heatmap, histogram, range/timeline, waterfall and boxplot, plus metrics, text, typed tables and positioned diagrams. document.ts defines the strict, bounded JSON schema; rules.md is the agent-facing reference. Version 1 documents still replay.

Rendering and interaction

Rust/GPUI draws every shape, label, control and floating tooltip. A pinned, DOM-free D3 math subset computes scales, numeric/UTC ticks and paths inside QuickJS; there is no browser, JavaScript UI execution, graph auto-layout, network, file, process or plugin-storage access. math.LICENSES.txt includes provenance and licenses; scripts/bundle-visuals-math.mjs reproduces the readable bundle.

Charts use the native canvas's onResize({width,height}) logical-pixel measurement, not a scaled viewBox. Text remains readable as panels grow or wrap. Compact legends keep original domains, colors and percentage denominators when hidden. Null breaks paths and stays explicitly missing in shared bar inspection.

Hover opens a clamped floating tooltip; click pins/unpins. Arrow keys, Home/End traverse observations, Enter/Space pin, Escape clears inspection. In the expanded overlay Escape unpins first, then closes. Tables retain stable raw row identity, sort nulls last in either direction, paginate and expose full selected values. Diagram node details remain accessible through a native picker.

State and cached geometry belong to each artifact's ctx.state, with separate panels and inline/expanded inspection. Hover does not recompute paths. Native resize reports are change-only and coalesced off-paint. No JavaScript frame loop or render-time notification runs.

Persistence and compatibility

The host owns immutable transcript artifacts. ctx.artifacts.publish persists the complete document independently of the surrounding tool result, including discarded acknowledgements and later code-mode errors. The plugin registers removable visual renderers for versions 1 and 2; Source retains original input and Data retains all raw values, including hidden series and nulls.

This plugin requires the updated native host and canvas resize API. Older hosts without artifact support fail with an explicit update message. Invalid persisted documents keep their readable summary and expose a validation error.

Verification

node --import ./plugins/testing/register.mjs --test plugins/visuals/*.test.ts
pnpm -C plugins typecheck
pnpm -C plugins exec prettier --check 'visuals/**/*.{ts,json,md}'

Geometry tests use asymmetric data and independent numeric expectations; view tests exercise real native element factories and handlers. Native desktop checks and app-owned Metal captures supplement these tests; they do not use a browser.

Permissions in 0.2.0

  • Provide agent tools tools.provideMediumGives tools to agents.Let agents preview and publish HTML pages and bounded visual replies into the current chat

Files

main.test.ts17.1 KB
// Run the real Effect adapter against a tiny local API: registration lifetimes and tool publications.
// No host persistence is faked in plugin storage; the record stands for the host-owned transcript.
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import { nodes } from "../sdk/ui-testing.ts";
import { test } from "node:test";
import type {
  Api,
  ArtifactRendererDefinition,
  Registration,
  ToolContext,
  ToolDefinition,
  ToolResult,
  ViewContext,
} from "convergence";
import { richVisualDocumentSchema } from "./document.ts";
import { activate } from "./main.ts";
import { chartDocument } from "./testing.ts";

type HostCall = (method: string, params: unknown) => Promise<unknown>;
function fakeApi(rendererAvailable = true, host: { call?: HostCall; kernel?: HostCall; tabs?: unknown } = {}) {
  const views = new Map<string, (ctx: ViewContext) => unknown>();
  const active = new Set<string>();
  const removals: string[] = [];
  const notifications: string[] = [];
  const tools = new Map<string, ToolDefinition>();
  const renderers: ArtifactRendererDefinition[] = [];
  const registration = (name: string): Registration => {
    active.add(name);
    return {
      remove: () => {
        active.delete(name);
        removals.push(name);
      },
    };
  };
  // This is a locally constructed API fixture; unused API members are never needed by this program.
  const api = {
    plugin: "visuals",
    id: "convergence/visuals",
    artifactRenderer: rendererAvailable
      ? (definition: ArtifactRendererDefinition) => {
          renderers.push(definition);
          return registration(`renderer-${definition.type}-${definition.version}`);
        }
      : undefined,
    tools: {
      register: (definition: ToolDefinition) => {
        tools.set(definition.name, definition);
        return registration(`tool-${definition.name}`);
      },
    },
    slot: (slot: string, render: (ctx: ViewContext) => unknown) => {
      views.set(slot, render);
      const handle = registration(`view-${slot}`);
      return { id: 42, update: () => {}, remove: handle.remove };
    },
    services: {
      when: (name: string, fn: (client: unknown) => void) => {
        fn(name === "convergence/tabs" ? host.tabs : undefined);
        return () => {};
      },
    },
    app: { state: () => ({ workspaceId: "w1" }) },
    host: {
      call: host.call ?? (async (method: string) => Promise.reject(new Error(`unknown host method ${method}`))),
      kernel: host.kernel ?? (async (method: string) => Promise.reject(new Error(`unknown kernel method ${method}`))),
    },
    notify: (message: string) => {
      notifications.push(message);
    },
  } as unknown as Api;
  return {
    api,
    active,
    removals,
    notifications,
    tool: (name = "publish_visual") => tools.get(name),
    tools,
    renderers,
    views,
  };
}
function toolContext(publish: ToolContext["artifacts"]["publish"]): ToolContext {
  return {
    chatId: "chat-1",
    scope: "turn-1",
    signal: new AbortController().signal,
    artifacts: { publish },
    subagent: {
      start: async () => {
        throw new Error("Visuals must not start subagents");
      },
    },
  };
}
const tick = () => new Promise<void>((resolve) => setImmediate(resolve));

test("activation registers the visual and HTML renderers and three tools; all unload once", async () => {
  const fake = fakeApi();
  const unload = activate(fake.api);
  const registered = [
    "view-visuals-page",
    "renderer-visual-1",
    "renderer-visual-2",
    "renderer-html-1",
    "tool-html_preview",
    "tool-html_render",
    "tool-publish_visual",
  ];
  assert.deepEqual([...fake.active], registered);
  assert.deepEqual(
    fake.renderers.map((r) => [r.type, r.version]),
    [
      ["visual", 1],
      ["visual", 2],
      ["html", 1],
    ],
  );
  assert.equal(fake.tool("html_preview")?.direct, true, "the PNG reaches the agent as an image in code mode too");
  assert.equal(fake.tool("html_render")?.direct, true);
  assert.equal(fake.tool()?.direct, undefined, "publish_visual stays callable from execute");
  await unload();
  await unload();
  assert.equal(fake.active.size, 0);
  assert.deepEqual(fake.removals.sort(), [...registered].sort());
  assert.deepEqual(fake.notifications, []);
});

test("publication sends the complete document to the host, returns a small acknowledgement and is not tied to outer results", async () => {
  const fake = fakeApi();
  const unload = activate(fake.api);
  const publications: unknown[] = [];
  const ctx = toolContext(async (value) => {
    publications.push(value);
    return { id: "artifact-1" };
  });
  const tool = fake.tool();
  assert.ok(tool);
  try {
    assert.deepEqual(await tool.run(chartDocument, ctx), { artifactId: "artifact-1", published: true });
    assert.deepEqual(publications, [
      { type: "visual", version: 1, title: chartDocument.title, summary: chartDocument.summary, data: chartDocument },
    ]);
    assert.equal(
      "owner" in Object(publications[0]),
      false,
      "ownership is stamped by the host, never claimed by the plugin",
    );
    await assert.rejects(async () => {
      await tool.run(chartDocument, ctx);
      throw new Error("outer execute failed");
    }, /outer execute failed/);
    assert.equal(publications.length, 2, "a later outer failure does not retract a publication");
  } finally {
    await unload();
  }
});

test("the adapter validates before publishing and publication failures surface instead of acknowledging success", async () => {
  const fake = fakeApi();
  const unload = activate(fake.api);
  const tool = fake.tool();
  assert.ok(tool);
  let publications = 0;
  const ctx = toolContext(async () => {
    publications++;
    throw new Error("transcript unavailable");
  });
  try {
    await assert.rejects(
      () => Promise.resolve(tool.run({ ...chartDocument, summary: "" }, ctx)),
      /ParseFailed|summary/,
    );
    assert.equal(publications, 0);
    await assert.rejects(() => Promise.resolve(tool.run(chartDocument, ctx)), /transcript unavailable/);
    assert.equal(publications, 1);
  } finally {
    await unload();
  }
});

test("v2 publication uses its own version and complete composition after validation", async () => {
  const fake = fakeApi(),
    unload = activate(fake.api);
  const document = richVisualDocumentSchema.parse({
    version: 2,
    title: "Current spend",
    summary: "Spend is 120 USD.",
    panels: [{ id: "spend", body: { kind: "metric", label: "Spend", value: 120 } }],
  });
  const calls: unknown[] = [];
  try {
    const tool = fake.tool();
    assert.ok(tool);
    assert.deepEqual(
      await tool.run(
        document,
        toolContext(async (value) => {
          calls.push(value);
          return { id: "v2" };
        }),
      ),
      { artifactId: "v2", published: true },
    );
    assert.deepEqual(calls, [
      { type: "visual", version: 2, title: document.title, summary: document.summary, data: document },
    ]);
  } finally {
    await unload();
  }
});

test("older hosts fail clearly before offering a broken tool", async () => {
  const fake = fakeApi(false);
  const unload = activate(fake.api);
  await tick();
  assert.equal(fake.active.size, 0);
  assert.ok(fake.notifications.some((message) => message.includes("Update the app before enabling Visuals")));
  await unload();
});

test("missing publication support fails clearly even when renderer registration exists", async () => {
  const fake = fakeApi();
  const unload = activate(fake.api);
  const tool = fake.tool();
  assert.ok(tool);
  const { artifacts, ...older } = toolContext(async () => ({ id: "unused" }));
  // This local fixture deliberately models an older host, not unchecked wire input.
  try {
    await assert.rejects(
      () => Promise.resolve(tool.run(chartDocument, older as ToolContext)),
      /newer Convergence host/,
    );
  } finally {
    await unload();
  }
});

test("manifest is official code, on by default, with tools.provide only and agent rules", () => {
  const manifest = JSON.parse(readFileSync(new URL("./plugin.json", import.meta.url), "utf8"));
  assert.equal(manifest.id, "convergence/visuals");
  assert.equal(manifest.main, "main.ts");
  assert.equal(manifest.disabledByDefault, undefined, "fresh installs start with Visuals on");
  assert.deepEqual(Object.keys(manifest.permissions), ["tools.provide"]);
  assert.deepEqual(manifest.contributes.rules, ["rules.md"]);
});

test("html_preview returns T3's metadata as text and structured content, and the PNG as an image part", async () => {
  const calls: unknown[] = [];
  const fake = fakeApi(true, {
    call: async (method, params) => {
      calls.push({ method, params });
      return {
        available: true,
        png: "iVBORw0KGgo=",
        width: 390,
        contentHeight: 512,
        capturedHeight: 512,
        consoleMessages: [{ level: "log", text: "ok" }],
      };
    },
  });
  const unload = activate(fake.api);
  try {
    const page = '<!doctype html><html><head></head><body><img src="/Users/me/shot.png"></body></html>';
    const result = (await fake.tool("html_preview")!.run(
      { html: page, width: 390, appearance: "light" },
      toolContext(async () => ({ id: "x" })),
    )) as ToolResult;
    const structured = {
      width: 390,
      contentHeight: 512,
      capturedHeight: 512,
      consoleMessages: [{ level: "log", text: "ok" }],
      missingImages: ["/Users/me/shot.png"],
      screenshot: { mimeType: "image/png", width: 390, height: 512 },
    };
    assert.deepEqual(result, {
      content: [
        { type: "text", text: JSON.stringify(structured) },
        { type: "image", data: "iVBORw0KGgo=", mimeType: "image/png" },
      ],
      structuredContent: structured,
    });
    const sent = calls[0] as { method: string; params: { html: string; width: number; appearance: string } };
    assert.equal(sent.method, "host/web.preview");
    assert.equal(sent.params.width, 390);
    assert.equal(sent.params.appearance, "light");
    assert.match(
      sent.params.html,
      /^<!doctype html><html><head><meta charset="utf-8"><meta name="viewport"[^>]*><style id="divergence-theme">:root\{color-scheme:light;--background:#f3f7fd;/,
    );
  } finally {
    await unload();
  }
});

test("html_preview without a confined browser is an error with no image, never a made-up screenshot", async () => {
  const fake = fakeApi(true, { call: async () => ({ available: false, reason: "killed by signal 6" }) });
  const unload = activate(fake.api);
  try {
    const result = (await fake.tool("html_preview")!.run(
      { html: "<p>x</p>" },
      toolContext(async () => ({ id: "x" })),
    )) as ToolResult;
    assert.equal(result.isError, true);
    assert.equal(result.content?.length, 1);
    assert.match(
      result.content![0]!.type === "text" ? result.content![0]!.text : "",
      /HtmlBackendUnavailable: .*killed by signal 6/,
    );
    const older = fakeApi();
    const unloadOlder = activate(older.api);
    const old = (await older.tool("html_preview")!.run(
      { html: "<p>x</p>" },
      toolContext(async () => ({ id: "x" })),
    )) as ToolResult;
    assert.match(old.content![0]!.type === "text" ? old.content![0]!.text : "", /newer Divergence app/);
    await unloadOlder();
  } finally {
    await unload();
  }
});

test("html_preview and html_render keep T3's input bounds", async () => {
  const fake = fakeApi(true, { call: async () => ({ available: false, reason: "n/a" }) });
  const unload = activate(fake.api);
  const ctx = toolContext(async () => ({ id: "x" }));
  try {
    const preview = fake.tool("html_preview")!;
    const render = fake.tool("html_render")!;
    await assert.rejects(() => Promise.resolve(preview.run({ html: "" }, ctx)), /ParseFailed|html/);
    await assert.rejects(() => Promise.resolve(preview.run({ html: "x".repeat(512_001) }, ctx)), /ParseFailed|html/);
    await assert.rejects(
      () => Promise.resolve(preview.run({ html: "x", appearance: "sepia" }, ctx)),
      /ParseFailed|appearance/,
    );
    await assert.rejects(
      () => Promise.resolve(render.run({ html: "x", title: "", height: 100 }, ctx)),
      /ParseFailed|title/,
    );
    await assert.rejects(
      () => Promise.resolve(render.run({ html: "x", title: "t".repeat(201), height: 100 }, ctx)),
      /ParseFailed|title/,
    );
    await assert.rejects(
      () => Promise.resolve(render.run({ html: "x", title: "T", height: 1.5 }, ctx)),
      /ParseFailed|height/,
    );
  } finally {
    await unload();
  }
});

test("html_render saves the agent's page unchanged with a clamped height and answers like T3", async () => {
  const fake = fakeApi();
  const unload = activate(fake.api);
  const publications: unknown[] = [];
  const ctx = toolContext(async (value) => {
    publications.push(value);
    return { id: "artifact-7" };
  });
  try {
    const page = "<!doctype html><script>let n = 0</script><button onclick='n++'>+</button>";
    const result = (await fake
      .tool("html_render")!
      .run({ html: page, title: "  Counter  ", height: 5000 }, ctx)) as ToolResult;
    const answer = {
      htmlRender: { attachmentId: "artifact-7", title: "Counter", height: 2000 },
      message:
        "Shown to the reader above your reply. Don't mention or describe the page; reply with only what it doesn't already say.",
    };
    assert.deepEqual(result, { content: [{ type: "text", text: JSON.stringify(answer) }], structuredContent: answer });
    assert.deepEqual(publications, [
      {
        type: "html",
        version: 1,
        title: "Counter",
        summary: "HTML page: Counter",
        data: { html: page, title: "Counter", height: 2000 },
      },
    ]);
  } finally {
    await unload();
  }
});

test("Open in panel shows the page in a detail-pane view tab, live with its source", async () => {
  const opened: unknown[] = [];
  // The shape of a service client: each method also answers `.result` with an outcome.
  const open = Object.assign(async () => ({ id: "t" }), {
    result: async (params: unknown) => {
      opened.push(params);
      return { ok: true, value: { id: "t" } };
    },
  });
  const tabs = { open };
  const fake = fakeApi(true, { tabs });
  const unload = activate(fake.api);
  await tick();
  try {
    const html = fake.renderers.find((r) => r.type === "html")!;
    const page = "<!doctype html><button>+</button>";
    const artifact = {
      owner: "visuals",
      type: "html",
      version: 1,
      title: "Counter",
      summary: "HTML page: Counter",
      data: { html: page, title: "Counter", height: 300 },
      item: "artifact-9",
    };
    const inline = html.render(artifact, { state: {}, app: {}, plugin: "visuals", update: () => {} }) as unknown;
    const doc = nodes(inline).find((n) => n.type === "web_document");
    assert.deepEqual(doc, { type: "web_document", id: "page", height: 300 });
    assert.ok(!nodes(inline).some((n) => n.type === "text"), "inline is borderless: no title header");
    const button = nodes(inline).find((n) => n.id === "html-open-panel");
    assert.equal(button?.type, "button");
    assert.equal(button?.tooltip, "Open in panel");
    (button!.onClick as () => void)();
    await tick();
    await tick();
    assert.deepEqual(fake.notifications, [], "the tabs plugin is there");
    assert.deepEqual(opened, [
      { kind: "view", workspaceId: "w1", path: "html/artifact-9", title: "Counter", icon: "code", view: 42 },
    ]);
    const ctx = { state: {}, app: {}, plugin: "visuals", update: () => {} } as ViewContext;
    const panel = fake.views.get("visuals-page")!(ctx);
    assert.deepEqual(
      nodes(panel).find((n) => n.type === "web_document"),
      { type: "web_document", id: "panel-artifact-9", height: 2000, item: "artifact-9" },
    );
    (nodes(panel).find((n) => n.id === "html-panel-source")!.onClick as () => void)();
    const source = fake.views.get("visuals-page")!(ctx);
    const code = nodes(source).find((n) => n.type === "code");
    assert.ok(code && JSON.stringify(code).includes("<button>+</button>"));
  } finally {
    await unload();
  }
});

test("a panel that cannot open says why, in a notice and in the panel", async () => {
  const open = Object.assign(async () => ({ id: "t" }), {
    result: async () => ({ ok: false, error: { _tag: "ServiceUnavailable", message: "the detail pane is closed" } }),
  });
  const fake = fakeApi(true, { tabs: { open } });
  const unload = activate(fake.api);
  await tick();
  try {
    const html = fake.renderers.find((r) => r.type === "html")!;
    const artifact = {
      owner: "visuals",
      type: "html",
      version: 1,
      title: "Counter",
      summary: "HTML page: Counter",
      data: { html: "<p>x</p>", title: "Counter", height: 120 },
      item: "artifact-3",
    };
    const inline = html.render(artifact, { state: {}, app: {}, plugin: "visuals", update: () => {} }) as unknown;
    (nodes(inline).find((n) => n.id === "html-open-panel")!.onClick as () => void)();
    for (let i = 0; i < 5; i++) await tick();
    assert.equal(fake.notifications.length, 1);
    assert.match(fake.notifications[0]!, /could not open in a panel: .*the detail pane is closed/);
    const panel = fake.views.get("visuals-page")!({
      state: {},
      app: {},
      plugin: "visuals",
      update: () => {},
    } as ViewContext);
    assert.match(JSON.stringify(panel), /the detail pane is closed/);
  } finally {
    await unload();
  }
});

Versions

VersionPublishedPlugin APISizePermissionsStatus
0.2.2latestNov 21, 2026>=2 <4349.5 KB1 permissionListed
0.2.1Oct 25, 2026>=2 <4349.2 KB1 permissionListed
0.2.0Oct 23, 2026>=2 <4349.3 KB1 permissionListed

Reviews and comments

0 threads · 0 reviews

No comments yet.