visuals
HTML pages (html_preview, html_render) and native inline charts, positioned diagrams and sortable tables in agent replies.
The app opens the listing; nothing installs until an agent in your Plugins workspace has read the files and you enable the plugin. In a terminal: dvg install divergence/visuals@0.2.2
Visuals — official plugin
divergence/visuals provides publish_visual for interactive, native visual replies. It ships enabled on a fresh install; an existing install keeps the choice it has, and users can turn it off or on in Settings → Plugins. It starts only when a reply first contains one of its fences and loads Pele only on the first Mermaid fence, so a chat without fences pays nothing. Its only requested permission is tools.provide.
HTML pages (T3 Code's html_preview and html_render)
html.ts holds T3's contract (pingdotgg/t3code b24f0fbb): the tool names, descriptions, limits (HTML 1–512000 UTF-16 units, title 1–200, preview width 240–1600 with 728 by default, frame height 80–2000), the result shapes and messages, and the theme bootstrap with Divergence's colours. Both tools are direct, so in code mode they stay tools of their own and the preview's PNG reaches the agent as an MCP image part.
html_previewsends the themed page tohost/web.preview. With a web view it answers T3's metadata as text and structured content plus the PNG. Without one it is an error,HtmlBackendUnavailable: <raw reason>, with no image, and no screenshot is made up.html_rendersaves the agent's page unchanged as ahtmlv1 transcript artifact (the host stamps the owner) and answers T3'shtmlRenderreference and message. Measuredheightsare not supplied: the frame follows the live page's own height, and the agent's height shows only until the page has measured itself.- Inline (
html-view.ts) the page is live (ui.webDocument): borderless in the reply, as wide as the column and as tall as the page, with one small glass Open in panel control. The panel is a detail-pane tab (thedivergence/tabsservice): the page live again, its Source, Copy and Save (kernel/copy_text,kernel/save_text, both after a click).
The renderer is the system's web view (crates/host/src/web): WKWebView on macOS, so a page behaves as it would in Safari, WebView2 on Windows, as in Edge, and WebKitGTK on Linux, which the app draws itself. The difference from T3 is that pages are offline: remote URLs and local image paths never load (missingImages lists the paths a page names). Where the web view cannot start, the page says why and its source, copy and save still work.
Version 2 supports responsive multi-panel compositions: bar (grouped/stacked/ percent; horizontal/vertical), line, area, scatter/bubble, pie/donut, heatmap, histogram, range/timeline, waterfall and boxplot, plus metrics, text, typed tables and positioned diagrams. document.ts defines the strict, bounded JSON schema; rules.md is the agent-facing reference. Version 1 documents still replay.
Rendering and interaction
Rust/GPUI draws every shape, label, control and floating tooltip. A pinned, DOM-free D3 math subset computes scales, numeric/UTC ticks and paths inside QuickJS; there is no browser, JavaScript UI execution, graph auto-layout, network, file, process or plugin-storage access. math.LICENSES.txt includes provenance and licenses; scripts/bundle-visuals-math.mjs reproduces the readable bundle.
Charts use the native canvas's onResize({width,height}) logical-pixel measurement, not a scaled viewBox. Text remains readable as panels grow or wrap. Compact legends keep original domains, colors and percentage denominators when hidden. Null breaks paths and stays explicitly missing in shared bar inspection.
Hover opens a clamped floating tooltip; click pins/unpins. Arrow keys, Home/End traverse observations, Enter/Space pin, Escape clears inspection. In the expanded overlay Escape unpins first, then closes. Tables retain stable raw row identity, sort nulls last in either direction, paginate and expose full selected values. Diagram node details remain accessible through a native picker.
State and cached geometry belong to each artifact's ctx.state, with separate panels and inline/expanded inspection. Hover does not recompute paths. Native resize reports are change-only and coalesced off-paint. No JavaScript frame loop or render-time notification runs.
Persistence and compatibility
The host owns immutable transcript artifacts. ctx.artifacts.publish persists the complete document independently of the surrounding tool result, including discarded acknowledgements and later code-mode errors. The plugin registers removable visual renderers for versions 1 and 2; Source retains original input and Data retains all raw values, including hidden series and nulls.
This plugin requires the updated native host and canvas resize API. Older hosts without artifact support fail with an explicit update message. Invalid persisted documents keep their readable summary and expose a validation error.
Verification
node --import ./plugins/testing/register.mjs --test plugins/visuals/*.test.ts
pnpm -C plugins typecheck
pnpm -C plugins exec prettier --check 'visuals/**/*.{ts,json,md}'
Geometry tests use asymmetric data and independent numeric expectations; view tests exercise real native element factories and handlers. Native desktop checks and app-owned Metal captures supplement these tests; they do not use a browser.
Permissions in 0.2.2
Files
// The only accepted visual format: bounded, inert JSON, never a UI tree or executable source.
// Validate at publication and again when reading persisted artifacts across plugin updates.
import * as z from "zod";
export const LIMITS = {
label: 120,
title: 160,
summary: 2000,
magnitude: 1e12,
coordinate: 10_000,
series: 8,
values: 1000,
nodes: 100,
edges: 200,
rows: 200,
columns: 20,
} as const;
const meaningful = (value: string) => /[\p{L}\p{N}]/u.test(value);
const label = z.string().min(1).max(LIMITS.label).refine(meaningful, "Use a meaningful plain-text label");
const identifier = z
.string()
.min(1)
.max(LIMITS.label)
.regex(/^[a-zA-Z0-9_-]+$/);
const number = z.number().min(-LIMITS.magnitude).max(LIMITS.magnitude);
const coordinate = z.number().min(0).max(LIMITS.coordinate);
const dimension = z.number().min(1).max(LIMITS.coordinate);
export const chartSchema = z
.strictObject({
kind: z.literal("chart"),
chart: z.enum(["bar", "line"]),
categories: z.array(label).min(1).max(LIMITS.values),
series: z
.array(z.strictObject({ name: label, values: z.array(number.nullable()).min(1).max(LIMITS.values) }))
.min(1)
.max(LIMITS.series),
yLabel: label.optional(),
})
.superRefine((body, ctx) => {
let count = 0;
body.series.forEach((series, index) => {
count += series.values.length;
if (series.values.length !== body.categories.length)
ctx.addIssue({ code: "custom", path: ["series", index, "values"], message: "Match the categories length" });
});
if (count > LIMITS.values)
ctx.addIssue({ code: "custom", path: ["series"], message: "At most 1000 values across all series" });
});
export const diagramSchema = z
.strictObject({
kind: z.literal("diagram"),
width: dimension,
height: dimension,
nodes: z
.array(
z.strictObject({
id: identifier,
label,
x: coordinate,
y: coordinate,
width: dimension,
height: dimension,
detail: z.string().min(1).max(LIMITS.summary).optional(),
}),
)
.min(1)
.max(LIMITS.nodes),
edges: z.array(z.strictObject({ from: identifier, to: identifier, label: label.optional() })).max(LIMITS.edges),
})
.superRefine((body, ctx) => {
const ids = new Set<string>();
body.nodes.forEach((node, index) => {
if (ids.has(node.id))
ctx.addIssue({ code: "custom", path: ["nodes", index, "id"], message: "Node ids must be unique" });
ids.add(node.id);
if (node.x + node.width > body.width || node.y + node.height > body.height)
ctx.addIssue({ code: "custom", path: ["nodes", index], message: "Keep the whole node inside the diagram" });
});
body.edges.forEach((edge, index) => {
if (!ids.has(edge.from) || !ids.has(edge.to))
ctx.addIssue({ code: "custom", path: ["edges", index], message: "Edges must reference existing nodes" });
});
});
const cell = z.union([z.string().max(LIMITS.label), number, z.boolean(), z.null()]);
export const tableSchema = z
.strictObject({
kind: z.literal("table"),
columns: z
.array(z.strictObject({ key: identifier, label, type: z.enum(["text", "number", "boolean"]) }))
.min(1)
.max(LIMITS.columns),
rows: z.array(z.array(cell).min(1).max(LIMITS.columns)).max(LIMITS.rows),
})
.superRefine((body, ctx) => {
const keys = new Set<string>();
body.columns.forEach((column, index) => {
if (keys.has(column.key))
ctx.addIssue({ code: "custom", path: ["columns", index, "key"], message: "Column keys must be unique" });
keys.add(column.key);
});
body.rows.forEach((row, index) => {
if (row.length !== body.columns.length)
ctx.addIssue({ code: "custom", path: ["rows", index], message: "Match the columns length" });
body.columns.forEach((column, columnIndex) => {
const value = row[columnIndex];
const expected = column.type === "text" ? "string" : column.type;
if (value !== null && typeof value !== expected)
ctx.addIssue({
code: "custom",
path: ["rows", index, columnIndex],
message: `Expected ${column.type} or null`,
});
});
});
});
export const legacyVisualDocumentSchema = z.strictObject({
version: z.literal(1),
title: z.string().min(1).max(LIMITS.title).refine(meaningful, "Use a meaningful title"),
summary: z.string().min(1).max(LIMITS.summary).refine(meaningful, "Explain the visual in plain text"),
body: z.discriminatedUnion("kind", [chartSchema, diagramSchema, tableSchema]),
});
export type LegacyVisualDocument = z.infer<typeof legacyVisualDocumentSchema>;
export type Chart = z.infer<typeof chartSchema>;
export type Diagram = z.infer<typeof diagramSchema>;
export type Table = z.infer<typeof tableSchema>;
export type Cell = z.infer<typeof cell>;
// V2 describes analytical intent, not executable UI. Every panel shares the
// publication and native geometry budgets; composition cannot multiply them.
export const DOCUMENT_LIMITS = { panels: 12, observations: 4000, cells: 8000, commands: 8000, pathPoints: 40000 } as const;
export const numberFormatSchema = z
.strictObject({
style: z.enum(["number", "compact", "percent", "currency"]).default("number"),
precision: z.number().int().min(0).max(6).optional(),
currency: z
.string()
.regex(/^[A-Z]{3}$/)
.optional(),
unit: label.optional(),
})
.superRefine((format, ctx) => {
if (format.style === "currency" && !format.currency)
ctx.addIssue({ code: "custom", path: ["currency"], message: "Supply the three-letter currency code" });
if (format.style !== "currency" && format.currency)
ctx.addIssue({ code: "custom", path: ["currency"], message: "Currency belongs to currency formatting" });
});
const time = z.number().int().min(-8_640_000_000_000_000).max(8_640_000_000_000_000);
const axis = z.discriminatedUnion("type", [
z.strictObject({ type: z.literal("category"), values: z.array(label).min(1).max(1000) }),
z.strictObject({ type: z.literal("number"), values: z.array(number).min(1).max(1000) }),
z.strictObject({ type: z.literal("time"), values: z.array(time).min(1).max(1000) }),
]);
const series = z
.array(z.strictObject({ name: label, values: z.array(number.nullable()).min(1).max(1000) }))
.min(1)
.max(8);
const chartOptions = {
kind: z.literal("chart"),
xLabel: label.optional(),
yLabel: label.optional(),
format: numberFormatSchema.optional(),
};
const references = {
references: z
.array(z.strictObject({ value: number, label }))
.max(8)
.optional(),
bands: z
.array(z.strictObject({ from: number, to: number, label }))
.max(8)
.optional(),
};
const stack = z.enum(["none", "stacked", "percent"]).default("none");
const bar = z.strictObject({
...chartOptions,
...references,
chart: z.literal("bar"),
x: z.strictObject({ type: z.literal("category"), values: z.array(label).min(1).max(1000) }),
series,
orientation: z.enum(["vertical", "horizontal"]).default("vertical"),
stack,
});
const line = z.strictObject({
...chartOptions,
...references,
chart: z.literal("line"),
x: axis,
series,
interpolation: z.enum(["linear", "step"]).default("linear"),
});
const area = z.strictObject({
...chartOptions,
...references,
chart: z.literal("area"),
x: axis,
series,
interpolation: z.enum(["linear", "step"]).default("linear"),
stack,
});
const scatter = z.strictObject({
...chartOptions,
chart: z.literal("scatter"),
xFormat: numberFormatSchema.optional(),
sizeLabel: label.optional(),
series: z
.array(
z.strictObject({
name: label,
points: z
.array(z.strictObject({ x: number, y: number, size: number.min(0).optional(), label: label.optional() }))
.min(1)
.max(1000),
}),
)
.min(1)
.max(8),
});
const parts = z
.array(z.strictObject({ label, value: number.min(0) }))
.min(1)
.max(24);
const pie = z.strictObject({
kind: z.literal("chart"),
chart: z.literal("pie"),
parts,
format: numberFormatSchema.optional(),
});
const donut = z.strictObject({
kind: z.literal("chart"),
chart: z.literal("donut"),
parts,
format: numberFormatSchema.optional(),
centerLabel: label.optional(),
});
const heatmap = z.strictObject({
...chartOptions,
chart: z.literal("heatmap"),
x: z.array(label).min(1).max(64),
y: z.array(label).min(1).max(64),
cells: z
.array(z.strictObject({ x: z.number().int().min(0), y: z.number().int().min(0), value: number.nullable() }))
.min(1)
.max(1600),
scale: z.enum(["sequential", "diverging"]).default("sequential"),
});
const histogram = z.strictObject({
...chartOptions,
chart: z.literal("histogram"),
bins: z
.array(z.strictObject({ from: number, to: number, count: number.min(0) }))
.min(1)
.max(200),
});
const range = z.strictObject({
kind: z.literal("chart"),
chart: z.literal("range"),
axis: z.enum(["number", "time"]).default("number"),
xLabel: label.optional(),
format: numberFormatSchema.optional(),
items: z
.array(
z.strictObject({
label,
start: z.number().min(-8_640_000_000_000_000).max(8_640_000_000_000_000),
end: z.number().min(-8_640_000_000_000_000).max(8_640_000_000_000_000),
lane: label.optional(),
}),
)
.min(1)
.max(200),
});
const waterfall = z.strictObject({
...chartOptions,
chart: z.literal("waterfall"),
entries: z
.array(z.strictObject({ label, value: number, role: z.enum(["delta", "total"]) }))
.min(1)
.max(200),
});
const box = z.strictObject({
...chartOptions,
chart: z.literal("box"),
items: z
.array(
z.strictObject({
label,
min: number,
q1: number,
median: number,
q3: number,
max: number,
outliers: z.array(number).max(100).optional(),
}),
)
.min(1)
.max(100),
});
export const richChartSchema = z
.discriminatedUnion("chart", [bar, line, area, scatter, pie, donut, heatmap, histogram, range, waterfall, box])
.superRefine((chart, ctx) => {
const issue = (message: string, path: (string | number)[] = []) => ctx.addIssue({ code: "custom", path, message });
if (chart.chart === "bar" || chart.chart === "line" || chart.chart === "area") {
chart.series.forEach((series, i) => {
if (series.values.length !== chart.x.values.length) issue("Match the x values length", ["series", i, "values"]);
if (
chart.chart !== "line" &&
chart.stack === "percent" &&
series.values.some((value) => value !== null && value < 0)
)
issue("Percent stacks require nonnegative values", ["series", i, "values"]);
});
if (chart.x.type !== "category") {
const values = chart.x.values;
if (values.some((value, i) => i > 0 && value <= values[i - 1]!))
issue("Numeric and UTC-time x values must be strictly increasing; duplicate x values are not allowed", [
"x",
"values",
]);
}
chart.bands?.forEach((band, i) => {
if (band.from >= band.to) issue("A reference band starts before it ends", ["bands", i]);
});
if (chart.chart !== "line" && chart.stack === "percent" && (chart.references?.length || chart.bands?.length))
issue("Percent stacks use a 0–100% axis; omit raw-value reference lines and bands");
}
if (chart.chart === "pie" || chart.chart === "donut") {
if (new Set(chart.parts.map((part) => part.label)).size !== chart.parts.length)
issue("Part labels must be unique", ["parts"]);
}
if (chart.chart === "heatmap") {
const seen = new Set<string>();
chart.cells.forEach((cell, i) => {
if (cell.x >= chart.x.length || cell.y >= chart.y.length)
issue("Cells reference existing axis labels", ["cells", i]);
const key = `${cell.x}:${cell.y}`;
if (seen.has(key)) issue("Each heatmap coordinate appears once", ["cells", i]);
seen.add(key);
});
}
if (chart.chart === "histogram") {
chart.bins.forEach((bin, i) => {
if (bin.from >= bin.to) issue("A bin starts before it ends", ["bins", i]);
if (i > 0 && bin.from < chart.bins[i - 1]!.to) issue("Bins must be ordered and cannot overlap", ["bins", i]);
});
}
if (chart.chart === "range") {
chart.items.forEach((item, i) => {
if (item.start >= item.end) issue("An interval starts before it ends", ["items", i]);
if (chart.axis === "time" && (!Number.isInteger(item.start) || !Number.isInteger(item.end)))
issue("UTC-time coordinates are integer epoch milliseconds", ["items", i]);
if (
chart.axis === "number" &&
(Math.abs(item.start) > LIMITS.magnitude || Math.abs(item.end) > LIMITS.magnitude)
)
issue("Numeric interval coordinates are bounded to ±1e12", ["items", i]);
});
}
if (chart.chart === "box") {
chart.items.forEach((item, i) => {
if (!(item.min <= item.q1 && item.q1 <= item.median && item.median <= item.q3 && item.q3 <= item.max))
issue("Supply an ordered five-number summary: min ≤ q1 ≤ median ≤ q3 ≤ max", ["items", i]);
});
}
});
export type RichChart = z.infer<typeof richChartSchema>;
export type NumberFormat = z.infer<typeof numberFormatSchema>;
const metric = z.strictObject({
kind: z.literal("metric"),
label,
value: number,
format: numberFormatSchema.optional(),
comparison: z
.strictObject({
label,
value: number,
format: numberFormatSchema.optional(),
tone: z.enum(["neutral", "positive", "negative"]).default("neutral"),
})
.optional(),
});
const text = z.strictObject({ kind: z.literal("text"), text: z.string().min(1).max(LIMITS.summary) });
export const visualPanelSchema = z.strictObject({
id: identifier,
title: label.optional(),
caption: z.string().min(1).max(LIMITS.summary).optional(),
span: z.union([z.literal(1), z.literal(2)]).default(1),
body: z.union([richChartSchema, tableSchema, diagramSchema, metric, text]),
});
export type VisualPanel = z.infer<typeof visualPanelSchema>;
// Conservative draw-cost bounds reserve room for axes, controls, tooltip,
// and the simultaneously retained inline + expanded native trees.
export function panelCost(panel: VisualPanel) {
const body = panel.body;
let observations = 0,
cells = 0,
commands = 80,
pathPoints = 0,
nodes = 0,
edges = 0;
if (body.kind === "table") cells = body.rows.length * body.columns.length;
else if (body.kind === "diagram") {
nodes = body.nodes.length;
edges = body.edges.length;
commands += nodes * 2 + edges * 3;
pathPoints += edges * 12;
} else if (body.kind === "chart") {
switch (body.chart) {
case "bar":
case "line":
case "area":
observations = body.series.reduce((count, series) => count + series.values.length, 0);
commands += observations * 2 + body.series.length * 8;
// Each reference/band adds one command to both retained scenes, beyond
// the base reserve for complete nice-tick sequences and inspection.
commands += ((body.references?.length ?? 0) + (body.bands?.length ?? 0)) * 2;
pathPoints += observations * 8;
break;
case "scatter":
observations = body.series.reduce((count, series) => count + series.points.length, 0);
commands += observations * 2;
pathPoints += observations * 16;
break;
case "pie":
case "donut":
observations = body.parts.length;
commands += observations * 4;
pathPoints += observations * 40;
break;
case "heatmap":
observations = body.cells.length;
commands += observations * 2;
break;
case "histogram":
observations = body.bins.length;
commands += observations * 2;
break;
case "range":
observations = body.items.length;
commands += observations * 3;
break;
case "waterfall":
observations = body.entries.length;
commands += observations * 3;
pathPoints += observations * 2;
break;
case "box":
observations = body.items.reduce((count, item) => count + 5 + (item.outliers?.length ?? 0), 0);
commands += observations * 3;
pathPoints += observations * 16;
break;
}
}
return { observations, cells, commands, pathPoints, nodes, edges };
}
export const richVisualDocumentSchema = z
.strictObject({
version: z.literal(2),
title: z.string().min(1).max(LIMITS.title).refine(meaningful, "Use a meaningful title"),
summary: z.string().min(1).max(LIMITS.summary).refine(meaningful, "Explain the visual in plain text"),
columns: z.union([z.literal(1), z.literal(2)]).default(1),
panels: z.array(visualPanelSchema).min(1).max(DOCUMENT_LIMITS.panels),
})
.superRefine((document, ctx) => {
if (new Set(document.panels.map((panel) => panel.id)).size !== document.panels.length)
ctx.addIssue({ code: "custom", path: ["panels"], message: "Panel ids must be unique" });
const costs = document.panels.map(panelCost);
for (const [key, limit] of Object.entries({ ...DOCUMENT_LIMITS, nodes: LIMITS.nodes, edges: LIMITS.edges })) {
if (key === "panels") continue;
const total = costs.reduce((sum, cost) => sum + cost[key as keyof typeof cost], 0);
if (total > limit)
ctx.addIssue({
code: "custom",
path: ["panels"],
message: `The whole document exceeds the ${key} budget (${total}/${limit}); reduce samples or panels`,
});
}
});
export const visualDocumentSchema = z.discriminatedUnion("version", [
legacyVisualDocumentSchema,
richVisualDocumentSchema,
]);
export type RichVisualDocument = z.infer<typeof richVisualDocumentSchema>;
export type VisualDocument = z.infer<typeof visualDocumentSchema>;
No comments yet.