visuals
HTML pages (html_preview, html_render) and native inline charts, positioned diagrams and sortable tables in agent replies.
The app opens the listing; nothing installs until an agent in your Plugins workspace has read the files and you enable the plugin. In a terminal: dvg install convergence/visuals@0.2.0
Visuals — official plugin
convergence/visuals provides publish_visual for interactive, native visual
replies. It ships enabled on a fresh install; an existing install keeps the
choice it has, and users can turn it off or on in Settings → Plugins. It starts
only when a reply first contains one of its fences and loads Pele only on the
first Mermaid fence, so a chat without fences pays nothing. Its only requested
permission is tools.provide.
HTML pages (T3 Code's html_preview and html_render)
html.ts holds T3's contract (pingdotgg/t3code b24f0fbb): the tool names,
descriptions, limits (HTML 1–512000 UTF-16 units, title 1–200, preview width
240–1600 with 728 by default, frame height 80–2000), the result shapes and
messages, the frame-height rule, and the theme bootstrap with Divergence's
colours. Both tools are direct, so in code mode they stay tools of their
own and the preview's PNG reaches the agent as an MCP image part.
html_previewsends the themed page tohost/web.preview. With a confined renderer it answers T3's metadata as text and structured content plus the PNG. Without one it is an error,HtmlBackendUnavailable: <raw reason>, with no image: nothing renders unconfined and no screenshot is made up.html_rendersaves the agent's page unchanged as ahtmlv1 transcript artifact (the host stamps the owner) and answers T3'shtmlRenderreference and message. Measuredheightsare not supplied.- Inline (
html-view.ts) the page is live (ui.webDocument): borderless in the reply, as wide as the column and as tall as the page up to the agent's height, with one small glass Open in panel control. The panel is a detail-pane tab (theconvergence/tabsservice): the page live again, its Source, Copy and Save (kernel/copy_text,kernel/save_text, both after a click).
The renderer is the host's: Debian's Chromium in a disposable container of
the local Docker engine, admitted through chrome://sandbox
(crates/host/src/web). Differences from T3 and from a system web view:
pages are offline (remote URLs and local image paths never load;
missingImages lists the paths a page names), fonts and text metrics are
Linux's (DejaVu, Liberation, Noto Color Emoji), there is no GPU, and native
browser menus, pickers, IME composition, accessibility, clipboard and drag
from the page are not available. Without Docker or the built renderer image
the page says why and its source, copy and save still work.
Version 2 supports responsive multi-panel compositions: bar (grouped/stacked/
percent; horizontal/vertical), line, area, scatter/bubble, pie/donut, heatmap,
histogram, range/timeline, waterfall and boxplot, plus metrics, text, typed tables
and positioned diagrams. document.ts defines the strict, bounded JSON schema;
rules.md is the agent-facing reference. Version 1 documents still replay.
Rendering and interaction
Rust/GPUI draws every shape, label, control and floating tooltip. A pinned,
DOM-free D3 math subset computes scales, numeric/UTC ticks and paths inside
QuickJS; there is no browser, JavaScript UI execution, graph auto-layout, network,
file, process or plugin-storage access. math.LICENSES.txt includes provenance
and licenses; scripts/bundle-visuals-math.mjs reproduces the readable bundle.
Charts use the native canvas's onResize({width,height}) logical-pixel measurement,
not a scaled viewBox. Text remains readable as panels grow or wrap. Compact
legends keep original domains, colors and percentage denominators when hidden.
Null breaks paths and stays explicitly missing in shared bar inspection.
Hover opens a clamped floating tooltip; click pins/unpins. Arrow keys, Home/End traverse observations, Enter/Space pin, Escape clears inspection. In the expanded overlay Escape unpins first, then closes. Tables retain stable raw row identity, sort nulls last in either direction, paginate and expose full selected values. Diagram node details remain accessible through a native picker.
State and cached geometry belong to each artifact's ctx.state, with separate
panels and inline/expanded inspection. Hover does not recompute paths. Native
resize reports are change-only and coalesced off-paint. No JavaScript frame loop
or render-time notification runs.
Persistence and compatibility
The host owns immutable transcript artifacts. ctx.artifacts.publish persists
the complete document independently of the surrounding tool result, including
discarded acknowledgements and later code-mode errors. The plugin registers
removable visual renderers for versions 1 and 2; Source retains original input
and Data retains all raw values, including hidden series and nulls.
This plugin requires the updated native host and canvas resize API. Older hosts without artifact support fail with an explicit update message. Invalid persisted documents keep their readable summary and expose a validation error.
Verification
node --import ./plugins/testing/register.mjs --test plugins/visuals/*.test.ts
pnpm -C plugins typecheck
pnpm -C plugins exec prettier --check 'visuals/**/*.{ts,json,md}'
Geometry tests use asymmetric data and independent numeric expectations; view tests exercise real native element factories and handlers. Native desktop checks and app-owned Metal captures supplement these tests; they do not use a browser.
Permissions in 0.2.0
Files
# HTML pages
When a chart, table, diagram, image collage, or mockup would say more than prose, build a self-contained HTML page, check it with `html_preview`, then publish it with `html_render` before your final reply. The reader sees the page above that reply, so don't announce or restate it; add only what it doesn't say.
By default, make pages look like part of the reply rather than a dashboard: a short title and one-line takeaway, key figures as plain large numbers with a muted label, a chart with its legend in its header row, and a plain table. By default, avoid stacked or nested cards, outlines around every section, glows and gradients; colour goes to data and status, through the theme variables. When the user asks for a specific look (a dashboard, a card layout, a mockup of a particular screen or brand), build that instead; the defaults are for when no direction is given. Reach for an interactive layout (a selectable list beside a detail panel, tabs, filters) only when selecting or filtering does real work, and make every control change what is shown. Label sample data as illustrative. Keep type quiet by default: medium-weight (500) headings, a title about 18px and section headings about 14-15px, natural letter spacing, no spaced uppercase labels. When a filter narrows the content, make the takeaway and figures follow it or label them as overall, show the active scope, and give a no-match state a way to clear the filter. Preview at the default width and at width 360 with each caption on its own line under its heading rather than pushed to the right edge, and preview a copy that drives the page's own controls (including a filter with no match) before publishing. Give `html_render` a height of at least the previewed contentHeight.
Pages are offline in Divergence: remote URLs (CDN libraries, web fonts, images) and local file paths do not load. The app's own faces are served offline as `--font-sans` (Inter) and `--font-mono` (JetBrains Mono): inherit them so the page is set like the app. Inline every script and style, and embed images as `data:` URLs. If `html_preview` reports that no confined browser is available, it returns no screenshot; `html_render` still saves the page, and the reader can open its source.
# Charts in a reply
For one chart that belongs at a point in your reply, write a `chart` fence
there: prose, then the chart, then what it shows. It draws natively where it
stands; no tool call is needed. The fence holds one JSON object:
```chart
{
"version": 1,
"kind": "bar",
"title": "Revenue by month (USD)",
"labels": ["Jul", "Aug", "Sep"],
"series": [{ "name": "Revenue", "values": [18000, 22000, 27000] }]
}
```
- `kind`: `line`, `area`, `bar`, `pie`, `donut`, `radar`, `candlestick`,
`sankey`, `scatter`, `heatmap`, `range`, `waterfall` or `boxplot`.
`labels` are the categories (x values, bars, slices, radar spokes, sankey
nodes, scatter points, heatmap columns), up to 500.
- `series`: up to 5 of `{ name, values }`, one number per label. `bar`,
`pie` and `donut` take exactly one series; `line`, `area` and `radar`
take several and show a legend. `"horizontal": true` lays bars sideways.
- `scatter`: `"x": [...]`, one number per label, and up to 5 series of y
values.
- `heatmap`: each series is a row (its `name` labels it), up to 30, with one
value per column in `labels`.
- `waterfall`: one series of changes; each bar starts where the last ended.
- `range`: `"ranges": [[low, high], ...]`, one per label.
- `boxplot`: `"boxes": [[min, q1, median, q3, max], ...]`, one per label.
- `candlestick`: `"candles": [[open, high, low, close], ...]`, one per label.
- `sankey`: `"links": [{ "source": 0, "target": 1, "value": 45 }, ...]`,
where source and target are indexes into `labels`.
- Optional: `title` (put the unit in it), `height` (120 to 600 pixels).
No other fields are allowed. A fence that does not match shows its source and
the error instead of the chart. Markdown tables draw natively too; use one for
exact figures. For several panels, metrics or a chart family not listed here,
use `publish_visual` below.
For a flowchart, sequence, class, state, ER or gantt diagram, write a
`mermaid` fence with ordinary Mermaid syntax. It draws natively in the app's
colours; a diagram that does not parse shows its error with the source.
# Native visual replies
Use `publish_visual` when an interactive visual makes your reply easier to
understand. The renderer draws a calm report: the title and summary lead (write
the summary as the one-sentence takeaway), metric panels line up as a row of
figures, and charts, tables and diagrams follow without boxes. Put metrics
first, then the one or two charts or tables that carry the point. Prefer **version 2**: a complete, inert JSON document with a meaningful
title, explanatory summary and 1–12 panels. No HTML, CSS, SVG, script, UI trees,
event handlers, external resources or links are executed. Rendering is native.
```json
{
"version": 2,
"title": "Revenue and channel mix",
"summary": "Revenue rose from 18k to 27k USD. Enterprise contributes 60% of the current total.",
"columns": 2,
"panels": [
{
"id": "revenue",
"title": "Monthly revenue",
"body": {
"kind": "chart",
"chart": "area",
"x": { "type": "category", "values": ["Jul", "Aug", "Sep"] },
"series": [{ "name": "Revenue", "values": [18000, 22000, 27000] }],
"format": { "style": "compact", "unit": "USD" }
}
},
{
"id": "mix",
"title": "Current channel mix",
"body": {
"kind": "chart",
"chart": "donut",
"centerLabel": "Total USD",
"parts": [{ "label": "Enterprise", "value": 16200 }, { "label": "Self-serve", "value": 10800 }],
"format": { "style": "compact" }
}
}
]
}
```
## Panels and composition
Each panel is `{id,title?,caption?,span?:1|2,body}`. Use unique identifier ids.
`columns` is 1 (default) or 2; two columns wrap to one in narrow panes.
`span:2` takes a whole row. Give related panels clear titles and units. A panel
body is one of the following:
- **Chart:** the families below. Hover tooltips, click-to-pin, keyboard inspection,
legends and stable original scales are automatic; do not write interaction code.
- **Metric:** `{kind:"metric",label,value,format?,comparison?:{label,value,format?,tone?:"neutral"|"positive"|"negative"}}`.
Comparison is a separate raw value with a label, not an automatically calculated delta.
- **Text:** `{kind:"text",text}`; plain text, not markup.
- **Table:** `{kind:"table",columns:[{key,label,type:"text"|"number"|"boolean"}],rows:[[...]]}`.
Unique column keys; positional rows match types/width. Null is missing.
Maximum 200 rows and 20 columns per table; sorting, paging and selection are native.
- **Positioned diagram:** `{kind:"diagram",width,height,nodes,edges}`.
Nodes are `{id,label,x,y,width,height,detail?}`; edges are `{from,to,label?}`.
Position nodes explicitly inside dimensions 1–10000. Unique node ids, existing
edge endpoints, at most 100 nodes/200 edges across the document. No auto-layout.
## Chart families
All charts have `kind:"chart"`. Use actual raw observations, never fabricated
samples, normalized values or statistics you did not compute. Omit optional
fields when unnecessary. `format` is `{style:"number"|"compact"|"percent"|"currency",precision?,currency?,unit?}`;
currency requires a three-letter code; percent formats a fraction (0.2 → 20%).
- **Bar:** `chart:"bar",x:{type:"category",values:[labels]},series:[{name,values:[number|null]}]`.
Optional `orientation:"vertical"|"horizontal"`, `stack:"none"|"stacked"|"percent"`.
Bars include zero. Percent stacks require nonnegative raw values; the renderer
calculates shares using the original total. Zero total is undefined, not 100%.
- **Line / area:** `chart:"line"|"area",x:{type:"category"|"number"|"time",values:[...]},series:[{name,values:[number|null]}]`.
Numeric and UTC epoch-millisecond x values must be strictly increasing.
Null breaks paths. Optional `interpolation:"linear"|"step"`; area also accepts
`stack:"none"|"stacked"|"percent"`. Every series matches the x-array length.
- Bar/line/area optionally accept `xLabel`, `yLabel`, `references:[{value,label}]`
and `bands:[{from,to,label}]` (up to 8 each). Bands start before they end.
References/bands use raw values and are not accepted on percent stacks.
- **Scatter / bubbles:** `chart:"scatter",series:[{name,points:[{x,y,size?,label?}]}]`.
Nonnegative `size` controls bubble **area**, not radius. Optional `xLabel`,
`yLabel`, `sizeLabel`, `xFormat`, `format`. Use at most 8 series.
- **Pie / donut:** `chart:"pie"|"donut",parts:[{label,value}]`.
Unique labels, nonnegative values, up to 24 parts. Donut optionally accepts
`centerLabel`. Hiding parts does not change original percentages.
- **Heatmap:** `chart:"heatmap",x:[labels],y:[labels],cells:[{x:index,y:index,value:number|null}]`.
Each coordinate occurs once and references existing labels. Optional
`scale:"sequential"|"diverging"`, `xLabel`, `yLabel`, `format`.
- **Histogram:** `chart:"histogram",bins:[{from,to,count}]`.
Supply computed nonnegative counts in ordered, nonoverlapping bins. Each bin
is `[from,to)`; gaps are preserved. Optional `xLabel`, `yLabel`, `format`.
- **Range / timeline:** `chart:"range",axis:"number"|"time",items:[{label,start,end,lane?}]`.
Intervals start before they end; UTC time uses integer epoch milliseconds.
Only explicit equal `lane` values group intervals. Optional `xLabel`, `format`.
- **Waterfall:** `chart:"waterfall",entries:[{label,value,role:"delta"|"total"}]`.
Deltas change the running value; totals are absolute values that reset it.
Optional `xLabel`, `yLabel`, `format`.
- **Boxplot:** `chart:"box",items:[{label,min,q1,median,q3,max,outliers?:[numbers]}]`.
Supply ordered five-number summaries (`min ≤ q1 ≤ median ≤ q3 ≤ max`) and
actual outliers. Optional `xLabel`, `yLabel`, `format`.
## Bounds, durability and replay
Objects are strict; extra properties fail validation. Finite numeric observations
are bounded to ±1e12. Labels/text cells max 120 characters, title 160, required
summary 2000. The whole document has budgets of 4000 observations, 8000 table
cells, 8000 estimated drawing commands and 40000 path points. Reduce samples or
panels if validation reports a budget error; composition does not bypass limits.
Version 1 remains supported for saved replies: `{version:1,title,summary,body}`
with legacy bar/line `{kind:"chart",chart,categories,series,yLabel?}`, table or
positioned diagram. New replies should use version 2.
Visuals appear inline outside collapsed tool groups. Expand offers **Visual**,
**Data** (complete raw values per panel) and **Source** (original JSON). Null
means missing, never zero. Do not repeat the dataset as prose or place special
artifact markup in an assistant message.
`publish_visual` returns `{artifactId,published:true}` only after durable host
publication. An enclosing `execute` need not return this acknowledgement; even a
later execute error does not retract it. Do not publish again to recover an id.
No comments yet.