0
Fork 0
mirror of https://github.com/zeronsh/comet.git synced 2026-11-16 18:57:19 +00:00
Commit graph

3 commits

Author SHA1 Message Date
Wing
2b2c1e8df4 show_visual: hover must not move the page
The tool's description now tells agents that anything they show only
under the pointer (a custom readout, a detail line, a hint, a row's or
canvas's hover state) must not change a visual's height: reserve its space
with a placeholder or an empty line of the same height, or draw it where
it takes no layout space. ui.chart's own readout already keeps its place.
crates/visual checks the advice's example.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: zeronsh[bot] <346184242+zeronsh[bot]@users.noreply.github.com>
2026-11-11 00:45:17 -07:00
Wing
8c8535b7c1
Plugins and templates: consent bound to capabilities, engine-held decisions, hardened runtime (#1139)
* Plugins and templates: consent bound to capabilities, engine-held decisions, hardened runtime

- Consent binds a capability digest (programs, MCP servers, tools, network hosts, services, module replacement); it re-asks only when that grows. UI targets never ask.
- "You" is this device's key; code-running powers are never auto-granted; keys are revoked on sign-out and rotation, listed and revocable in Settings, and new keys are announced.
- Decisions and signing need a one-time review the consent window holds; agents get a scoped IPC token; agent-created chats can't exceed their creator's access; peers can't allow; relayed calls carry the edge-verified device.
- Runtime: text-only Luau, one command policy (env allow-list, hardened git/gh), plugin network blocks private ranges unless declared, safe path joins, verified snapshots, hardened built-in Git service.
- App: ui.change can't override modules, Settings sealed, decision controls protected, plugins get ids and resolve data through grants, consent sheet hardened, dev switches debug-only.
- Templates: unapproved templates run as previews with Keep/Revert, a host-owned way back on every platform, safe mode, no builtin-theme spoofing, increasing versions.
- Edge: step-up for key registration and account deletion, key list/revoke, reserved names, invitation/image/admin limits, relay device stamp.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: zeronsh[bot] <346184242+zeronsh[bot]@users.noreply.github.com>

* Signing keys register on the first sign-in, revocations act at once, setup and layout fixes

- A sign-in registers this device's key with its fresh ticket, also on a local-mode engine; new keys are noticed at start; a revoked own key rotates at once (peers are told), drops code-running grants and shows a notice.
- Calls during unfinished account setup answer setup_incomplete; plugins and templates load once an organization is chosen.
- The host strip's wrapper fills the window.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: zeronsh[bot] <346184242+zeronsh[bot]@users.noreply.github.com>

* Engine: only plugin and template calls answer setup_incomplete during onboarding; everything else waits as before

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: zeronsh[bot] <346184242+zeronsh[bot]@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: zeronsh[bot] <346184242+zeronsh[bot]@users.noreply.github.com>
2026-11-03 11:33:36 -07:00
Wing
8a0fded891
Native visuals: agents show interactive Luau visuals rendered by GPUI (#900)
* Render agent visuals natively from sandboxed Luau scripts

show_visual now takes a Luau script instead of an HTML page. The script
builds its data and state once and returns a view function whose ui.*
tree (layout, text, stats, tables, charts, tabs, toggles, sliders,
buttons, links, 2D canvases, 3D scenes) Zeron renders with its own GPUI
elements, so a visual scrolls, clips, blurs under the glass composer and
takes input like any transcript row on every platform.

zeron-visual is the GPUI-free runtime: a Luau sandbox with no I/O, a
memory cap and per-call time budgets, seeded randomness, strict props
whose style rules are the API (theme color tokens, the transcript's type
scale, no fills or borders), canvas drawing commands, and 3D scene
projection. The engine runs every script through check() before showing
it and returns line-numbered errors to the agent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Render visual scenes and shaders on the GPU

- Scenes rasterize with a depth buffer (no more painter's-algorithm
  occlusion errors), clip at the near plane, shade curved solids
  smoothly, and paint as one image; the software path renders in
  parallel row bands at full resolution.
- With a GPU (wgpu, new crate zeron-visual-gpu), scenes render with PBR
  materials (roughness, metal, glow, glass with ior and dispersion),
  soft shadows, MSAA, and bloom; the software renderer is the fallback.
- ui.shader runs Shadertoy-style GLSL on the GPU: iTime, iResolution,
  iMouse, iCamera (orbit), named inputs bound to state, iNoise, and up to
  four buffer passes read as iChannel0-3 with feedback. Composited by
  blend (light by default, so black is clear; alpha; opaque) with
  optional bloom. Each pass is validated with naga so errors reach the
  agent with their line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add visual assets and iOS visuals, and harden untrusted visual input

Assets: show_visual takes files (glTF models, PNG/JPEG/WebP/HDR images)
from the agent's working directory. The engine checks and stores them
beside the script; scenes gain s:model, textures, environment maps and
sprites, and shaders get iTexture0-3. Limits (16 files, 8 MB each, 10 MB
together) fit the MCP server's one WebSocket message to the engine.

Rendering: GPU scenes and shaders fade out toward their edges; plain
translucent surfaces blend unbent (only glass refracts) and, with sprites,
draw before glass so it refracts them; bloom composites correctly in
alpha mode; shader images sample upright as on Shadertoy.

iOS: each visual gets a Rust worker thread in zeron-mobile that runs the
script, lays the tree out, draws canvases and charts, and renders scenes
and shaders with wgpu; the transcript takes each visual's measured
height, and VisualView only paints what arrives and forwards touches.
The demo gains an "Agent visuals" chat (ZERON_DEMO_VISUALS can point it
at a desktop's visuals).

Hardening (security and code review):
- glTF: images stay encoded and are size-checked when drawn; accessor
  shapes are validated; node walks are bounded; parsing can't panic.
- Images: a pixel budget besides the side limit; textures larger than
  the GPU allows are refused.
- wgpu errors are caught or logged, never a panic; the texture cache is
  keyed by content, bounded, and decodes outside its lock.
- The engine checks for a running turn first and removes partial
  writes; asset names may not differ only by case; the MCP server reads
  only regular files, bounded.
- Desktop: released visuals free their GPU images; iTime and iMouse
  follow the docs; no second render when a drag ends.
- iOS: fonts are shared, not leaked per visual; offscreen and
  backgrounded visuals pause; commands see the tree the last one left;
  failed loads retry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Render desktop visual frames off the main thread and reuse GPU targets

Desktop scenes and shaders rendered and read back inside GPUI's paint,
blocking the main thread on the GPU every animated frame, so a heavy
agent shader could stall the app. Paint now does only the cheap parts
(input, uniforms, the scene's budgeted Lua draw) and hands the render
to a background thread, one frame in flight per visual; the view shows
the newest finished frame and repaints when the next lands.

Scenes also allocated ~70 MB of render targets every frame. They now
live in a SceneState kept per visual (desktop and iOS) and are remade
only when the size changes; scenes and shaders keep their readback
staging buffer too. The test room's steady frame went from ~8 ms to
~3 ms at 1440x720.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: wing-anara <wing@anara.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:04:16 -07:00