Guides

Desktop & browser

Boxes from the desktop template come with a Linux desktop and Chromium. An agent can see the screen and drive mouse and keyboard over the SDK; you can watch along, take over, and hand it back.

Watch and take over

Open a box in the dashboard and choose the Screen tab. You see what the agent sees, live. Click into the desktop to take control — the agent’s input is held while you have it — and press Hand back when you’re done, or just leave it: control returns to the agent after a minute without input.

Let an agent drive it

TypeScript · @moil/sdk 
import { Box } from '@moil/sdk'

const box = await Box.create({ template: 'desktop' })
await box.desktop.start({ width: 1280, height: 800 })

await box.desktop.open('https://example.com')   // opens Chromium
const png = await box.desktop.screenshot()      // PNG bytes for a vision model
await box.desktop.click(640, 400)               // { button: 'right' }, { double: true }
await box.desktop.type('moil.dev')
await box.desktop.press(['ctrl', 'l'])          // one key or a chord
await box.desktop.scroll({ direction: 'down', amount: 5 })

Also: move(x, y), drag(from, to), cursor() and status(). Calling start() again is safe; it leaves a running desktop alone.

A minimal agent loop

for (let step = 0; step < 25; step++) {
  const png = await box.desktop.screenshot()
  const action = await model.nextAction(png)    // your model decides
  if (action.type === 'done') break
  if (action.type === 'click') await box.desktop.click(action.x, action.y)
  if (action.type === 'type') await box.desktop.type(action.text)
  if (action.type === 'key') await box.desktop.press(action.key)
}

Chromium keeps its profile under /data, so sign-ins survive pauses, restarts and snapshots — sign in once by hand, save the box, and every agent that starts from it is already logged in.

Share a link to the screen

const { url, expiresAt } = await box.desktop.link({
  control: false,     // the default: watch only
  ttlSeconds: 3600,   // 60 seconds to 24 hours
})
<iframe src="LINK_URL" width="1280" height="800" style="border:0"></iframe>
  • One box only. A link opens that box’s desktop and nothing else; it is not an API key.
  • Watch-only is enforced on our side. Keyboard, mouse and clipboard input from a watch-only link never reach the box.
  • Expiry stops new viewers. Someone already watching stays connected until they leave.
  • Revoking ends everything. It invalidates every link for the box and closes open viewers within seconds. Deleting the box does the same.

From the CLI

moil desktop screenshot run-42 > shot.png
moil desktop link run-42 --ttl 1h            # watch-only link
moil desktop link run-42 --control --ttl 10m # a link that can click and type
moil desktop revoke run-42                   # every link stops working

Desktops for your own users

Keep your API key on your server. When a user opens their desktop, make a fresh short-lived link for their box and send them that; revoke when their access ends. Tag boxes with a label such as customer=acme so you can find each user’s box again.