Skip to main content
Use OS-level controls to move and click the mouse, type and press keys, scroll, drag, and capture screenshots from a running browser session.
Computer control actions are only available for browsers created after October 15, 2025. Older browser instances do not support these features.

​
Click the mouse

Simulate mouse clicks at specific coordinates. You can select the button, click type (down, up, click), number of clicks, and optional modifier keys to hold. 
import Kernel from '@onkernel/sdk';

const kernel = new Kernel();
const kernelBrowser = await kernel.browsers.create();

// Basic left click at (100, 200)
await kernel.browsers.computer.clickMouse(kernelBrowser.session_id, {
  x: 100,
  y: 200,
});

// Double right-click while holding Shift
await kernel.browsers.computer.clickMouse(kernelBrowser.session_id, {
  x: 100,
  y: 200,
  button: 'right',
  click_type: 'click',
  num_clicks: 2,
  hold_keys: ['Shift'],
});

​
Move the mouse

Move the cursor to specific screen coordinates. Optionally hold modifier keys during the move. 
import Kernel from '@onkernel/sdk';

const kernel = new Kernel();
const kernelBrowser = await kernel.browsers.create();

await kernel.browsers.computer.moveMouse(kernelBrowser.session_id, {
  x: 500,
  y: 300,
  hold_keys: ['Alt'],
});

​
Take screenshots

Capture a full-screen PNG or a specific region. 
import fs from 'fs';
import { Buffer } from 'buffer';
import Kernel from '@onkernel/sdk';

const kernel = new Kernel();
const kernelBrowser = await kernel.browsers.create();

// Full screenshot
{
  const response = await kernel.browsers.computer.captureScreenshot(kernelBrowser.session_id);
  const blob = await response.blob();
  const buffer = Buffer.from(await blob.arrayBuffer());
  fs.writeFileSync('screenshot.png', buffer);
}

// Region screenshot
{
  const response = await kernel.browsers.computer.captureScreenshot(kernelBrowser.session_id, {
    region: { x: 0, y: 0, width: 800, height: 600 },
  });
  const blob = await response.blob();
  const buffer = Buffer.from(await blob.arrayBuffer());
  fs.writeFileSync('region.png', buffer);
}

​
Type text

Type literal text, optionally with a delay in milliseconds between keystrokes. 
import Kernel from '@onkernel/sdk';

const kernel = new Kernel();
const kernelBrowser = await kernel.browsers.create();

await kernel.browsers.computer.typeText(kernelBrowser.session_id, {
  text: 'Hello, World!',
});

await kernel.browsers.computer.typeText(kernelBrowser.session_id, {
  text: 'Slow typing...',
  delay: 100,
});

​
Press keys

Press one or more key symbols (including combinations like “Ctrl+t” or “Ctrl+Shift+Tab”). Optionally hold modifiers and/or set a duration to hold keys down. 
import Kernel from '@onkernel/sdk';

const kernel = new Kernel();
const kernelBrowser = await kernel.browsers.create();

// Tap a key combination
await kernel.browsers.computer.pressKey(kernelBrowser.session_id, {
  keys: ['Ctrl+t'],
});

// Hold keys for 250ms while also holding Alt
await kernel.browsers.computer.pressKey(kernelBrowser.session_id, {
  keys: ['Ctrl+Shift+Tab'],
  duration: 250,
  hold_keys: ['Alt'],
});

​
Scroll

Scroll the mouse wheel at a specific position. Positive delta_y scrolls down; negative scrolls up. Positive delta_x scrolls right; negative scrolls left. 
import Kernel from '@onkernel/sdk';

const kernel = new Kernel();
const kernelBrowser = await kernel.browsers.create();

await kernel.browsers.computer.scroll(kernelBrowser.session_id, {
  x: 300,
  y: 400,
  delta_x: 0,
  delta_y: 120,
});

​
Drag the mouse

Drag by pressing a button, moving along a path of points, then releasing. You can control delay before starting, the granularity and speed of the drag via steps_per_segment and step_delay_ms, and optionally hold modifier keys. 
import Kernel from '@onkernel/sdk';

const kernel = new Kernel();
const kernelBrowser = await kernel.browsers.create();

await kernel.browsers.computer.dragMouse(kernelBrowser.session_id, {
  path: [
    [100, 200],
    [150, 220],
    [200, 260],
  ],
  button: 'left',
  delay: 0,
  steps_per_segment: 10,
  step_delay_ms: 50,
  hold_keys: ['Shift'],
});