Computer use

Computer use lets an Agent use the desktop of the machine the runtime runs on the way it already uses the terminal and the browser: launch applications, read windows, click, type, press keys, and scroll. It is a tool panel, not a remote-desktop product and not a second session system. It is also the highest-risk tool: the Agent acts on your real desktop and real accounts, while you may be using the same machine. Vibex constrains it with an install gate, per-action approvals, hard denies, an emergency stop, and a redacted ledger.

​

Enable computer use

1. Open Settings → Computer use.
2. Click Install to install the desktop driver. The driver runs in its own helper process and owns the operating-system grants.
3. Grant the required permissions in the OS. macOS usually needs Accessibility and Screen Recording; some systems require a restart.
4. Turn on the Computer use master switch.

The switch requires an installed driver. When the driver is missing, the switch is unavailable and says what is missing. Sessions started after the switch is on receive the desktop tools; turning it off stops new sessions from receiving them.

​

Approvals and hard denies

Settings → Computer use → What needs approval offers three policies: Whatever the policy, these always require a per-action decision or are refused outright:

• Password managers, secure fields, and credential controls: hard deny, with no approval card.
• Vibex’s own windows and process: hard deny by default; only an explicit “operate Vibex’s own windows” setting allows it, for testing automation against Vibex itself.
• Destructive controls, file deletion, externalising data, clipboard reads, foreground takeover, and quitting an application: per-action approval, never remembered.
• Launching an application and clipboard writes: remembered for the session, but only for reversible, non-destructive actions.

“Remember for the session” is scoped to the application, not to the Agent. The strictest applicable class wins: an action that is both destructive and foreground is approved once. The approval card names the target application, action class, and reason; destructive and foreground actions also carry a screenshot of the target window. The screenshot is for the card and is never returned as tool content.

​

Panel and emergency stop

Open the panel from Settings → Computer use → Panel:

• Window: a separate window for longer observation.
• Floating: a small strip that stays above other windows where the platform allows it.

The panel shows what the Agent sees, the current action, and the redacted ledger. Its emergency stop:

1. refuses new desktop calls;
2. drops queued input that has not started;
3. releases every held key and mouse button;
4. stops the helper from accepting input;
5. records the stop and the actions that ran;
6. requires a human to re-enable it; the Agent cannot undo a stop.

Step 3 matters most: a stuck modifier key can make the whole machine unusable. When the client disconnects, new actions are refused and pending approvals are void. A longer disconnect clears the queue and releases held keys. Recovery requires your confirmation, through the same path as the emergency stop.

​

Platform support

On Linux the helper must not run as root in a user session. A headless macOS runtime does not start the helper either, because the grants belong to the application in the launch chain.

​

Security boundary

• Application identity is resolved by the runtime, never trusted from the model. An ambiguous name is an error, not a guess.
• Element references are short-lived: observe again after every action. A stale reference or changed window tree is refused.
• Results are graded: verified, unverified, and failed. Missing metadata means unverified, never success.
• The ledger is redacted: no screen pixels, typed text, clipboard contents, or credentials.
• The emergency stop outranks everything: only a human can re-enable it.

​

Troubleshooting