Desktop Operate

Operate mode for VibeOS: brief → choose surface → capture/act/verify → scorecard. Prefer MCP / terminal / browser_* when they fit; use computer_use for native apps. Cap stuck GUI attempts at 2, then escalate to a human. CAPTCHA → ChallengeGate.

Quick path

# Permissions / driver
vibeos computer-use doctor

# Pack recipe
vibeos operate preset desktop-operate

# Readiness (skills + computer_use probe)
vibeos operate doctor

# Scaffold a workspace
vibeos operate run --title "Open Downloads in Finder"

# Offline smoke
scripts/smoke-desktop-operate.sh

Workspace layout

Under ~/.vibeos/operate/<id>/:

File	Role
operate-brief.md	Goal, app, surface, max retries
prompt.md	Session prompt for the agent
scorecard.md	Status, attempts, escalate, evidence
artifacts/	Screenshots / exports
qa.md	Extra notes

Scorecard eval

python skills/desktop/desktop-operate/scripts/eval_operate.py \
  --workspace ~/.vibeos/operate/<id>

# Or latest scaffold:
vibeos operate scorecard

Checks: brief filled, Status not pending, Attempts ≤ 2, Escalate field set.

Skills

Load desktop-operate plus:

Need	Skill
Surface routing	choose-interaction-surface
Capture / click	computer-use (+ OS desktop skill)
macOS Finder/Notes/Mail/Preview	macos-app-playbooks
Photoshop / Affinity / Photopea	raster-edit
NLE / Figma	video-nle / figma-edit
Bot walls	challenge-gate

Self-heal policy

1. Capture (SoM/AX) → act by element index → verify (re-capture).
2. Empty tree / wrong window → refocus once (counts as a retry).
3. Wrong surface → switch once (browser ↔ CU).
4. Still stuck after 2 tries → ask the user; mark scorecard partial/fail.

Purchases, Send, delete, overwrite masters → human gate before acting.

Durable board

Seed a linked Operate pipeline on Kanban:

vibeos kanban template apply operate --slug desk-ops --switch
vibeos kanban list

Golden tasks (offline + manual): docs/plans/operate-golden-tasks.yaml.

Related

• Creative Studio (Create mode)
• ChallengeGate / CAPTCHA
• Professional research (Investigate mode)
• Kanban