Перевод: оболочка страницы на русском; тело навыка (SKILL.md) пока на английском — это исходные инструкции агента.

Desktop Operate

Run app-control tasks with self-heal and scorecard.

Метаданные навыка

Источник	Встроенный (установлен по умолчанию)
Путь	skills/desktop/desktop-operate
Версия	1.0.0
Автор	VibeOS
Лицензия	MIT
Платформы	macos, windows, linux
Теги	operate, computer-use, desktop, self-heal, scorecard
Связанные навыки	choose-interaction-surface, computer-use, macos-app-playbooks, windows-app-playbooks, linux-app-playbooks, challenge-gate, raster-edit, video-nle, figma-edit

Справка: полный SKILL.md

к сведению

Ниже полное определение навыка, которое VibeOS загружает при активации. Это инструкции, которые видит агент.

Desktop Operate Skill

Orchestrate Operate mode: pick a surface, drive a native app (or browser when appropriate), self-heal with re-capture, and stop cleanly within ≤2 retries before human escalate. Write an honest scorecard.

This skill does not replace OS playbooks or creative editor skills — it routes to them and enforces reliability rules.

When to Use

• User wants the agent to complete a concrete GUI task in a desktop app.
• Creative editor handoffs (raster-edit, video-nle, figma-edit GUI path).
• Office / Finder / Preview style tasks via OS app playbooks.

Do not use for deep web research (Investigate) or greenfield media generation without an editor (Create via creative-studio tools first).

Prerequisites

• vibeos computer-use doctor green enough for the OS (permissions granted)
• Load choose-interaction-surface before acting
• Purchases, Send mail, delete, overwrite masters → human gate

How to Run

1. Write operate-brief.md from templates/operate-brief.md.
2. Choose surface; announce it; open the target app.
3. Loop: capture → act by element → verify (re-capture).
4. On failure: one alternate-surface attempt, then escalate (≤2 tries total).
5. Fill scorecard.md + qa.md; run scripts/eval_operate.py.

Quick Reference

Step	Rule
Surface	MCP/API/terminal > browser_* > computer_use
Action	Element index from SoM/AX — no blind pixels
Retry budget	2 stuck attempts then human
CAPTCHA	challenge-gate (never CU-first on Chrome puzzles)
Destructive	Stop and ask

Procedure

1. Brief

Copy templates/operate-brief.md. One goal, one primary app, success check.

2. Self-heal policy

capture → act → verify (capture_after / re-capture)
  ├─ success → next step
  ├─ empty tree / wrong window → refocus app, re-capture (counts as retry)
  ├─ wrong surface → switch once (browser↔CU / MCP note)
  └─ still stuck after 2 tries → clarify / ask user; partial scorecard

Never burn the budget on the same blind click.

3. Scorecard

Update scorecard.md (template in templates/scorecard.md):

• Goal met? (yes / partial / no)
• Attempts used (1–2)
• Escalate reason (if any)
• Artifacts paths

4. Creative editors

Still polish → raster-edit. NLE → video-nle. Figma → figma-edit (MCP first).

Pitfalls

• Do not thrash permissions dialogs — point the user at vibeos computer-use permissions grant.
• Do not claim success without a verify capture or on-disk artifact.
• Messaging platforms may disable computer_use — say so and stop.

Verification

• operate-brief.md filled; scorecard.md Status is honest.
• Artifact or screenshot proof when the brief requires it.
• Offline eval: python scripts/eval_operate.py --workspace <dir>.
• CLI: vibeos operate run|doctor|preset desktop-operate.
• Smoke: scripts/smoke-desktop-operate.sh.