Visual Source Extract

Extract cited facts from images, scans, and video.

Skill metadata

Source	Bundled (installed by default)
Path	skills/research/visual-source-extract
Version	1.0.0
Author	VibeOS
License	MIT
Platforms	linux, macos, windows
Tags	research, vision, video, screenshots, pdf, citations
Related skills	professional-research, source-critique, choose-interaction-surface

Reference: full SKILL.md

信息

The following is the complete skill definition that VibeOS loads when this skill is triggered. This is what the agent sees as instructions when the skill is active.

Visual Source Extract Skill

Pull facts from screenshots, scanned PDFs, charts, and short videos — with explicit uncertainty. Prefer structured tools over guessing pixels.

When to Use

• Source is an image, chart, UI screenshot, scanned page, or short clip.
• User asks “what does this show?” for research / due diligence.
• Computer-use or browser captured a frame that must enter a cited brief.

Do not use for:

• Pure text already on disk → read_file.
• Live GUI control → computer_use / browser skills first, then this skill.

Prerequisites

• vision_analyze for stills (path or URL).
• video_analyze when the clip is the evidence (not a single keyframe).
• Optional: computer_use / browser_vision to capture the frame first.
• Helper: scripts/format_visual_facts.py to validate the fact card JSON.

How to Run

1. Capture or locate the media path/URL.
2. Ask the vision/video tool a fact-seeking question (numbers, labels, dates, UI text) — not “describe the vibe”.
3. Fill the fact card below. Mark each claim certain | likely | uncertain.
4. Validate JSON with the helper before merging into a research brief.

python3 scripts/format_visual_facts.py <<'EOF'
{
  "media": {"path": "/tmp/chart.png", "kind": "image"},
  "question": "What revenue figures and years are shown?",
  "facts": [
    {
      "claim": "Revenue 2024 = $12.4M",
      "confidence": "likely",
      "where": "top bar label",
      "needs_corroboration": true
    }
  ]
}
EOF

Fact card (per media item)

### Media: <filename or URL>
- Kind: image | scan | chart | ui | video
- Tool: vision_analyze | video_analyze | browser_vision
- Question asked: ...

| # | Claim | Confidence | Where on media | Needs corroboration |
|---|-------|------------|----------------|---------------------|
| 1 | ... | certain/likely/uncertain | ... | yes/no |

## Unreadable / uncertain
- ...

## Do not claim
- Anything not visibly supported (no invented axes, logos, or numbers).

Quick Reference

Media	Tool
Still image / chart / scan	vision_analyze
Short video	video_analyze
Live page pixels	browser_vision then cite the saved path
Native app window	computer_use capture → vision_analyze

Procedure

1. Prefer OCR-able stills over video when a single frame holds the claim.
2. For charts: read axis units, legend, and exact labels before summarizing.
3. For UI: quote visible strings; do not infer hidden state.
4. For scans: note blur/cutoff; lower confidence when text is partial.
5. Feed A/B claims into professional-research / source-critique — visual extracts are rarely primary alone.

Pitfalls

• Vision models invent plausible numbers — keep confidence honest.
• Do not treat a marketing screenshot as audited financials.
• Do not skip needs_corroboration for any numeric claim used in a brief.
• Keep prompt-cache safe: do not swap toolsets mid-session.

Verification

• Every claim has a confidence tag.
• Unreadable regions are listed (or “none”).
• No URL/title fabricated from the image alone.
• Helper script accepts the fact-card JSON (ok: true).