How to Let Claude Watch a Video (crv)

TL;DR + what you need

claude-real-video (crv) is an open-source, MIT-licensed CLI — plus a local web UI — that converts a video URL or file into scene-aware keyframes and a transcript, so an LLM that can only read images and text — every current LLM, Claude included — can actually reason about what happened in the video.

• One command: pip install claude-real-video, then crv <url-or-file>.
• One system dependency: ffmpeg, not pip-installable, required for extraction and audio.
• One output folder: crv-out/frames/*.jpg + crv-out/transcript.txt + crv-out/MANIFEST.txt.
• Three ways to run it: the crv CLI, an agent skill (Claude Code and others), or crv-web with zero terminal.

The project’s own worked example is the number to anchor on: a 58-second clip sampled at a fixed 1 fps produces 58 frames. crv’s scene-change detection plus dedup keeps the 26 that actually differ, and --grid packs those into 3 contact sheets. Fewer tokens into whichever LLM you use, nothing visually distinct left out.

The tool moved fast in its first ten days — a Hacker News front-page thread, a community pull request merged the same day it landed, and a full rewrite of the install story. The maintainer, HUANGCHIHHUNG (@LeoAidoAI), summed up one week of it like this:

Why Claude can’t just watch a video

Precision matters here, because the gap is easy to overstate. Claude can read images — that part already works fine. What it can’t do, on any current surface, is accept a video file or a video URL and step through it frame by frame the way a person would. Ask it to summarize a YouTube link directly and it either declines or free-associates from the title and any cached knowledge of the video, not from the footage itself. crv doesn’t teach Claude to understand video; it converts the video into the image-and-text format Claude already understands.

Three status-quo failure modes, as the project frames them:

• ChatGPT reads the transcript when you paste a YouTube link — not the picture.
• Claude won’t take a video file at all.
• Gemini does ingest video natively, but has to send it to Google’s servers and samples at a fixed interval (1 fps by default), so fast cuts slip past and static screens get oversampled.

If all you actually need is words, MCP.Directory already catalogs transcript-only tooling — our YouTube Transcript server entry fetches and analyzes a video’s transcript specifically “for video content analysis without watching,” by its own listed description. That is precisely the failure mode crv is built to go beyond: a transcript tells you what was said, not what was on screen.

Takeaway: crv is not a smarter model. It is a translation layer — video in, images and text out — aimed at the one input format none of today’s mainstream chat LLMs accept directly.

The pipeline: five pieces, one folder

Everything crv does is one chronological pass through six named stages, run locally, ending in a folder any LLM can read:

“Extract” is a single ffmpeg select filter pass, deliberately — the tool’s own source notes that keeping frames in strict time order in one pass matters, because dedup needs to compare true chronological neighbours, and two separate passes used to interleave frames out of order. “Dedup” compares real pixel difference on downscaled RGB, not a perceptual hash — hashes go blind on flat colours and equal-luma hue changes, per the project’s own comments in the code.

Takeaway: nothing here is a black box of ML models. It is classical video processing — ffmpeg, pixel diffing, optional Whisper — chained in a fixed order, which is also why the whole thing installs in seconds and needs no GPU.

Smallest end-to-end example

pip install claude-real-video              # core: frames + dedup
pip install "claude-real-video[whisper]"   # + audio transcription

# ffmpeg is required and NOT pip-installable
brew install ffmpeg          # macOS
sudo apt install ffmpeg      # Linux
winget install Gyan.FFmpeg   # Windows

Then point it at a link or a local file:

crv "https://www.youtube.com/watch?v=..." --grid

A run against that same 58-second worked example, following the CLI’s own reporting format, looks like this:

✓ Done → crv-out
  26 frames  (deduped from 58 extracted)  in crv-out/frames
  manifest:   crv-out/MANIFEST.txt
  transcript: crv-out/transcript.txt
  grids:      3 contact sheet(s) in crv-out/grids

Drop the frames (or the contact sheets, which read as a sequence instead of scattered stills) plus MANIFEST.txt into Claude, ChatGPT, or Gemini, and ask your question. It also works as a plain Python call, no CLI involved:

from claude_real_video import process

r = process("https://youtu.be/...", "out", lang="en")
print(r.frame_count, r.transcript_path)

Takeaway: the entire surface is one positional argument (source) and a handful of optional flags. There is no config file, no account, no API key — the only thing gating a first run is having ffmpeg on your PATH.

Deep dive: scene-change detection vs fixed sampling

Most “let an LLM watch a video” scripts — and Gemini’s own pipeline — grab a frame every N seconds. That over-samples a static screencast and under-samples a fast-cut reel. crv instead fires on ffmpeg’s scene-change score, with a --fps-floor (default 1.0s) guaranteeing at least one frame even during a scene that never technically “changes.”

Two flags exist because “scene change” is a fuzzier signal than it sounds: --adaptive compares each frame to its rolling neighbourhood instead of a fixed constant, catching a 2–3 second squash-and-stretch that never spikes hard enough to trip a static threshold; --text-anchors forces extra frames at subtitle-cue timestamps, for content where the meaning changes faster than the pixels do (slides, burned-in captions). Both shipped in direct response to real user reports — more on that in Common Mistakes.

Takeaway: scene-aware extraction beats fixed sampling for anything with cuts, and the two extra modes exist precisely because “scene change” alone doesn’t cover slow motion or static-but-changing content. Default to plain --scene; reach for --adaptive or --text-anchors only when a first pass visibly misses something.

Deep dive: sliding-window dedup

Extraction alone would still resend a shot every time an A-B-A cutaway returns to it — two people talking, cut back and forth, is the classic case. crv’s dedup step compares each newly extracted frame against the last --dedup-window frames it already kept (default 4), not just the single previous frame, so a shot the model has already seen doesn’t reappear after a brief cutaway. Set it to 1 for classic consecutive-only comparison.

--dedup-threshold (default 8) sets what percentage of pixels must change for a frame to count as new — raise it and you keep fewer, more different frames. If you want to see the tool’s reasoning rather than trust it blindly, --report keeps every dropped frame in ./dropped and writes a report.html showing each keep/drop decision with its diff percentage — genuinely useful for tuning the threshold on footage that behaves unusually.

Takeaway: dedup, not extraction, is what makes the headline number (58 frames → 26 kept) work. Extraction alone over-collects on purpose; dedup is the step actually doing the token-saving.

Deep dive: transcript — subtitles first, Whisper as fallback

If a video already has subtitles — a sidecar .srt/.vtt next to a local file, or an embedded subtitle track — crv uses those directly. That’s both faster and more accurate than re-transcribing audio that already has an authoritative text version. Only when there are no subtitles does it fall back to Whisper, skipped cleanly if the video has no audio track at all.

--lang controls Whisper’s language (default auto-detect); --whisper-model picks the model size (tiny/base/small/medium/large, default base). The bundled Claude Code skill is explicit about the cost of the default: the first transcription downloads an openai-whisper base model, about 139 MB, automatically, once. Bigger models trade a larger download and a slower run for better accuracy on tricky audio.

Takeaway: the subtitles-first order is the detail worth remembering — if you’re debugging a transcript that looks wrong, check whether the source actually shipped a (possibly bad) subtitle track before assuming Whisper mis-heard the audio.

Three ways to run it

crv ships three interfaces, and picking the right one matters more than any individual flag.

As an agent skill

The README’s own path installs the bundled skill straight into Claude Code:

pip install claude-real-video
mkdir -p ~/.claude/skills && cp -r skills/claude-real-video ~/.claude/skills/

After that, paste a video link into Claude Code and ask about it — no manual crv invocation. The skill’s own instructions define a tight four-step loop for the agent: run the extractor with --grid, read MANIFEST.txt first, read the contact sheets in grids/ before falling back to individual frames, then answer citing timestamps from the manifest.

A second, community-contributed path goes wider: a merged pull request added a separate skills/claude-real-video-for-agents bundle plus an install-skill.sh script that auto-detects and symlinks the skill into seven agent platforms at once — Claude Code, Codex, Gemini CLI, OpenCode, Pi, MiMoCode, and a generic .agents convention. It was opened and merged the same day, one of several PRs the maintainer folded in within hours of a report landing.

crv-web — no terminal

Running crv-web starts a local page — stdlib-only Python, no extra dependencies — at http://127.0.0.1:8642 and opens it in your browser automatically. Paste a YouTube or Reels link (or a local file path), tick --adaptive/--text-anchors/--grid as checkboxes instead of flags, click Analyze, and open the result viewer when the background job finishes. The interface ships in three languages — Traditional Chinese, Simplified Chinese, and English — toggled in the corner and persisted in localStorage.

Takeaway: the skill path is for standing, repeatable use inside an agent loop; crv-web is for a one-off “what’s in this clip” without opening a terminal at all. Both call the exact same extraction pipeline underneath.

What we got wrong

Researching this from the outside, in the order reality corrected us:

• We assumed “runs locally” meant fully private end to end. It means the extraction, dedup, and transcription pipeline runs on your machine — nothing is uploaded by the tool itself. The moment you paste the resulting frames into Claude, ChatGPT, or Gemini, that provider’s ordinary data handling applies, same as pasting any screenshot. A Hacker News commenter made exactly this point at launch, and a reply narrowed it correctly: pair crv with a model that also runs on your machine if you need the whole chain to stay local.
• We assumed the project was a CLI, full stop. It shipped three interfaces — CLI, a portable agent skill across seven platforms, and a three-language local web UI — within roughly its first ten days.
• We assumed “Claude can’t watch video” meant Claude has no vision. It has vision; it reads images fine. What it lacks is a way to ingest a raw video file or stream directly. That precision is the whole reason crv exists as a translation layer rather than a model upgrade — exactly this confusion surfaced on the launch thread, where one commenter’s video-analysis experience with Claude prompted another to ask what “can’t view video” even meant.

Real-world workflows

Takeaway: crv is a preprocessing step, not a chat feature. The workflow that works is always “extract locally, then paste,” never “paste the raw link and hope.”

Common mistakes

• Assuming ffmpeg comes bundled. Root cause: pip install succeeds fine because it only installs the Python package; ffmpeg/ffprobe are native binaries pip can’t install. The first crv run then fails until you brew install ffmpeg (macOS) or your distro’s equivalent.
• Expecting scene detection to catch slow, gradual motion. Root cause: a user’s bug report showed a 2–3 second squash-and-stretch animation technique slipping past detection — a gradual change never spikes any single frame’s score high enough. Fixed by --adaptive, which compares each frame to its rolling neighbourhood instead of a fixed threshold.
• Losing sync on burned-in captions or slide decks. Root cause: a feature request pointed out that scene detection watches pixels, not meaning — a slide whose bullet points change without the background changing doesn’t register as a new scene. --text-anchors now forces a frame at each subtitle-cue timestamp instead, capped at one forced frame per second so it can’t flood the output.
• Treating ffmpeg’s scene filter as gospel. Root cause: ffmpeg’s built-in scene-change heuristic isn’t perfectly reliable on every kind of footage, a limitation raised directly on the launch thread — exactly why --report, --dedup-threshold, --dedup-window, and --adaptive all exist as tuning knobs rather than a black box you have to trust blindly.
• Expecting the free tier to explain camera work. Root cause: a stack of keyframes literally cannot show pans, zooms, or gesture timing between frames. That is the exact gap the paid crv Pro add-on targets with a separate motion-analysis pass; the free tool only ever promises what is on screen, never how it moves.

Limits, cost, and privacy

• ffmpeg is a hard requirement, not bundled. Budget one manual install step per machine before the first run.
• First transcription downloads a model. The default Whisper base model is roughly 139 MB, fetched automatically once; larger --whisper-model choices download more and run slower in exchange for accuracy.
• Token cost is downstream, not crv’s to set. The tool doesn’t call any LLM API itself, so there is no single “cost per video” figure — it depends entirely on which LLM you paste the output into and how many frames you send. --max-frames (default 150), --grid, and --dedup-threshold are your levers; the 58-to-26-to-3-sheets example shows the kind of reduction to expect.
• Login-gated sources need your own, authorized access. --cookies and --cookies-from-browser (Chrome, Safari, Firefox, Edge) exist for content you already have the right to fetch — the project is explicit that this is not a paywall bypass and that you should never ship a cookie file in a repo.
• Free core, paid depth layer. The extraction/dedup/transcript pipeline is MIT-licensed and free forever. A separate one-time paid tier, crv Pro, adds camera-move classification and pacing analysis on top — entirely optional, and not required for the workflows in this guide.

Who it’s for, who it isn’t

Good fit if you want Claude, ChatGPT, or any LLM to reason over a specific video’s actual visual content without routing through Gemini’s native (and cloud-only) video ingestion; if you use Claude Code or a similar agent and want video-watching as a standing skill rather than a one-off script; if you run repeatable video-to-notes workflows and want --kb to file results straight into a knowledge base; or if you want a paste-a-link experience with no terminal at all.

Look elsewhere if you need real-time or live video — crv processes a finished file or download, not a stream. If you specifically need camera moves, editing rhythm, or gesture timing out of the box, that is the paid crv Pro layer’s job, not the free tool’s. And if your video is dense, fast, and long, budget time to tune --max-frames, --scene, and --adaptive rather than expecting perfect defaults on the first run.

Community signal

The Hacker News launch thread, which reached the front page, split cleanly into three camps. Builders working on similar problems showed up first — several commenters mentioned shipping comparable tools of their own and offered to contribute, and the maintainer answered directly with what needed help: “the codebase is small enough to read in one sitting… additional transcript backends and smarter grid packing” were the areas named.

The precision skeptics raised the sharpest technical objection:

A second thread of skepticism was about cost, not correctness:

Both objections are fair and neither is really a rebuttal: crv doesn’t claim frames beat native video ingestion on cost or that stills capture motion — it claims Claude and ChatGPT have no native video path at all, so “expensive but possible” beats “impossible.” A separate subthread pushed back on the “Claude” branding itself, and the maintainer shipped a same-day fix rather than argue: “Took this — pip install llm-real-video works now, same tool. Kept the original repo name so existing links don’t break,” confirmed live on PyPI as an alias package under the neutral name.

The maintainer also cross-posted a launch recap to Reddit, where the framing was the same numbers, aimed at builders rather than the HN crowd:

The verdict

What we’d watch next: whether --adaptive and --text-anchors — both shipped in direct response to real bug reports within days — keep arriving at that pace, and whether the free/Pro split holds as a clean line or drifts. A project shipping three interfaces and two detection modes inside its first ten days is either about to stabilize or about to sprawl; the issue tracker is the tell.

The bigger picture: generating video vs understanding it

MCP.Directory covers a lot of video tooling, and it is worth being precise about which side of the video story each piece sits on. Our Runway MCP guide, the Claude Remotion skill, and our MiniMax, Pika, and Higgsfield coverage are all about generating video — text or images in, new footage out. crv does the opposite: existing footage in, structured understanding out. Confusing the two leads to a category error — no amount of scene-change tuning will make a frame extractor produce new video, and no video-generation tool will tell you what actually happened in a clip someone handed you.

crv is not itself an MCP server — it is a CLI and an agent skill, installed with pip rather than registered as an MCP connection. But it solves the same category of problem the Model Context Protocol exists to solve: giving a model reliable access to something it couldn’t reach on its own. MCP standardizes how a model calls out to a tool or data source over a live connection; crv solves a narrower, offline version of the same problem — turning a format the model can’t read into one it can, once, before the conversation starts.

Takeaway: if you are choosing a video tool for your stack, ask “generate or understand” first. It is a five-second question that saves you from installing the wrong category of tool entirely.

Frequently Asked Questions

Glossary

Sources

• Primary — repository, README, and source (cli.py, core.py, serve.py, viewer.py): github.com/HUANGCHIHHUNGLeo/claude-real-video (MIT, Python)
• Primary — packages: claude-real-video on PyPI · llm-real-video (alias) on PyPI
• Primary — agent skill definitions: skills/claude-real-video/SKILL.md · skills/claude-real-video-for-agents/SKILL.md in the repository above
• Community — Hacker News: “Claude-real-video — any LLM can watch a video”
• Community — Reddit: r/SideProject launch recap
• Community — maintainer on X: @LeoAidoAI, July 7, 2026 · @LeoAidoAI, July 8, 2026
• Community — issue tracker referenced above: text-aware frame forcing (issue #5) · multi-agent skill install PR (#4) · slow-motion detection report (issue #2) · Whisper engine doc fix (PR #1), all in the repository above
• Optional paid add-on referenced for completeness: crv Pro (not required for anything covered in this guide)
• Canonical MCP.Directory entry for the transcript-only alternative discussed above: /servers/youtube-transcript
• Related guides: What is MCP? · Runway MCP · Claude Remotion skill