Smart crop: how the bot tells apart text, faces, and a cursor

In the Vertix interface, smart crop looks like a single toggle — "Smart" or "Center." But behind that one toggle sit several different cropping strategies, and the bot decides on its own which one applies based on what's actually happening in the frame. Blind face-tracking works great for a talking head and terribly for a screen recording with a mouse cursor — so there's no single recipe that covers everything.

First: what kind of video is this

Before deciding how to crop anything, the bot classifies the source footage by content: a talking head, a screen recording, a podcast with two stable faces in frame, or something that doesn't fit any of those categories. Everything downstream depends on that result — the cropping strategy adapts to the content, not the other way around.

Talking head — the frame tracks the face

The obvious case: someone talking on camera. This is where smart crop lives up to its name — a vertical 9:16 window tracks the face and keeps it in frame even as the person moves or drifts from their starting position in the horizontal source.

Screencast — you can't crop to a face that isn't there

A screen recording is its own case, where ordinary face tracking is meaningless: there's either no face at all, or it's a tiny webcam bubble in a corner. What matters here is what's on the screen — text, UI, the cursor. Cropping that down to a narrow 9:16 sliver would reliably chop off part of the text or interface off the edge of the frame.

So for a screencast, the bot doesn't narrow the frame to a vertical strip at all — it builds a window roughly 55% of the source screen's width that tracks the cursor, over a blurred full-frame background of the same video. The cursor is where a viewer's attention actually goes on a screen recording, so the frame follows it, not the geometric center.

A two-person podcast — not a crop, a split screen

When exactly two faces sit stably in the horizontal frame — the classic podcast layout — smart crop doesn't try to guess who to follow and doesn't jump back and forth between the two speakers. Instead it switches to a static split screen: the frame divides into two vertical panels, one face in each. That's steadier and more predictable than trying to guess who's talking at any given moment, and it avoids the jittery-camera effect on the cut.

When the video doesn't fit any category

For content that couldn't be confidently classified — not clearly a talking head, a screen, or a podcast — there's a safe fallback: a full, blurred-background frame instead of an aggressive crop made on a guess. That's more honest than pretending the algorithm is confident when it actually isn't.

Why it's one setting, not five

From the user's side, all of this stays a single "Smart crop" toggle — you don't need to know in advance what kind of footage you have and pick a matching mode by hand. Classification happens automatically before cropping, which is exactly why adding a new content type to this logic has never required a new setting in the interface — just a new rule inside the same "smart" mode.