Ask a video model for "the firefly girl hides under a leaf" and you get a different firefly girl every time. Ask an image model the same thing while showing it her reference sheet, and you get her. Then ask a video model to animate that image, and she moves. That is the whole idea behind the scene pipeline we shipped with Story to film, and it is the single most effective consistency tool we have.

Why text-to-video forgets

A text prompt describes a distribution, not an individual. "A small firefly girl with translucent wings" matches thousands of drawings, and each call samples a different one. Video models make it worse: they also have to keep the character stable across frames, and they trade fidelity to any reference for motion quality. Without engineering, thirty independent text-to-video calls give you thirty slightly different heroines and a film that needs subtitles to tell you who is who.

The old path

Character scenes used a multi-image reference endpoint on the video provider: a few reference images plus a prompt, and the promise that the character would stay consistent while the model invented the composition. It worked reasonably for human faces. It never handled non-human characters well, and it was recently retired by the provider without much notice. A pipeline that depends on one provider's proprietary endpoint breaks on that provider's next product decision, so we treated the retirement as a reason to change the design rather than to find a replacement endpoint.

The new path

  1. The director writes the scene prompt and lists the characters present, by cast id.
  2. The studio renders a keyframe: the image model receives the cast sheets, the scene prompt and the style guide, with the instruction to keep each design faithful to its reference.
  3. The keyframe becomes the first frame of the clip through image-to-video, at any quality tier. The video prompt describes only what changes: the motion, the camera move, the expression.

Scenes without a cast member, such as an establishing shot of the valley, still go straight to text-to-video with the style sentence prepended. Scenes marked as continuous start from the last frame of the previous clip instead of a fresh keyframe, so the set and lighting carry over.

Side effects we like

  • Cinema tiers can now use characters; before, only fast tiers could, because the reference endpoint only existed there.
  • You can inspect the keyframe before the clip exists, and the Library keeps it with its prompt.
  • A failed keyframe fails cheaply; the expensive video step never runs.
  • Composition is separated from motion. If a scene looks wrong, the problem is visible in the still, and fixing it is an edit to one prompt.

What it costs

One image generation per scene, a few cents at real cost. Compared with the video clip it precedes, it is close to free. A four-minute film with about 32 scenes adds roughly 1.30 dollars of keyframes to about 30 dollars of clips. We considered making the keyframe optional to save that amount and decided against it; the saving is trivial and the consistency loss is not.

How to get the most out of it

  • Describe the subject in the keyframe prompt, and only the motion in the video prompt. Describing the character again at the video stage invites the model to redraw her.
  • Keep the style sentence first in every prompt. Models weight the beginning.
  • Forbid borders and text in the keyframe. Image models sometimes draw a frame line or a caption, and the video model then animates a poster.
  • For drawn styles, prefer Standard. The fast tier keeps flat colour flat; Cinema tiers may add photoreal texture to a drawn keyframe.

Pitfalls

Image-to-video motion is bounded by the frame. A character running toward the camera or a fight can look constrained, because the model is reluctant to leave the composition it was given. Shorter clips help, and so does a fresh keyframe for the next beat. Some models also drift toward the end of a long clip; on the fast tier, 5 to 8 seconds per scene is the sweet spot, with narration carrying the story across cuts.

Three or more cast members in one frame is where current models start to blend features. The director is told to keep scenes to the characters who matter, and prompts name positions ("the frog on the left, the owl on the right") when two share a frame.

Frequently asked

Does the keyframe path work at every tier? Yes. Draft, Standard, Cinema and Cinema Pro all animate from the keyframe; only the durations differ, 5 or 10 seconds on the fast tiers and 4, 6 or 8 on Cinema.

Can I supply my own first frame? Yes. In the scene-by-scene workflow a storyboard frame you approve becomes the first frame of the clip through the same image-to-video step; the model starts from your picture, not from a description of it.

What if the video model ignores the keyframe? Shorten the clip, check that the keyframe has no border or text, and keep the video prompt to motion only. A framed image is read as a poster and reinterpreted.