How I Build an AI Short Film End to End

Most AI films fall apart for the same reason: someone generated beautiful clips before they knew what story those clips were serving. A generator will happily give you a hundred gorgeous seconds that add up to nothing. The pipeline below is the one I actually use in the studio, and every stage exists to protect the story from the tooling.

1. The one-sentence spine

Before a single prompt, I write the film in one line: who wants what, and what stands in the way. "A market trader in Lagos hides a letter she cannot read" is a film. "Cinematic Afrofuturist city" is a screensaver. If the spine will not survive being read aloud to someone who does not care about AI, the film will not survive either. I keep the spine at the top of the working document and re-read it whenever a shot tempts me to wander.

2. Beats before shots

Next comes a beat sheet of six to twelve lines for a two-minute piece. Each beat is a change of state, not a description. Beat four is not "she walks through the market" — it is "she realises the letter is about her son". Only then do I break beats into shots, and I keep the shot count low. A ninety-second film with eighteen shots feels considered; the same film with sixty shots feels like a trailer for something that does not exist.

3. Locking the look and the people

This is where AI films are won or lost. For every recurring character I build a short reference sheet: three angles, one neutral lighting setup, a fixed wardrobe description, and a written identity block I paste into every prompt — age, build, skin tone, hair, distinguishing features, clothing. I generate the reference images first, approve them, and treat them as canon. If a later shot drifts, I fix the shot, not the canon.

The same discipline applies to the world. I write a look block once — lens feel, colour palette, time of day, weather, grain — and reuse it verbatim. Consistency in AI work comes from repetition of language, not from cleverness.

4. Generation in passes, not in one heroic sitting

I generate in three passes. The blocking pass is fast and ugly: cheap settings, one take per shot, purely to see whether the sequence reads. The performance pass re-generates only the shots that carry emotion, with tighter prompts and several takes. The polish pass handles inserts, establishing shots and anything with visible hands, text or crowds — the places models still struggle.

Two habits save enormous time here. First, name every output file after its shot number, take number and the seed. Second, keep a rejected folder rather than deleting; the take you discarded at midnight is often the one that cuts perfectly at noon.

5. Working with the model's weaknesses

Rather than fighting known failure modes, I design around them. Hands go into pockets, behind backs, or out of frame. Signage is added in post instead of prompted. Long unbroken walks become two shorter shots with a cut on movement. Crowds live in soft focus behind the subject. Dialogue-heavy moments become reaction shots over voice-over, because a lip-sync that is 90 per cent right is worse than no lip-sync at all.

6. Sound is half the film

An AI film with weak audio always feels fake, no matter how good the frames are. I build four layers: dialogue or narration, a music bed, spot effects on every visible action, and a continuous room tone that ties unrelated clips into one space. That last layer does more for believability than any upscaler. Where I score the piece myself, I cut the music to the beat sheet, not to the finished edit, so the emotional shape is set before the pictures are locked.

7. The edit hides the seams

In the edit I trim every clip harder than feels comfortable — most generated shots are strongest in their first sixty per cent, before motion starts to wobble. Cutting on movement, on a sound, or on a look off-screen buries small inconsistencies between takes. A final grade across the whole timeline unifies clips that came from different generations and different days; a light grain pass over everything does the rest.

8. The last hour

Before export I watch the film three times: once with the sound off, checking that the story is legible in pictures alone; once with the picture off, checking that the audio alone still has shape; and once on a phone at arm's length, because that is where most people will meet it. Anything that only works on a large monitor gets fixed or cut.

What this pipeline is really for

None of this is about squeezing more realism out of a model. It is about keeping authorship where it belongs. The tools change every few months — the beat sheet, the identity block, the three-pass generation rhythm and the sound layers have survived every change so far. Start there, and the next model release becomes an upgrade rather than a rebuild.