Google DeepMind researchers have found that changing the picture fed into a video model, not the text describing the task, can be the more effective lever for improving how well the model reasons. The team, whose project page lists Robert Geirhos as the core-team byline alongside nine co-authors, describes the technique in a project page and accompanying arXiv preprint posted this month. Prompt engineering has been treated as a text-only discipline since the earliest days of language models; this work argues the same leverage exists on the input image.
The researchers call the technique visual prompt engineering, or VIPE, and tested it across six visual reasoning tasks. These include a physics task that asks which bucket a ball lands in after rolling through obstacles, a conjunctive visual search task, a dot-connecting task, a three-number sorting task, maze solving at several grid sizes, and a sliding-block puzzle called RushHour. In each case, the model was first shown a plain, schematic version of the task image, then a version whose visual style had been altered.
VIPE comes in two forms on the project page. Freeform VIPE hands the task image to an image-editing model with an open stylistic instruction, turning a diagram of ramps and obstacles into, in one successful example, an industrial factory setting with steel chutes. Atomic concept edit, or ACE, VIPE instead makes one narrow, targeted change, giving two flat, uniformly colored circles the shading and reflections of polished glass, or thickening the stroke used to connect two dots. Across the task examples the project page shows, attempts that failed on the plain image succeeded once the model was given an edited variant of the same scene.
The explanation the researchers offer is about training data, not cleverness. Video models learn their sense of physics, material, and motion from the footage they were trained on, and that footage is overwhelmingly the textured, photographic real world: factories, kitchens, billiard tables, arcades. A flat sketch of circles, lines, and ramps sits far outside that distribution. Rendering the same scene with the shine and depth of a felt-topped table, or a maze rendered as a classic arcade game, moves the input toward imagery the model has actually seen in volume, rather than forcing it to generalize to a schematic style it rarely encountered in training.
The project’s abstract makes a comparative claim: image edits moved these tasks further than two established techniques did, rewriting the instructions in words, and letting the model spend more computation working through the problem at inference time. That claim comes from the researchers’ own abstract; the page does not publish an independent, third-party accuracy table alongside it. What it shows instead is task-by-task, split-by-split demonstration evidence: mazes at 5x5, 7x7, 9x9, and irregular layouts, and RushHour puzzles across multiple difficulty levels, each paired with a failed attempt on the original image and a successful attempt on an edited one.
For anyone building on video models, the finding reframes preprocessing as a design decision with roughly the same weight as prompt wording. A pipeline that feeds a raw diagram, chart, or synthetic render straight into a video model may be leaving reasoning accuracy on the table that a single image-editing pass, pushing the input closer to photorealistic or otherwise natural footage, could recover. Teams evaluating video models on structured or diagrammatic inputs should test an edited version of their images before assuming the model has hit a capability ceiling.
Google DeepMind researchers, credited on the “Visual Prompt Engineering for Video Models” project page with Robert Geirhos as core-team byline, published the findings and an accompanying arXiv preprint (arXiv
.25537) this month.
