Fourteen AI models tried to rebuild a photo in Blender

The job

For this experiment, an AI agent turns a photo into an editable Blender scene by writing and running code, rather than generating a mesh directly with a neural network. The agent follows an iterative process: it measures the picture, models the objects, sets the camera, lights the scene, renders, looks at its render next to the photo, and goes again. The result is a real Blender file with named, editable parts.

I am building a photo-to-Blender tool whose agent is powered by a language model. Until this test, I had used OpenAI's GPT-6 Astra, at about $4 a scene. That price is the reason for this test: if a cheaper model could do the job even nearly as well, the cost of a scene would fall several times over. So I built a bench where every model gets the same job under the same limits, and put fourteen on it.

The written brief asks for a faithful, editable reconstruction, forbids pasting the photo onto geometry, and tells the agent to save a usable scene early and then improve it, because the clock and the budget are enforced from outside by terminating it.

How I judged it

The tool normally scores the picture the agent hands in. For this benchmark, I instead open each delivered scene file and render it again at the reference photo’s aspect ratio, with a maximum dimension of 1,100 pixels and 96 samples. The scoring code then compares that render with the reference. A run that delivers no usable scene scores 0 and stays in the average.

The judge is deterministic image-and-mesh code, not an LLM: scorecard version 2, image metrics version 1.0-numpy, mesh audit version 1.0. Shape match uses estimated silhouette overlap (30%); detail uses edge F1, a measure of matching image edges (40%); colour uses multiscale RGB error (15%); and mesh health uses the weakest of five topology checks (15%). Those checks cover non-manifold and open edges, duplicate faces, inconsistent winding and degenerate triangles after position welding. Image comparisons use a maximum dimension of 512 pixels. The measurements are mapped through calibrated piecewise-linear scales to 0–100; these are comparison scores, not percentages of the scene recovered. All 36 delivered scenes have all four components. Download the run scores, costs and frozen board settings, and the same for the newest three.

Two more checks. A second render swings the camera 35° round the subject, so a flat stand-in behind the matched view would show. And every image file a scene uses is compared with the photo, since the brief forbids using it. No reference-file match was flagged in the delivered scenes, and I saw no obvious pasted-photo stand-in in the side views. Filename and digest checks can miss a renamed, re-encoded image; a single side view is also an inspection aid, not proof of complete 3D geometry.

The bench renders each scene itself. The picture a model hands in is never the one that gets scored.

Four things the board says

1. Astra works until the money runs out

Astra averaged 66 and was first on every photo. It was also stopped by the cost cap on every scene: it had a first test render 4 minutes in, then kept refining until, at about $3.90 and 18 minutes, the gateway refused the next request. It never ran out of time. The cost cap ended these three runs before the time limit. A larger budget would allow further attempts at refinement; whether those attempts improve the result needs another experiment.

2. The cheap OpenAI models stop early

GPT-5.6 Sol scored 49 for $0.57 a scene and GPT-5.6 Terra 44 for $0.33. Neither reached a time, cost or request cap. They declared the job finished after about 8 minutes, having spent $0.57 and $0.33 of a $4 budget on average. Their scenes are simpler than Astra's, not broken, and they had most of their budget and time left. That makes them a useful next experiment: a brief that asks for additional refinement within the remaining limits. If Sol moves from 49 towards 60 at a dollar a scene, the economics of the product change.

GPT-6.1 Sol, the newest of them, does not stop early. It worked for about 18 minutes a scene and scored 61 for 36 cents an attempt.

Sol and Terra stopped with substantial time and budget remaining. I have not yet tested whether more refinement would improve their scores.

3. Claude, and the cache

Claude Fable 5.1 scored 48 and Claude Opus 5 45, both at Astra's price. Fable's desk is the closest to Astra's on the board; its car melted and its shopfront is a pale single wall. Opus's shopfront (56) is its best scene and its car (34) its worst. The newest Claude models do much better: Claude Opus 5.5 scored 60 at $1.19 an attempt and Claude Sonnet 5.5 56 at 50 cents, the quickest on the board at about five minutes a scene.

Their first attempt did not count, and it is worth telling. In the gateway configuration used for the first attempts, Anthropic requests lacked the cache-control markers needed to enable caching. So on the first attempt 0% of their context was served from cache, against 96% for the OpenAI models. Each request incurred the full input charge for the conversation again. The archived Claude runs that reached the budget cap did so after eight to eighteen requests; Astra averaged about thirty-six. The gateway now adds the marker for Anthropic models, both were run again at 96–97% cached, and those are the numbers on the board. The archived uncached attempts logged about $16.39. They are excluded from the ranking but included in the experiment’s spending summary below.

4. The slow thinkers

DeepSeek V4.1 Flash and Qwen3.8 Max delivered nothing in 20 minutes, on any photo. Neither is incapable of the work. Both spend most of their output on private reasoning (83% and 68% of what they wrote), measuring the photo and fitting the camera at length, and the clock ran out before a scene was saved. DeepSeek made about as many requests as Astra in its 20 minutes; it just never rendered.

So I gave them more time, on separate boards so the results never mix with the 20-minute ones. With 60 minutes, DeepSeek built a lumpy car (44) and a one-wall shopfront (42) for about 60 cents each, and still failed the desk. With 120 minutes and the full $4, Qwen built a decent desk (51) and a car the score puts at 55 though it is a flattened body with a lump for a roof, and lost the shopfront when it called a web-search tool it had never been given. DeepSeek, given 120 minutes, delivered nothing at all, alongside a large slowdown in provider response speed, with individual responses taking 26 and 30 minutes. That makes it hard to separate model behaviour from serving conditions.

Also on the board

• Pareto (47, $0.55 estimated). This model ran anonymously as Union Alpha. It delivered every scene and sits with the older models in the 40s. OpenRouter charged $0 during the preview; repricing the logged tokens at Pareto’s revealed rates gives $0.55 per scene.
• Kimi K3 (41, $1.55). Busy and quick to a first render, and hit the request cap on two scenes. One run got an empty first answer from the model and ended after twelve seconds; the bench now restarts a run that ends within a minute with nothing saved, for every model alike, and that scene was run again.
• GLM 5.3 Flash (36, $0.089). Nine cents a scene, and it delivered all three. Crude, but its low cost makes it worth another look. Scores are a calibrated comparison scale, so dividing them by dollars is not a reliable measure of useful output.
• Gemini 3.8 Flash (34, $2.10). The Google endpoint used in this test rejected requests that ended on the model’s own message, which the agent tool sends whenever a model says what it will do next without calling a tool. Two attempts died on that before the gateway learned to add a closing turn for Google models. Its prompt was also only 47% cached, which contributed to its $2.10 average cost per attempt.

What I would not conclude

The newest three, GPT-6.1 Sol, Claude Opus 5.5 and Claude Sonnet 5.5, ran later than the rest, on the current version of the brief and in their makers’ own agent CLIs. To see how much that changes, I ran GPT-6 Astra the same way: it scored 63 rather than 66. Read the gap between those three and the older models with that in mind.

There is one run per model per image. Small differences should be treated cautiously, and even Astra’s larger lead needs repeated runs to establish its stability. The brief was written and refined while Astra was the agent: every model got the same text, but it was shaped around one model’s habits. The OpenAI agent CLI also uses generic fallback settings for other models. Two providers needed gateway fixes before their three models could compete reliably. These are results for this model-plus-agent setup, not a model-neutral ranking or a proven lower bound on any model’s ability. Models running at the same time shared a GPU, so elapsed time also reflects contention.

The image components compare one rendered view with the reference; the mesh component checks a limited set of topology defects. Neither establishes the shape of hidden surfaces, physical dimensions, semantic part quality or ease of editing. Estimated silhouettes can be unreliable in full-frame photos, open surfaces may be intentional, and the mesh audit does not test self-intersections.

Cost, and what is next

The 42 accepted runs across the three boards logged $63.07: $50.79 for the main board, $1.68 for the 60-minute board and $10.60 for the 120-minute board. Archived attempts add about $17.64, including $16.39 for uncached Claude attempts, giving about $80.71 in recorded API spending. Some archived cost snapshots contain unsettled requests, so that is not a reconciled invoice total. Pareto’s preview was free; its estimated $1.64 replacement cost is excluded from these charged totals. Accepted runs alone cost about sixteen Astra attempts; including archived spending, about twenty-one. Local rendering and hardware costs are not included. The newest three ran on subscriptions; at list prices their nine attempts come to about $6.13, and the check run of Astra to about $5.08.

Next: all fourteen models on the current brief, three runs per image for the top group so that spread is visible, and a larger image set. Longer term, a model-neutral agent loop would remove the biggest asterisk on this board.