All posts

Engineering

GPT-6.1 Terra matches Sol at less than half the price

GPT-6.1 Terra scores the same as GPT-6.1 Sol on grounded video QA for 43% of the cost. Most of what it gives up comes from following one instruction to the letter.

HOTA against API cost per hour of video, log scale. GPT-6.1 Terra 0.617 at $8.19, GPT-6.1 Sol 0.616 at $19, GPT-6 Astra 0.622 at $105, Claude Opus 5.5 0.594 at $43, Gemini 4 Argon 0.593 at $16, Gemini 3.7 Flash 0.570 at $3.70.

OpenAI released GPT-6.1 Terra on 26 October at $1 / $5 per million input / output tokens, half of GPT-6.1 Sol's rate. We ran it through the same pipeline and the same 300 questions as GPT-6 Astra and Gemini 4 Argon.

Terra scores 0.6169 HOTA. That is Sol's score for 43% of Sol's cost, and a score we cannot separate statistically from Astra's, for 8% of Astra's cost. Terra is also the first model that is clearly ahead of Gemini 3.7 Flash at about twice Flash's price. The breakdown shows where its remaining points go: Terra applies one instruction in our pipeline more strictly than any other model we have tested, and the pipeline does not handle that well.

The short version: GPT-6.1 Terra scores 0.617 HOTA, the same as GPT-6.1 Sol (0.616) and close to GPT-6 Astra (0.622), for $8.19 per hour of video against $19 and $105. It is clearly ahead of Gemini 3.7 Flash and Claude Opus 5.5. It is the best of six models on single-object answers and the weakest on answers with four or more objects. Most of its deficit to Astra comes from questions where it found no frame that met the visibility rule exactly.

The setup

The pipeline, the 300-question sample from the Perception Test training split, and the official HOTA scorer are unchanged from the first post. Stage 3 refers to objects by number, as introduced in the Astra post. The Sol and Astra v3 runs reused their first two stages from earlier runs. Terra ran all three model stages from scratch, and SAM 3 carried its boxes on the same RTX PRO 6000 Blackwell as the other v3 runs.

Two details matter if you call Terra yourself:

  • It takes Sol's request unchanged. We changed only the model name and kept the reasoning effort at high, as for Sol and Astra. Over 1,880 calls, no call failed, was refused or ran out of output tokens.
  • Each call was slower than with Sol. The median call took 22 seconds, about the same as Astra and against 13 seconds for Sol. We measured this in Terra's launch week, so it can change. For now, allow for it when you set your concurrency.

Results

Model HOTA Right object Run cost Per video hour Terra minus model, 95% CI
GPT-6.1 Terra 0.6169 94.3% $15.68 $8.19
GPT-6 Astra 0.6224 97.3% $200.42 $105 −0.006 [−0.021, +0.009]
GPT-6.1 Sol 0.6161 96.7% $36.31* $19 +0.001 [−0.015, +0.017]
Claude Opus 5.5 0.5944 92.7% $82.41 $43 +0.023 [+0.001, +0.045]
Gemini 4 Argon 0.5933 92.3% $30.72† $16 +0.024 [+0.004, +0.044]
Gemini 3.7 Flash 0.5702 90.0% $7.08 $3.70 +0.047 [+0.023, +0.072]

All models on the same pipeline and 300 questions. Cost is the API bill for one full run; the run covers 1.9 hours of video. The interval is a paired bootstrap of the per-question difference. *Sol reused stages 1 and 2 from an earlier run, so its cost is an estimate. †Introductory price; it doubles on 1 December 2026.

HOTA against API cost per hour of video, log scale: Gemini 3.7 Flash 0.570 at $3.70, GPT-6.1 Terra 0.617 at $8.19, Gemini 4 Argon 0.593 at $16, GPT-6.1 Sol 0.616 at $19, Claude Opus 5.5 0.594 at $43, GPT-6 Astra 0.622 at $105. A dashed line joins Flash, Terra and Astra.

Against Sol, Terra is a tie: the mean difference is +0.001, and Terra wins 107 questions and loses 122. Against Astra it is −0.006. The interval includes zero, and the signed-rank test does not separate the two (p = 0.07).

Below the two GPT-6 models, Terra is ahead. It beats Gemini 3.7 Flash by 0.047 (p = 0.03) and Claude Opus 5.5 by 0.023 (p = 0.03), at 19% of Opus's cost. Against Gemini 4 Argon the interval excludes zero, but the signed-rank test does not separate them (p = 0.32).

Terra's run cost less than half of Sol's for two reasons. Each token costs half as much, and Terra used fewer tokens per call: 6,861 input and 296 output on average, against 7,602 and 393 for Sol's full run at the same reasoning effort. Gemini 3.7 Flash is still the cheapest model, at $3.70 per hour of video. Until now, the cheapest model with a clear accuracy gain over Flash cost five times as much per video hour. With Terra it costs 2.2 times as much.

The gap to the GPT-6 models is object choice

We sorted the questions by which model picked the right object:

Questions n Terra minus Sol n Terra minus Astra
Both pick the right object 280 +0.006 282 +0.002
Only the other model picks it 10 −0.011 10 −0.010
Only Terra picks it 3 +0.006 1 +0.003
Neither does 7 +0.000 7 −0.001
All questions 300 +0.001 300 −0.006

Each cell is that group's contribution to the mean difference over all 300 questions.

When both models choose the right object, Terra's tracks are as good as Sol's and Astra's. Its whole deficit is in about ten questions where it chose a wrong object or no object. On single-object questions, Terra has the highest score of the six models: 0.630, against 0.624 for both GPT-6 models. That margin is too small for us to separate from noise.

Terra also avoids both of the shell-game errors from the Argon post. Its keyframe boxes changed identity in 10 of the 104 shell games, against 12 for Sol, 8 for Astra and 27 for Argon. It answered an occluded-object question with the occluder at most once.

So where do Terra's ten questions come from? Most of them come from one sentence in the prompt.

One rule, read to the letter

Stage 2 asks for times at which the answer objects are "plainly visible — in view, not covered by anything, not cut off by the frame edge". The prompt adds that a model should give fewer times rather than guess. If the model gives no time at all, the pipeline uses the middle of the clip instead.

All three OpenAI models sometimes give no time. Terra did this on 18 questions, Sol on 12 and Astra on 11. Opus and Argon always gave at least one time. On 8 questions, Terra gave no time and Sol gave some. On four of those 8, the middle frame was good enough and Terra scored as well as Sol. On the other four, Terra lost more than 0.3 HOTA. In two of them, Terra had also named the wrong objects, or too few, in stage 1. The other two are the rule at work.

Asked for objects added while the camera looked away, GPT-6.1 Terra says one end of the pen is obscured by the tissue packet and gives no keyframe time; the middle of the clip, frame 274, shows the camera still pointing away from the table, so there is no box and HOTA is 0.00. GPT-6.1 Sol calls the pen clearly visible, gives five times and boxes it at frame 401; HOTA 0.76.

Terra saw that the tissue packet covers one end of the pen, so it gave no time. Sol and Astra saw the same packet and called the pen "clearly visible". By the letter of the prompt, Terra is right. But the middle of the clip is from before the camera returns to the table, so stage 3 had nothing to box.

Asked which shapes the person put on the table, GPT-6.1 Terra says the large yellow circle is cut off by the right edge and gives no keyframe time; at the middle of the clip, frame 511, no shape is on the table yet and Terra boxes circles drawn on the paper; HOTA 0.01. GPT-6.1 Sol gives times for the three shapes that are whole and boxes all four at frame 953; HOTA 0.62.

In the second example, both models named the same four shapes. Sol gave times for the three shapes that were whole, and stage 3 boxed the fourth on those same frames. Terra declined because the fourth shape touched the frame edge. At the middle of the clip, no shape was on the table yet.

Together, the 8 questions cost Terra 0.009 against Sol. Against Astra, the equivalent 7 questions cost 0.008, which is more than Terra's whole gap to Astra.

This is the reverse of the problem in the Astra post. There, a model that reworded its labels broke a stage that expected exact copies. Here, the model that follows the instruction most strictly loses points to a fallback that assumes models rarely give up. The fix belongs in the pipeline: when stage 2 gives no time, ask for the best partly visible moment instead of jumping to the middle of the clip. We have not tested that yet.

Answers with many objects

On questions with four or more answer objects, Terra scores 0.486, the lowest of the six models (Sol 0.532, Astra 0.543, over 23 questions). The shapes question above accounts for more than half of its gap to Sol there. Two other patterns add to it. On 6 of the 121 multi-object questions, Terra named fewer objects than the answer has, against 3 for Sol and 2 for Astra. And on one question with six letters, it named every letter correctly but drew loose boxes: an overlap of 0.29 to 0.55 with the answer, against 0.62 to 0.90 for Astra. On three-object answers Terra is also below Astra (−0.018, CI [−0.036, −0.001]) and level with Sol.

The result is a clear profile: Terra is the best model we have tested on single objects and the weakest of the six on long lists of objects.

What this means in practice

If you use Sol today, Terra gives the same accuracy for 43% of the price. Before you switch, check what your pipeline does when the model declines to answer. Terra declines more often than Sol, and usually for a reason that is correct by the letter of your prompt.

If you use Gemini 3.7 Flash for continuous monitoring, Flash is still the cheapest option. Terra is the first model that is clearly more accurate at a cost of the same order: $8.19 per hour of video against $3.70.

If your questions are about many objects at once, such as sets of parts or letters, Sol and Astra are still the safer choice.

Limitations

  • Every figure comes from the same 300 training questions as the earlier posts. Results for groups of fewer than 30 questions, such as the 23 answers with four or more objects, are indicative only.
  • Terra ran all three stages. Sol and Astra reused stages 1 and 2 from their earlier runs and ran stage 3 again. Stage 3 samples each box again on every run, so part of every difference is run-to-run variation.
  • One Terra stage-1 call reported no token counts, so its cost is missing from the $15.68.
  • We measured call latency in Terra's launch week.

Agnify answers questions like these from the cameras already on a production floor. To see the pipeline on your own footage, read about Agnify for manufacturing or book a demo.

FAQ

How does GPT-6.1 Terra compare with GPT-6.1 Sol on grounded video QA?

They are tied. Terra scores 0.617 HOTA and Sol 0.616, a paired difference of +0.001 (95% CI [−0.015, +0.017]). Terra costs $8.19 per hour of video against $19 for Sol. Sol picks the right object slightly more often. When both models pick it, Terra's tracks are as good as Sol's.

Is GPT-6.1 Terra better than Gemini?

Than Gemini 3.7 Flash, yes: Terra is +0.047 HOTA (95% CI [+0.023, +0.072], p = 0.03) and costs 2.2 times as much. Against Gemini 4 Argon, Terra is +0.024 (95% CI [+0.004, +0.044]), but the signed-rank test does not separate them (p = 0.32). Terra costs about half as much as Argon at Argon's introductory price.

How much does GPT-6.1 Terra cost for video?

On this benchmark, $15.68 for a 300-question run, or $8.19 per hour of video, at $1 / $5 per million input / output tokens. That is 43% of GPT-6.1 Sol's cost and 8% of GPT-6 Astra's.

Why does GPT-6.1 Terra sometimes return no box?

The keyframe stage asks for moments when the object is fully visible, not covered and not cut off by the frame edge. Terra applies that rule more strictly than other models. It gave no time on 18 of 300 questions, against 12 for Sol and none for Gemini 4 Argon or Claude Opus 5.5. Our pipeline then falls back to the middle of the clip, which sometimes comes before the object appears.