Engineering
Gemini 4 Carbon and Hydrogen fix Argon's shell game, mostly
Gemini 4 Carbon and Hydrogen make about a third as many shell-game mistakes as Gemini 4 Argon and tie the GPT-6.1 models. Hydrogen gets there by thinking much longer, at five times the cost of Gemini 3.7 Flash.

Google opened Gemini 4 Carbon to paid API users on 10 November, at the same introductory price as Gemini 4 Argon: $2 / $10 per million input / output tokens. Two days later it released Gemini 4 Hydrogen, at a flat $1.50 / $7.50, as "a major standalone upgrade for teams migrating from Gemini 3.8 Flash". A month ago, Argon lost its points in one place, the shell game. It tracked the cup instead of the object under it, and it moved the tracker onto an identical cup. We ran both new models through the same pipeline and the same 300 questions to see if the next Gemini 4 models still do that.
Mostly, they do not. Carbon scores 0.6016 HOTA and Hydrogen 0.6006, the best Gemini results in this series so far, and we cannot separate either of them statistically from GPT-6.1 Terra, GPT-6.1 Sol or Claude Opus 5.5. Both make Argon's shell-game errors about a third as often. When they choose the same object as the GPT-6.1 models, their tracks score the same, so their remaining gap is in which object they choose.
The two new models give almost the same answers. They differ in how they get there. Hydrogen writes 2.6 times as many output tokens as Carbon, and most of the extra goes into the stage that only draws boxes, where it buys nothing.
The short version: Gemini 4 Carbon scores 0.602 HOTA for $14.60 per hour of video, and Gemini 4 Hydrogen 0.601 for $18.50. Both are clearly ahead of Gemini 3.7 Flash and level with GPT-6.1 Terra (0.617), GPT-6.1 Sol (0.616), Claude Opus 5.5 (0.594) and Gemini 4 Argon (0.593). Carbon names the cup instead of the hidden object on 3 of 44 questions, against 13 for Argon; Hydrogen never does. Hydrogen thinks about five times as long as Carbon per box, for the same boxes. From 1 December, Carbon's price doubles and Hydrogen becomes the cheaper of the two. Terra is still more accurate for less money than either.
The setup
The pipeline, the 300-question sample from the Perception Test training split, and the official HOTA scorer are unchanged from the first post. Stage 3 refers to objects by number, as introduced in the Astra post. Carbon and Hydrogen ran all three model stages from scratch, and SAM 3 carried their boxes on the same RTX PRO 6000 Blackwell as the other runs.
Four details matter if you call these models yourself:
- We used each model's default thinking level. Carbon accepts
low,mediumandhigh. Hydrogen addsxhigh, and Google says that its default ishigh. On a short puzzle about three cups, Carbon with no level set used 51 to 64 thought tokens, about the same as atmedium(50 to 78). Athighit used about 1,200. Hydrogen used 89 to 201 with no level set, and about the same atlow,mediumandhigh(84 to 202). Onlyxhighused more, about 1,300. Argon ran athigh, which we set ourselves. - Leave room for the thought. On 13 of about 2,000 calls, Carbon spent all of the 8,192 output tokens of stages 1 and 2 and returned no answer. Two more calls stopped with a finish reason that the Flash models never returned,
CONTINUATION. Those 15 questions ran once more with the same cap, and 3 failed again. They count as zero. Hydrogen hit the same cap on 14 calls. Those ran once more with a 32,768-token cap, as Argon's did, and 13 answered. If they had scored zero, Hydrogen would read 0.584. - The output limit is 32,768 tokens for Carbon and 65,536 for Hydrogen. Our stage-3 request asks for 65,536. The API reduces the request to the limit and does not reject it.
- Hydrogen is fast. With the same 16 parallel requests, its stages 1 to 3 took 44 minutes, against 63 for Carbon, even though it wrote far more tokens.
Results
| Model | HOTA | Right object | Per video hour | Carbon minus model | Hydrogen minus model |
|---|---|---|---|---|---|
| GPT-6 Astra | 0.6224 | 97.3% | $105 | −0.021 [−0.040, −0.002] |
−0.022 [−0.044, −0.001] |
| GPT-6.1 Terra | 0.6169 | 94.3% | $8.19 | −0.015 [−0.035, +0.004] |
−0.016 [−0.038, +0.005] |
| GPT-6.1 Sol | 0.6161 | 96.7% | $19* | −0.015 [−0.034, +0.004] |
−0.015 [−0.036, +0.004] |
| Gemini 4 Carbon | 0.6016 | 94.3% | $14.60 → $29.20† | −0.001 [−0.019, +0.017] |
|
| Gemini 4 Hydrogen | 0.6006 | 95.1% | $18.50 | +0.001 [−0.017, +0.019] |
|
| Claude Opus 5.5 | 0.5944 | 92.7% | $43 | +0.007 [−0.010, +0.025] |
+0.006 [−0.013, +0.025] |
| Gemini 4 Argon | 0.5933 | 92.3% | $16 → $32† | +0.008 [−0.005, +0.022] |
+0.007 [−0.013, +0.027] |
| Gemini 3.7 Flash | 0.5702 | 90.0% | $3.70 | +0.031 [+0.010, +0.053] |
+0.030 [+0.010, +0.052] |
All models on the same pipeline and 300 questions. The cost is the API bill for one full run, divided by the 1.9 hours of video that the run covers. Under each difference, its 95% interval from a paired bootstrap of the per-question difference. *Sol reused stages 1 and 2 from an earlier run, so its cost is an estimate. †The introductory price, then the price from 1 December 2026, which is twice as much; the chart below uses the later price. Hydrogen is at its list price, which does not change.
Carbon and Hydrogen are the first Gemini 4 models that are clearly ahead of Gemini 3.7 Flash: +0.031 HOTA (p = 0.01) and +0.030 (p = 0.002). Argon was +0.023 with an interval that included zero.
Between Carbon and Hydrogen there is no difference: 0.001. Against Argon, both are a little ahead, but the intervals include zero. Against Terra, Sol and Opus, the intervals also include zero. Against Astra, both intervals are just below zero, but the signed-rank test separates neither model from Astra (p = 0.37 and 0.85).
Carbon's run cost $27.94, 9% less than Argon's at the same per-token price. Hydrogen's cost $35.35, more than both, although its tokens are 25% cheaper than Carbon's. All three read the clips as video and sent the same 7.6 million input tokens. The difference is the output: Carbon wrote 1.23 million tokens, Argon at high 1.54 million, and Hydrogen 3.19 million.
The ranking by price changes on 1 December, when Carbon's and Argon's introductory price ends. Carbon's run then costs about $56, or $29 per hour of video, and Hydrogen's stays at $18.50. Against Flash, the model that Google suggests Hydrogen replaces, Hydrogen costs five times as much on this workload for +0.030 HOTA.
The shell game, revisited
The Argon post found two shell-game errors. We measured both of them again, in the same way, for every model:
| Carbon | Hydrogen | Argon | Sol | Astra | Opus | |
|---|---|---|---|---|---|---|
| Occluded-object questions answered with the occluder only (of 44) | 3 | 0 | 13 | 0 | 0 | 4 |
| … answered with the hidden object and its occluder | 0 | 4 | 0 | 0 | 0 | 0 |
| Keyframe boxes on a different object | 2.8% | 3.0% | 8.7% | 0.8% | 0.6% | 3.0% |
| Games where one object's boxes change identity (of 104) | 17 | 18 | 27 | 12 | 8 | 20 |
| HOTA on those games | 0.488 | 0.520 | 0.355 | 0.555 | 0.628 | 0.523 |
| HOTA on all 104 shell games | 0.525 | 0.528 | 0.498 | 0.547 | 0.556 | — |
The first error was a reading error. Asked for the hidden object, Argon named the cup that hides it. Carbon does this on 3 of the 44 occluded-object questions. That is the level of Opus, but not yet of the GPT-6 models, which never did it.
This is the question that we used for Argon's error. Carbon now gives the same answer as Sol, with almost the same keyframes, and the same score, 0.70. On the 11 questions where Argon named the cup and Carbon names the hidden object, Carbon's mean HOTA is 0.27, against 0.05 for Argon. That is still low. The right name is necessary, but a coin that is under a cup most of the time is hard to track for every model.
Hydrogen never names the cup alone. On 4 questions, it names both: "the marble" and "the cup that is placed over the marble". The answer has one object, so the second track counts against it, and those 4 questions score 0.16 on average. It is the same doubt as Argon's, answered both ways.
The second error was in the tracker's anchors. Each keyframe box re-anchors the same SAM 3 track, so one box on an identical cup sends the track to that cup. Carbon's and Hydrogen's boxes land on a different object a third as often as Argon's. When they do, the damage is smaller: 0.488 and 0.520 on those games, against 0.355.
This is the game from the Argon post. Argon scored 0.17 and Sol 0.88. Carbon scores 0.89.
Over all shell games, Carbon is +0.027 against Argon, CI [−0.002, +0.061], and Hydrogen is level with Carbon (+0.003). Carbon still trails Sol by 0.021, CI [−0.043, −0.000], and Astra by 0.031. The new models fixed most of the shell game, but not all of it.
The same habits as Argon
Apart from the shell game, Carbon and Hydrogen behave much like Argon:
- They almost always ask for several keyframes. Carbon and Hydrogen each used one keyframe on 2 of 300 questions (Argon 1, Sol 18, Terra 28). On questions about what changed while the camera looked away (23 questions), this helps. Carbon is +0.056 against Sol, CI [+0.007, +0.142], with 5 wins and no losses. Argon had the same advantage. This sample is small.
- They give short names. The median name is 4 words for Carbon and 5 for Hydrogen (Argon 4, Sol 6).
- Their boxes are as tight as Argon's. On the frame that stage 2 chose, the median overlap with the answer is 0.934 for Carbon, 0.936 for Hydrogen and 0.940 for Argon, against 0.931 for Sol and 0.928 for Terra. These differences are small.
- They do not lose on long lists. On the 23 questions with four or more answer objects, Carbon and Argon both score 0.527 and Hydrogen 0.515, about the same as Sol (0.532) and above Terra (0.486). On single objects, Carbon and Hydrogen are behind the three OpenAI models (0.612 and 0.605, against 0.624 to 0.630).
On containers and pouring (19 questions), Argon's best family, Hydrogen is the best of all models: 0.799, against 0.792 for Argon and 0.749 for Carbon. Carbon's lower score there includes one call that failed twice.
Carbon and Hydrogen: the same answers, five times the thinking per box
The two new models agree on most questions. They give the same verdict, right object or not, on 283 of 300 questions. On the 266 questions where both choose the right objects, Hydrogen is +0.001 against Carbon. Their tracks are equally good.
Their mistakes are of different kinds. Hydrogen chose wrong on 29 questions, Carbon on 22 and Argon on 27. But 20 of Hydrogen's are partial: one object of a list missing, or one object too many, like the hedged answers above. Only 6 are a wrong object outright, against 8 for Carbon and 13 for Argon. A partial answer keeps part of the score, so the totals come out level. Fifteen questions defeat all three Gemini 4 models, mostly launches and letters, and on 8 of them a GPT model is right.
The large difference is in the thinking:
| Output tokens | Carbon | Hydrogen | Argon (high) |
|---|---|---|---|
| Stages 1 and 2, the whole run | 0.64 M | 1.11 M | 0.64 M |
| Stage 3, the whole run | 0.59 M | 2.08 M | 0.90 M |
| Stage 3, median call, one object to box | 47 | 387 | 239 |
| Stage 3, median call, four or more objects | 444 | 1,971 | 716 |
| Stage 3, longest call | 22,039 | 62,138 | 25,941 |
Stage 3 only puts boxes on objects that stage 1 already chose, on frames that stage 2 already chose. Hydrogen spends 65% of its output tokens there, about $15.60 of the $35.35. It buys nothing that we can measure. We sorted the 266 questions where both models chose right into thirds by how long Hydrogen thought in stage 3. In each third, Hydrogen and Carbon score the same, within 0.003.
This frame shows both reasons for a long stage-3 call. The first is a name that describes an action. "The letter T" can be found in a still frame. "The letter that the person touches after pouring water" cannot: the model must first work out from one frame which letter that was. Names like this make every model think longer in stage 3. Carbon's median call is 333 tokens with such a name and 80 without, and Hydrogen's 1,637 and 407. Hydrogen does not use them more often: about a quarter of the questions, for both models. The second reason is Hydrogen itself. With the same kind of names, it thinks about five times as long as Carbon.
What is left: which object answers the question
We sorted the questions by which model chose the right object:
| Questions | n | Carbon minus Sol | n | Carbon minus Astra |
|---|---|---|---|---|
| Both choose the right object | 275 | +0.000 | 276 | −0.004 |
| Only the other model chooses it | 12 | −0.018 | 12 | −0.018 |
| Only Carbon chooses it | 3 | +0.003 | 2 | +0.001 |
| Neither does | 10 | +0.000 | 10 | −0.001 |
| All questions | 300 | −0.015 | 300 | −0.021 |
Each cell is that group's contribution to the mean difference over all 300 questions.
When Carbon and Sol choose the same object, their scores are the same to three decimals. All of Carbon's gap to Sol, and most of its gap to Astra, is in about a dozen questions where it chose a wrong object. Hydrogen's split against Sol is the same: +0.001 where both choose right, −0.018 where only Sol does.
These questions are mostly not new. Carbon chose a wrong object, or too few objects, on 22 questions, and Argon also failed on 17 of them. Carbon fixed 10 of Argon's 27. The clearest shared weakness is launches: Carbon chose a wrong object on 6 of 70 launch questions, and Argon and Hydrogen on 7 each, against 2 for Astra.
In this example, Carbon's stage-2 reasoning describes the scene correctly: the ball "bounces off the standing book and rolls back toward the left, where the person grabs and stops it". It then answers with the person, who brings the ball to rest, not the book, which stops its motion after the first launch. Argon gave the same answer. Like the occluder error, this is not a failure to see. The model reads the question differently from the annotators.
Strengths, weaknesses and what could be better
Strengths. Carbon and Hydrogen are the best Gemini models in this series, the first Gemini 4 models with a clear gain over Flash, and a tie with Terra, Sol and Opus. Their tracks are as good as Sol's when they choose the right object. They are strong on long object lists and on objects that appear while the camera looks away. Hydrogen is fast, and it is the best of all models on containers and pouring.
Weaknesses. Both read some questions differently from the annotators, mostly launches and hidden objects. Their keyframe boxes still land on another cup almost four times as often as Sol's. At their default thinking levels, a few calls think until they hit the output cap. Hydrogen thinks far longer than the task needs, which makes it the most expensive Gemini 4 model until 30 November.
What could be better. Three changes are worth testing. We have not run any of them.
- A larger output cap for stages 1 and 2. If the 3 Carbon questions that failed twice had Carbon's mean score, the run would read 0.608. Argon's and Hydrogen's retries with a 32,768-token cap almost all answered, so this is probably easy to recover. It does not change Carbon's place in the table.
- Less thinking in Hydrogen's stage 3. If Hydrogen's stage 3 used as many tokens as Carbon's, its run would cost about $24, or $12.60 per hour of video, below Carbon at its launch price. Setting
lowis not a sure way to get there: on our puzzle,lowthought as long as the default. A stage-3 prompt that asks for the boxes without an explanation might do more. - A clearer question in stage 1. Both remaining error types are about which noun answers the question. A stage-1 step that restates the target in plain words ("the hidden object, not the cup", "the object the ball hits first") could catch them. It could also ask for names that describe what an object is, not what happens to it. That change applies to every model, so it would need a full re-run of the field.
Google suggests pairing Carbon and Hydrogen in one workflow. In this pipeline, a pair gives no gain: the two models choose the same objects, and their boxes are equally good. A hybrid with GPT-6.1 for stages 1 and 2 and Carbon for stage 3 is not worth it either. Where both models choose the same object, Carbon's boxes give no gain, and Terra alone already costs less than Carbon.
What this means in practice
If you use Gemini 4 Argon, switch to Carbon or Hydrogen. Both make about a third as many shell-game errors. Until 30 November, Carbon is the cheaper of the two. From 1 December, Hydrogen is: $18.50 per hour of video, against about $29 for Carbon.
If you use Gemini 3.7 or 3.8 Flash, Hydrogen is a real gain, +0.030 HOTA, but on this workload it costs about five times as much. Check your own bill before you migrate.
If you choose a model for accuracy per dollar, Terra is still the better choice: more accurate than either new Gemini for 56% of Carbon's launch price.
If your questions are about which of several identical objects did something, the GPT-6 models are still a little safer. The new Geminis make fewer shell-game errors than Argon, but they are the same kind: a smooth track on the wrong object, which a downstream count or alert cannot detect.
Limitations
- Every figure comes from the same 300 training questions as the earlier posts. Results for groups of fewer than 30 questions are indicative only.
- Each model ran at its default thinking level except Argon, which ran at
high. Hydrogen's default ishigh, and Carbon's thinks about as much asmedium. Part of every comparison between them is the thinking budget. - Hydrogen's failed calls ran again with a 32,768-token cap and Carbon's with 8,192. With zeros for its 14 failed calls, Hydrogen would read 0.584.
- Calls that failed report no token use, so up to about $1.50 is missing from each of the Carbon and Hydrogen costs.
- Carbon and Hydrogen ran all three stages. Sol and Astra reused stages 1 and 2 from earlier runs. Stage 3 samples each box again on every run, so part of every difference is run-to-run variation.
Agnify answers questions like these from the cameras already on a production floor. To see the pipeline on your own footage, read about Agnify for manufacturing or book a demo.
FAQ
How does Gemini 4 Carbon compare with Gemini 4 Argon on grounded video QA?
Carbon scores 0.602 HOTA and Argon 0.593, a paired difference of +0.008 (95% CI [−0.005, +0.022]), so the two are close. Carbon makes about a third as many shell-game errors: it named the hiding cup instead of the hidden object on 3 of 44 questions, against 13 for Argon. Its run cost $27.94 against $30.72 at the same introductory price.
Is Gemini 4 Hydrogen better than Gemini 4 Carbon?
No, the two are level: Hydrogen scores 0.601 and Carbon 0.602, a difference of 0.001 (95% CI [−0.017, +0.019]). They give the same verdict on 283 of 300 questions. Hydrogen writes 2.6 times as many output tokens, most of them in the box-drawing stage, where they do not change the boxes. Until 30 November Carbon is cheaper; from 1 December Hydrogen is.
Is Gemini 4 Hydrogen a good replacement for Gemini Flash?
It is more accurate: +0.030 HOTA against Gemini 3.7 Flash (95% CI [+0.010, +0.052]), with gains on launches and containers. But on this benchmark it costs $18.50 per hour of video, against $3.70 for Flash, because it thinks much longer at its default level.
Is Gemini 4 Carbon better than GPT-6.1?
It is level with both GPT-6.1 models: −0.015 HOTA against Terra (95% CI [−0.035, +0.004]) and −0.015 against Sol (95% CI [−0.034, +0.004]). When Carbon and Sol choose the same object, they score the same. Terra costs $8.19 per hour of video, against $14.60 for Carbon.
What thinking level should I use with Gemini 4 Carbon and Hydrogen?
We used the defaults: about medium for Carbon, high for Hydrogen. At those levels, 13 of about 2,000 Carbon calls and 14 Hydrogen calls used all 8,192 output tokens of our first two stages on thinking and returned no answer. Give the first stages a larger output cap than the answer needs. For Hydrogen, look at where the tokens go: in our pipeline, two thirds went to the stage that only draws boxes.
How much do Gemini 4 Carbon and Hydrogen cost for video?
On this benchmark, Carbon costs $27.94 for a 300-question run, or $14.60 per hour of video, at the introductory rate of $2 / $10 per million input / output tokens. The rate doubles on 1 December 2026, which makes it about $29 per hour of video. Hydrogen costs $35.35, or $18.50 per hour of video, at its flat rate of $1.50 / $7.50.





