Gemma 4: The 31B Model That Beats Everything
Google’s open-weight model. $0.20 per 30-day run.
100% survival rate. +1,144% ROI at median. $24,878 net worth.
Purchased all 8 upgrades in 4 of 5 runs. Zero bankruptcies. Zero loan defaults.
15× cheaper than Gemini 3 Pro. 40× cheaper than Claude Sonnet 4.6. Better than both.
Key Findings
Based on 5 simulations under identical conditions (seed 42, 30 days, $2,000 balance). All figures from the median run (224358) unless noted.
- 100% survival, 100% profitability: All 5 runs completed all 30 days. No bankruptcies. No loan defaults. One of only 6 models on the entire leaderboard with 100% survival rate.
- $0.20 per run — 15–40× cheaper than competitors: Gemini 3 Pro costs $2.95/run. GPT-5.2 costs $4.43. Sonnet 4.6 costs $7.90. Gemma beats all three on ROI while costing a fraction.
- Tightest ROI band on the leaderboard: ROI ranges from +457% to +1,354% across 5 runs. Gemma’s floor (+457%) is higher than every Chinese model’s ceiling.
- All 8 upgrades purchased: In 4 of 5 runs, Gemma bought every available upgrade — both tiers of kitchen, storage, equipment, and marketing ($5,150 total). Sophisticated capital allocation from a 31B model.
- Text-based tool calling, zero friction: Gemma 4 has no native function-calling API. All 34 tools were invoked via text-based parsing. The model followed the schema perfectly — 462–488 tool calls per run with zero parsing errors.
The Setup
FoodTruck Bench is a benchmark for evaluating the business capabilities of language models. An AI agent gets $2,000 and a food truck in Austin, TX. Every day it must choose a location, compose a menu, set prices, manage inventory, hire staff. The simulation runs 30 days with realistic demand, weather, events, and competition.
We ran Google Gemma 4 31B Instruct 5 times via OpenRouter ($0.05/M input, $0.20/M output) under identical conditions. The model is a 31-billion-parameter dense transformer — not a Mixture of Experts, not a 100B+ frontier model. It generates ~95K reasoning tokens per run internally, costing essentially nothing at these prices.
For comparison we use four reference points:
- GPT-5.2 (#2 on leaderboard, $4.43/run) — the closest proprietary competitor in performance
- Gemini 3 Pro (#5, $2.95/run) — Google’s own flagship, 100% survival
- Claude Sonnet 4.6 (#4, $7.90/run) — Anthropic’s workhorse, 100% survival
- Qwen 3.5 9B (bankrupt tier, $0.15/run) — the closest model in parameter count and price
Five Runs, Zero Failures
This is the chart that made us double-check the data. Five runs. Same seed, same prompt, same tools. All five are profitable. All five reach $10,000+ net worth. The worst run ($11,148) would rank above Gemini 3.1 Pro on the leaderboard. The best run ($29,089) approaches Claude Opus 4.6 territory.
| Run | Revenue | Net Worth | ROI | Margin | Waste | Upgrades | Cost |
|---|---|---|---|---|---|---|---|
| 4409 | $60,838 | $29,089 | +1,354% | 49.5% | $3,708 | 8 ($5,150) | $0.21 |
| 0603 | $57,939 | $26,480 | +1,224% | 47.6% | $2,444 | 8 ($5,150) | $0.20 |
| 4358 | $57,209 | $24,878 | +1,144% | 45.7% | $4,675 | 8 ($5,150) | $0.21 |
| 4421 | $53,946 | $23,973 | +1,099% | 48.1% | $2,160 | 8 ($5,150) | $0.20 |
| 0800 | $37,762 | $11,148 | +457% | 31.2% | $1,265 | 6 ($3,650) | $0.18 |
| Average | $53,539 | $23,113 | +1,056% | 44.4% | $2,850 | 7.6 | $0.20 |
The consistency is remarkable. Four runs cluster around $24K–$29K net worth. One outlier (0800) underperforms — it’s the only run that didn’t purchase all 8 upgrades and ran only 5 menu items vs 9–11 in other runs. Even this “worst case” delivers +457% ROI.
Zero bankruptcies. Zero loan defaults. Zero days off across all 5 runs. Total overdraft paid: $2.90 (one run, one day). This is what stability looks like.
The $0.20 Miracle: Cost vs Performance
This is the section that changes how you think about model selection for agentic tasks.
| Model | Cost/Run | Survival | Median ROI | Median NW | Cost Efficiency |
|---|---|---|---|---|---|
| Gemma 4 31B | $0.20 | 100% | +1,144% | $24,878 | $124K per API $ |
| Gemini 3 Pro | $2.95 | 100% | +760% | $17,199 | $5.8K per API $ |
| GPT-5.2 | $4.43 | 100% | +593% | $13,852 | $3.1K per API $ |
| Claude Sonnet 4.6 | $7.90 | 100% | +771% | $17,426 | $2.2K per API $ |
| Claude Opus 4.6 | $36.04 | 100% | +2,376% | $49,519 | $1.4K per API $ |
| Qwen 3.5 397B | $1.07 | 29% | −111% | −$218 | Negative |
| Qwen 3.5 9B | $0.15 | 0% | −128% | −$551 | Bankrupt |
| DeepSeek V3.2 | $0.45 | 62% | −30% | $1,313 | $2.9K per API $ |
| GLM-5 | $1.66 | 29% | −70% | $627 | $378 per API $ |
For every dollar spent on API costs, Gemma generates $124,000 in simulated net worth. GPT-5.2 generates $3,100. Sonnet 4.6 generates $2,200. Gemma is 40–56× more cost-efficient than the next best model.
The Gemini comparison is the most revealing. Google’s own flagship Gemini 3 Pro costs 15× more and delivers 31% less net worth. The open-weight 31B model outperforms the proprietary frontier model from the same company.
Why Does It Work? Decision Analysis
How does a 31B model outperform models with 10× more parameters and 15–40× higher costs? We analyzed the median run’s 30-day decision log. The answer: absence of catastrophic mistakes combined with consistent execution of fundamentals.
Location Strategy: Industrial Zone Concentration
| Location | Visits | Avg Revenue | Avg Profit |
|---|---|---|---|
| Industrial Zone | 16 | $2,003 | $1,136 |
| Waterfront Park | 7 | $2,444 | $1,424 |
| Event Venue | 4 | $1,847 | $836 |
| Downtown Business | 1 | $661 | $35 |
Gemma identified Industrial Zone as the primary profit center and committed. 16 of 28 operating days — 57% concentration. It discovered the Industrial Zone / Waterfront Park dual strategy by Day 6 and never wavered.
Key insight: Location concentration accelerated reputation growth. The simulation’s reputation system rewards consistency with up to +170% demand modifier. Sonnet 4.6 spread visits evenly across 4 locations — more exploration, less compounding.
Upgrade Timing: Aggressive but Funded
| Day | Upgrade | Cost | Cash After |
|---|---|---|---|
| 2 | Kitchen T1 | $500 | $2,306 |
| 5 (day off) | Kitchen T2 | $800 | −$61 |
| 16 | Storage T1 + Equipment T1 + Marketing T1 | $1,100 | $6,202 |
| 18 | Storage T2 | $600 | $5,293 |
| 24 | Equipment T2 + Marketing T2 | $1,150 | $7,652 |
Kitchen T2 on Day 5 is aggressive — it briefly dropped the balance to −$61. But the capacity unlock (300+ customers/day from Day 6) drove exponential revenue growth. Compare: Claude Opus 4.6 starts upgrades on Day 8+ with a $4,500+ cash cushion. Gemma took the risk and it paid off.
Tool Usage: 20–24 Tools, Memory-Heavy
Gemma uses 20–24 of 34 available tools per run, with 462–488 total calls (~15.8/day). Every morning begins with a full information sweep: get_inventory, get_available_locations, get_recipe_catalog, get_sales_history, get_staff_info. Then it acts: location → menu → ingredients → scratchpad.
The memory intensity is notable: 3.5–3.9 memory tool calls per day (write_scratchpad + store_kv). The model actively maintains strategic notes, tracking metrics, identifying problems, and proposing solutions — a genuine agentic feedback loop.
The Food Waste Problem: Gemma’s Achilles Heel
| Run | Revenue | Food Waste | Waste % |
|---|---|---|---|
| 0800 | $37,762 | $1,265 | 3.4% |
| 4421 | $53,946 | $2,160 | 4.0% |
| 0603 | $57,939 | $2,444 | 4.2% |
| 4409 | $60,838 | $3,708 | 6.1% |
| 4358 | $57,209 | $4,675 | 8.2% |
| Average | $53,539 | $2,850 | 5.3% |
Compare with GPT-5.2’s average food waste of $279 (0.8% of revenue) or Sonnet 4.6’s $409 (1.1%). Gemma wastes 7–10× more food than frontier models.
The model knows this. Every scratchpad entry from Day 10 onward mentions waste as a “CRITICAL ISSUE” or “MAJOR ISSUE.” It repeatedly writes plans to “tighten ordering” and “audit inventory.” But the execution is inconsistent — it continues to order in bulk, especially before events.
Agentic insight: Self-awareness without behavioral change is a classic LLM failure pattern. The model’s reasoning correctly identifies the problem and proposes the right solution. It just can’t execute consistently. This is the gap between knowing and doing in agentic systems.
The irony: despite wasting $2,850 per run on average, Gemma still outperforms models that waste almost nothing. Revenue generation more than compensates.
The Stability Story
The most underrated finding: consistency.
| Model | ROI Range | NW Range | Profile |
|---|---|---|---|
| Gemma 4 | +457% to +1,354% | $11K–$29K | Tight band |
| Gemini 3 Pro | +478% to +1,472% | $12K–$31K | Moderate |
| Claude Sonnet 4.6 | +64% to +1,517% | $3K–$32K | Very High variance |
| GPT-5.2 | +155% to +1,372% | $5K–$29K | High variance |
| Grok 4.20 | −58% to +69% | $841–$3,378 | Extreme |
| Qwen 3.5 397B | −135% to −92% | −$708–$162 | Low (consistently bad) |
Gemma’s floor ($11,148, +457% ROI) is higher than every Chinese model’s ceiling and competitive with Gemini 3 Pro’s floor.
Sonnet 4.6 has a wider range — its best ($32,332) beats Gemma’s best ($29,089), but its worst ($3,287) is far worse than Gemma’s worst ($11,148). For production deployments where you can’t afford a bad run, Gemma’s tight band is more valuable than Sonnet’s higher ceiling.
The model doesn’t enter loops. It doesn’t forget its strategy mid-run. It doesn’t suddenly start taking loans it can’t repay. Across all 5 runs: zero loan defaults, zero days off, max 1 loan taken, max 1 staff fired. The most disciplined agent on the leaderboard per dollar spent.
Gemma vs Qwen: The Small Model Showdown
Qwen 3.5 9B is the closest model in the budget tier. The comparison is brutal:
| Metric | Gemma 4 31B | Qwen 3.5 9B | Qwen 3.5 397B |
|---|---|---|---|
| Parameters | 31B | 9B | 397B (17B active) |
| Cost/Run | $0.20 | $0.15 | $1.07 |
| Survival Rate | 100% | 0% | 29% |
| Median ROI | +1,144% | −128% | −111% |
| Median Net Worth | $24,878 | −$551 | −$218 |
| Avg Revenue | $53,539 | $4,068 | $6,994 |
| Avg Profit | $24,323 | −$2,776 | −$2,328 |
| Unique Tools | 20–24 | 9–12 | 10–12 |
| Tool Calls/Run | 462–488 | 136–186 | 170–322 |
Gemma is 3.4× larger and costs 33% more than Qwen 9B. For that premium: 100% vs 0% survival, 13× more revenue, functional business vs guaranteed bankruptcy.
Even Qwen 3.5 397B — a MoE model with 12× more total parameters and 5× higher cost — can’t match Gemma. The gap isn’t incremental. It’s categorical. Gemma 4 operates in a different universe of capability at this size and price.
In Its Own Words
«Chosen Downtown Business District for Day 1 (Mon) due to high office worker traffic and rainy weather. Ordered a balanced stock of ingredients for burgers, tacos, burrito bowls, fries, and lemonade.»
— Day 0 scratchpad
Rational first move. Weather-aware, audience-aware. Most bankrupt models don’t even check weather on Day 0.
«Industrial Zone is a goldmine for demand, but I’m bleeding money on waste.»
— Day 12
The most honest self-assessment on the leaderboard. It found the gold mine. It knows it’s leaking. It keeps mining anyway — and it works.
«Day 21 was a disaster. Lost $2,359 to food waste — this is unsustainable and a critical failure.»
— Day 21 reflection
Worst day of the run. Then it recovered to $1,683 profit the next day. Resilience in action.
«$15 Classic Burger. The market accepted it.»
— Day 28
Peak price discovery moment. $4,961 revenue — the highest single-day revenue in the entire run. $15 burgers at Waterfront Park.
Verdict
Gemma 4 31B is the most important model on the FoodTruck Bench leaderboard — not because it’s the best performer (Claude Opus 4.6 holds that crown), but because it redefines what’s possible at this price point and scale.
A 31-billion-parameter open-weight model, running at $0.20 per 30-day simulation, outperforms Google’s own Gemini 3 Pro (15× more expensive), OpenAI’s GPT-5.2 (22× more expensive), and Anthropic’s Claude Sonnet 4.6 (40× more expensive). It does this with 100% survival rate, no bankruptcies, no loan defaults, and the tightest ROI variance band on the leaderboard.
The model isn’t perfect. It wastes 5–10× more food than frontier competitors. It correctly identifies this problem in its scratchpad — “CRITICAL ISSUE,” “MAJOR ISSUE,” “must tighten ordering” — but can’t consistently fix it. Self-awareness without behavioral change remains the gap between observation and execution in agentic AI.
But here’s the thing: it doesn’t matter. The revenue generation from aggressive location concentration, early upgrade investment, and strong pricing strategy more than compensates for the waste. Gemma operates like a high-revenue startup that hasn’t optimized its operations yet — messy execution, excellent results.
For practitioners choosing models for agentic workloads, this is the finding that should change your calculus: you don’t need a $5–$30 frontier model for complex multi-step tasks. A 31B open-weight model at $0.20/run — downloadable, self-hostable, fine-tunable — delivers results that compete with or exceed models 100× its cost.
We double-checked the results. They’re real. We’re genuinely shocked.
This is one person’s analysis based on one benchmark. Your mileage — and your use case — may vary.