Ever wondered when open-source models would finally catch up to the big proprietary players? Matthew Berman just dropped a video breaking down Google’s Gemma 4 release, and his enthusiasm is well-founded. This isn’t just another incremental update — it’s a statement about where local AI is headed.
In this post, we’ll walk through what makes Gemma 4 special, why the size-to-performance ratio matters so much, and what this means for anyone interested in running powerful AI on their own hardware.
The Chart That Tells the Story
Matthew opens with a chart that immediately captures why Gemma 4 is generating buzz. On the x-axis: total model size in billions of parameters. On the y-axis: ELO score from Arena AI. The goal is to be as high and as far left as possible — maximum performance with minimum size.
Gemma 4 nails it.
The 31B dense model and the 26B mixture-of-experts variant both score comparably to Qwen 3.5. Here’s the catch: Qwen 3.5 is a 397 billion parameter model with 17 billion active parameters. That’s a massive model requiring specialized hardware that most people simply don’t have access to.
Meanwhile, Gemma 4’s 31B model delivers similar performance at a fraction of the size. As Matthew puts it: “You can run this 31 billion parameter model locally. You can do it on most medium to high-end normal consumer hardware.”
That’s the paradigm shift. Frontier-level intelligence, running on hardware you might already own.
Four Models, Four Use Cases
Google released the Gemma 4 family in four sizes, each optimized for different deployment scenarios:
E2B (Effective 2 Billion) — The smallest variant, designed for mobile and IoT devices. The “effective” terminology refers to a technique called per-layer embeddings (PLE), which gives each decoder layer its own small embedding for every token. The embedding tables are large but only used for quick lookups, making the effective parameter count much smaller than the total.
E4B (Effective 4 Billion) — A step up from E2B while still prioritizing edge deployment. Both effective models feature native audio input for speech recognition and understanding — these are genuinely multimodal, not just vision-capable.
26B Mixture of Experts — This variant activates only 4 billion parameters during inference while having access to 26 billion total. The MoE architecture provides excellent latency characteristics while maintaining quality.
31B Dense — The flagship model, currently ranking as the #3 open model on Arena AI’s text leaderboard. When you need maximum quality and have the hardware to support it, this is the one.
What These Models Actually Do Well
Matthew breaks down the key capabilities, and they map directly to what matters for practical applications:
Advanced Reasoning: Multi-step planning and deep logic. Significant improvements on math benchmarks and instruction-following tasks. If you’ve been frustrated by smaller models losing the thread on complex problems, this is the target Google was aiming at.
Agentic Workflows: Native support for function calling, structured JSON output, and system instructions. You can build autonomous agents that interact with tools and APIs without wrestling with prompt engineering hacks. The model understands what it means to be an agent from the architecture level.
Matthew highlights a particularly impressive data point: Steve Vibe ran Tool Call 15 benchmarks across all four Gemma 4 models, and the 31B version scored perfect marks on tool calling. For anyone building agent-based systems, that’s a significant validation.
Code Generation: Google positions Gemma 4 as capable of turning “your workstation into a local-first AI code assistant.” Matthew offers an honest perspective here: if you’re doing serious coding, you’re probably still reaching for frontier hosted models like Claude Opus 4.6 or GPT-5.4. But for many use cases — autocomplete, simple refactoring, code review — local models are increasingly viable.
Multimodal Understanding: All models natively process video and images with variable resolution support. They excel at visual tasks like OCR and chart understanding. The edge models (E2B/E4B) add native audio input, making them genuinely multimodal across text, vision, and speech.
The Context Window Reality Check
Matthew does flag one area where he was hoping for more: context windows. The edge models support 128K tokens, which is reasonable for on-device use cases. But the larger models top out at 256K.
For many applications, 256K is plenty. You can fit substantial codebases, long documents, or extended conversation histories. But if you were hoping for million-token context windows to match some frontier models, that’s not here yet.
Edge Computing Gets Real
The E2B and E4B models represent something Matthew has been predicting for a while: open-source models getting smaller, better, and faster simultaneously.
Google collaborated with their Pixel team, Qualcomm Technologies, and MediaTek to optimize these models for real edge hardware. They run completely offline with near-zero latency on phones, Raspberry Pi, and NVIDIA Jetson Orin Nano.
The implications are significant. You can deploy genuinely capable AI on devices that don’t have internet connectivity. Privacy-sensitive applications become feasible. Latency drops to near-zero because you’re not making round trips to cloud servers.
Matthew makes a prediction: “We might start seeing them in Apple devices too.” Given Apple’s focus on on-device ML and their relationship with open models, that’s not a stretch.
The Benchmark Rundown
For those who want the numbers:
- Arena AI Text: 1452 ELO (31B model)
- MMLU (Multilingual): 85.2%
- AMY 2026: 89%
- LiveCode Bench: 80%
- T2 Bench: 86%
- GPQA Diamond: 84.3%
These are strong results for a 31B model. The comparison point Matthew emphasizes repeatedly: you’re getting performance comparable to models that are 10-20x larger.
Apache 2.0 Makes It Real
Gemma 4 ships under the Apache 2.0 license — commercially permissive with no restrictive barriers. You can use these models in production, fine-tune them for your specific use cases, and deploy them however you want.
This matters for anyone building products. Previous Gemma releases had more restrictive terms. Apache 2.0 removes the legal ambiguity that makes enterprise deployments complicated.
Where to Get Started
The models are available across the ecosystem:
- Hugging Face for raw weights and Transformers integration
- Ollama for one-line local deployment
- LM Studio for a polished desktop experience
- llama.cpp for optimized inference
- MLX for Apple Silicon
- NVIDIA NIM for production serving
- Unsloth for efficient fine-tuning
Matthew’s recommendation: download it, start playing with it, start fine-tuning it. The barrier to entry for running capable local models just dropped significantly.
The Bigger Picture
What makes Gemma 4 notable isn’t just the benchmarks — it’s what it represents for the trajectory of AI accessibility.
Matthew has been vocal about his vision for “hybrid” AI: using fully hosted frontier models for the hardest tasks while handling the vast majority of workloads on local hardware. Gemma 4 makes that vision more practical.
You don’t need a GB300 with 750GB of unified memory to run competitive models anymore. Consumer hardware — a good GPU, maybe even integrated graphics for the smaller variants — gets you into the game.
For developers, researchers, and businesses who want AI capabilities without per-token API costs or data leaving their infrastructure, that’s a meaningful shift.
The open-source AI ecosystem keeps accelerating. Google deserves credit for pushing the frontier here consistently. As Matthew notes: “Not every company is doing that, and not every company is doing it as consistently as Google is.”
Gemma 4 is proof that open models aren’t just catching up — they’re competing directly with proprietary offerings at a fraction of the resource requirements. The question isn’t whether to try local AI anymore. It’s which model fits your hardware and use case.
Time to start experimenting.