Google shipped four multimodal models under Apache 2.0, and the developer ecosystem ported them everywhere within hours.
Google finally did the right thing.
- They did not just slap a new version number on an old architecture.
- They dropped an entire family of four multimodal models, ranging from 2.3 billion effective parameters up to a 31 billion dense heavyweight.
But the architecture is only half the story :)
The real news is that Google released all of them under a clean Apache 2.0 license.
The open-source community reacted instantly.
Within hours of the launch, there were 96 MLX variants for Apple Silicon, browser-ready ONNX checkpoints, and full GGUF support.
So, what is actually inside Gemma 4, what the benchmarks mean for production, and why that 26B Mixture-of-Experts model is the only one you should care about.
Four Models, Two Different Architectures
Google shipped two fundamentally different designs across four size classes.
- The Edge Models (E2B and E4B): The “E” stands for effective parameters. The E2B has 2.3 billion effective parameters, and the E4B has 4.5 billion.
These are built for local hardware. Both support 128k context windows and process text, images, and audio. The audio is not bolted on; these models have a USM-style conformer audio encoder baked natively into the architecture.
- The Heavyweight (31B Dense): This is a 31 billion parameter dense model with a 256k context window. It handles text, images, and video.
It scores 89.2% on AIME 2026 and pulls a 2150 Codeforces ELO. It is a beast if you have the VRAM to run it.
- The Efficiency Play (26B MoE): This is the masterpiece of the release. It is a Mixture-of-Experts model with 26 billion total parameters but only 4 billion active per token. It scores an LMArena rating of 1441 compared to the dense 31B’s 1452.
You get 95% of the heavyweight performance using a fraction of the compute.
Architecture Upgrades That Move the Needle
Gemma 4 introduces a few clever engineering tricks that optimize memory and context handling.
Per-Layer Embeddings (PLE): Instead of a single embedding lookup that dilutes as it passes through the transformer layers, Gemma 4 maintains a second embedding table.
This provides residual signals directly to every decoder layer, maintaining token identity deep into the network.
Shared KV Cache: To optimize memory, the final layers reuse key-value tensors from the last non-shared layer instead of computing their own.
When you are pushing a 256k context window, saving every byte of KV cache is critical.
Dual RoPE: The model alternates between standard Rotary Position Embeddings for local sliding-window attention and proportional RoPE for global attention.
This is how Gemma 4 handles massive context without quadratic compute costs destroying your hardware.
The Benchmark Reality Check
The numbers are highly impressive, but context is required.
The 31B dense model is highly competitive, but the top-tier Chinese models like Qwen 3.5 and DeepSeek still slightly edge it out on aggregate leaderboards.
If you need the absolute peak of open-weight performance, Google is still playing catch-up :)
However, the 26B MoE changes the calculus for developers.
Running a dense 31B model in production is expensive.
Getting comparable performance out of 4 billion active parameters completely shifts your inference unit economics.
A quick warning on the context windows: just because a model accepts 256k tokens does not mean it can recall them perfectly.
The 31B model scores well on Needle In A Haystack tests at 128k, but the MoE and smaller models drop significantly in accuracy at high context lengths.
Apache 2.0: The Ultimate Feature
The most important part of Gemma 4 is not the code. It is the legal text attached to it.
Previous Gemma models had custom licenses, Acceptable Use Policies, and commercial limits.
Legal departments hate custom licenses.
By releasing Gemma 4 fully under Apache 2.0, Google removed all the friction.
Like there are no commercial limits and no strings attached.
For enterprise teams and startup founders, Gemma 4 is now legally safe to deploy anywhere :)
So, Google did not win the benchmark crown with Gemma 4.
But they won the deployment race.
By shipping a highly efficient MoE model, dropping the restrictive licensing, and enabling the open-source infrastructure layer to do what it does best, Google just handed developers the most complete model family of 2026.
In case we are meeting for the first time, come over here, it’ll be worth the roller coaster of articles that are gonna come up in the next few weeks.
I swear tracking these updates is a job in itself, lately.
Here’s the list which I’ve built and keep adding on.
And If you need help for analyzing UFC fights, here you go.