Why have useful language models become practical on more phones?

Hardware, memory and model design all matter

Running a language model on a phone depends on several factors: hardware that supports its operations, enough memory and bandwidth, and a model that fits the device and performs adequately for the task. The balance differs across phones and models.

Hardware and memory capabilities, model quality, and compression have changed over time, but their relative contributions vary. A model’s performance on older hardware depends on its runtime and the device’s available resources.

• Accelerators: some phone chips include dedicated hardware for selected machine-learning operations; availability and support vary by device.
• Bandwidth: memory bandwidth affects how quickly a resident model can generate text.
• Models: training, distillation and compression can make models smaller or more useful for a given task.

Models, hardware and software all contribute

Research on model scaling has explored how model size and training data interact. Smaller models can perform useful tasks when training, model design and workload align, but this does not establish a single timeline or cause for broader on-device use.

The rest is craft, and it compounds. Distillation trains a small model against a larger one's outputs. Careful data filtering raises quality per parameter. Attention variants that share key and value projections across groups of heads shrink the memory a conversation occupies, which is what limits how long a chat can run on a device. Compressing the weights themselves cuts the size again on top of all of it.

Dedicated accelerators, memory systems, model training and compression have all developed over time. Their effects overlap, and the importance of each depends on the phone, runtime and model. A single software or hardware curve does not explain every case.

So what does "possible" still depend on?

Mostly memory. A phone's RAM is shared with the operating system and every other running app, and only a portion is available to one of them. The practical ceiling is a compressed model's size plus the space its conversation history takes as that history grows, a cost that compounds with every reply rather than one paid up front.

Then heat. A phone can run flat out in bursts, but a sustained load warms it until the system throttles the chip to protect it, so a short answer finishes inside the burst and a long generation may not. And the task still matters: a model small enough to fit can be reliable at bounded, well-defined work and still be out of its depth on open-ended reasoning.