How can a language model answer you with no internet connection?

A trained model is a file, not a service

Training is the expensive part, and it produces exactly one artifact: a large set of numbers, called weights or parameters, written to disk. When training stops, that file is finished. Running it does not change it, and ten thousand conversations leave it byte for byte identical. It is closer to a font file or a chess engine’s opening tables than to an account on a website.

How big the file is depends on two things: how many parameters the model has, and how many bits each parameter is stored in. As a general rule, at four bits per parameter a model needs roughly half a byte per parameter, so a few billion parameters lands in the low gigabytes. That is the scale of a long video, not the scale of a data center.

Answering is arithmetic, not a lookup

Your words are cut into tokens, and each token is turned into a list of numbers. Those numbers are multiplied through the model’s fixed weight matrices, layer by layer, until the last layer produces a probability for every token the model knows. One token is chosen, appended to the text so far, and the whole pass runs again for the next one.

That loop is the entire act of answering. Nothing in it opens a socket, queries a database or consults an index, because there is no index. What the model knows is not a stored copy of documents; it is the arrangement of those numbers, shaped once by training. With no index and no network, the model has nothing outside itself to check an answer against, which is why a fluent wrong answer and a fluent right one cost it exactly the same effort. Why that happens is its own question, linked below.

Then why do so many AI apps need a connection?

Because the file is on someone else’s computer. The network is how your question reaches the hardware doing the arithmetic, and how the result gets back. It carries the question; it does not do any part of the thinking. The reason for that arrangement is practical rather than mysterious: the largest models need far more memory than a phone has, and shared accelerators are cheaper to rent out than to give away.

Some features really do need a connection, and it helps to be exact about which. Live information is one. A model cannot know today’s weather, today’s prices or the message that arrived a minute ago, wherever it runs, because none of that was in the file. A connected app fetches those facts and hands them to the model; the network supplies the information, not the reasoning. Move the same file onto your own device instead, and the network drops out of the picture entirely. The arithmetic does not change — only the hardware running it does, and what that trade costs is its own question, linked below.

• Live facts: weather, prices, arrival times, anything that has to be true this minute.
• Other people’s data: a shared calendar, a mailbox, a document held on a server.
• A model too large for the device, which then has to run on a machine that can hold it.