A Local LLM on a $9 VPS: 9 Tokens a Second, No Matter How Many Users

Published 6 September 2026. Measured on one live server we operate — 2 vCPU, 7.8 GiB, Ubuntu 24.04, no GPU — already running five production sites, by deploying Ollama, benchmarking two models, watching what it did to the sites, and removing it. We did not call the OpenAI API for this article and are not quoting prices we have not paid; what we measured is the local half, which is the half that decides.

1. The Ceiling Is Nine Tokens a Second, and It Does Not Move

Everything else in this article is detail. A 3-billion-parameter model on two virtual cores produced between 6.8 and 8.7 tokens per second depending on how long the answer ran, and a 1-billion model was barely faster at 9.1. A 300-token reply took 43 seconds.

Two things about that table are worth pausing on.

Time to first token is excellent — under a tenth of a second, because the model is already resident in memory and there is no network hop. That is the one metric where a local model beats a hosted API outright, and it is the metric nobody complains about.

The 1B model is only 5% faster than the 3B. Tripling the parameter count cost almost nothing, which tells you the bottleneck is not arithmetic. On two cores with this much memory bandwidth, both models are waiting on the same thing, and shrinking the model does not buy back speed.

2. Concurrency Is Where the Idea Dies

A micro-SaaS has more than one user. We ran identical 100-token generations at increasing concurrency and measured total tokens against wall-clock time:

Total throughput is flat. The server produces about nine tokens per second and that is all it produces, whether one person is waiting or four. Every additional user does not get a slower share of a bigger pie — the pie is fixed, and the queue simply gets longer.

Four concurrent users means a 43-second wait for a short answer. That is not a slow product; it is a broken one.

3. What It Costs the Rest of the Machine

The server was not idle. It runs five production sites, and we measured them while the model was generating:

Roughly double, on both. Inference pins every core it is allowed to touch — we watched the container sit at 150% of its 1.5-core limit for the entire generation. Capping it helps the sites and costs the model: restricted to 1.5 cores it produced 6.8 tokens per second against 8.7 unrestricted, so giving up a third of the CPU cost 22% of an already unusable speed.

There is no setting that makes this coexist well. Either the model is slow and your sites are fine, or the model is slightly less slow and your sites are twice as slow.

4. The Disk Bill Nobody Mentions

The image is 9 GB because it ships CUDA and ROCm runtimes regardless of whether you have a GPU. On a 50 GB VPS that is a quarter of the disk spent on GPU libraries that will never execute. Memory is more reasonable: 92 MiB idle, and 3.96 GiB with the 3B model resident.

Note the RAM figure against the model size. A 2.0 GB model occupied nearly 4 GiB once loaded, so size on disk is not the number to plan around — budget roughly double.

5. Comparing It to an API Without Pretending

We have no OpenAI account and did not open one to write this, so there are no latency or cost figures for the hosted side that we can stand behind. What we can do is state this server’s capacity precisely, so you can apply whatever the current rates are yourself.

Sustained output      :        9 tokens/second
Per hour              :   32,400 tokens
Per day, at 100% duty :  777,600 tokens
Per month             : ~23 million tokens

That monthly figure is the honest ceiling, and it is also fictional, because running inference continuously doubles the response time of everything else on the machine. A realistic duty cycle on a shared VPS is a fraction of that.

The structural differences matter more than the arithmetic:

The break-even question is usually framed as “how many tokens before the VPS is cheaper”. On a CPU-only server that framing is wrong, because the VPS cannot reach the volume where the comparison becomes interesting. You hit the capacity wall long before you hit the cost line.

6. When a Local Model on a Cheap VPS Does Make Sense

Nine tokens per second is unusable for a chat interface, where a reader consumes about 20 to 30 tokens per second and anything slower feels broken. It is perfectly adequate for work nobody is watching:

• Batch classification and tagging. Labelling a queue of support tickets overnight does not care that each one takes eight seconds.
• Short structured extraction. Pulling three fields out of an email is thirty tokens of output, not three hundred.
• Anything with a privacy constraint. If the data legally cannot leave your infrastructure, throughput stops being the deciding factor.
• Embeddings. A different workload entirely, far cheaper per item than generation, and genuinely practical on CPU.

The pattern that works is a queue, not a request. Accept the job, return immediately, process in the background, notify when done. That design tolerates nine tokens a second; a chat box does not.

docker run -d --name ollama \
  --cpus="1.5" --memory="4g" \
  -p 127.0.0.1:11434:11434 \
  -v ollama-models:/root/.ollama \
  ollama/ollama:latest

# Loopback only. Docker's iptables rules are consulted before UFW, so a
# port published on all interfaces is reachable from the internet even
# when the firewall says otherwise — and this one answers prompts.

7. Frequently Asked Questions

How fast is Ollama on a CPU-only VPS?

We measured 8.7 tokens per second for llama3.2:3b and 9.1 for the 1B model on two virtual cores with no GPU. A 300-token answer took 43 seconds. Time to first token was excellent at under 0.1 second, because the model is already resident and no network hop is involved.

Can a local LLM handle multiple users on one VPS?

No. Total throughput stayed flat at around nine tokens per second from one to four concurrent requests, so users queue rather than share. One user waited 10.7 seconds for a short answer; four waited 43.1 seconds each. The server’s capacity is fixed and additional users only lengthen the queue.

Does a smaller model run much faster on CPU?

Barely. llama3.2:1b managed 9.1 tokens per second against 8.7 for the 3B model — a 5% gain for a third of the parameters. The bottleneck on two cores is not arithmetic, so shrinking the model costs quality without buying back meaningful speed. Reach for a smaller model to fit memory, not to go faster.

How much disk and RAM does Ollama need?

The image alone is 9.19 GB because it ships CUDA and ROCm runtimes even without a GPU, plus 1.3 GB and 2.0 GB for the two models we pulled — about 12 GB total. Memory was 92 MiB idle and 3.96 GiB with the 3B model loaded, so budget roughly double the model’s size on disk.

Will running Ollama slow down my websites?

Yes, measurably. During inference our WordPress site went from 25 ms to 50 ms and a Django site from 41 ms to 92 ms, roughly double for both. Inference saturates every core it can reach. Capping the container to 1.5 cores protects the sites and costs 22% of the model’s already low speed.

8. The Short Version

• Nine tokens per second, and it does not improve with fewer parameters or more users.
• Total throughput is flat under concurrency; four users each wait 43 seconds.
• Inference doubles the response time of everything else on the server.
• 12 GB of disk, of which 9 GB is GPU runtimes you will not use.
• Excellent for background queues and privacy-bound work. Unusable behind a chat box.

For what else this hardware sustains, our capacity model across seven applications puts these figures beside the rest, and the Uptime Kuma benchmark covers the same measure-then-remove method on a workload that does fit comfortably.