The model you pick decides most of your cost. Bigger models work
better but need more memory.
Not sure? Llama 3.1 70B is a common choice that balances quality and
cost.
What the model needs (from its maker):
We get these details from the model's public specs. You can change
them in step 3 if you know better. (The AI holds some memory while
it works — the calculator uses these details to work that out.)
How precise should the AI be?
This sets how the AI stores numbers. More precise means better results
but more memory.
Step 2 of 5 — Prompts and responses
How long are your prompts and responses?
The full length is the context window: your prompt plus the response. This length drives how much memory the
AI holds while it works — longer prompts and responses need more memory
per person.
A token is a short piece of text — about 4 characters. A 1,000-word
document is roughly 1,333 tokens.
Total length = 4,500 tokens(your prompt plus the response — this is what the AI holds in memory)
Your system instructions, the user's message, and any documents you
include.
4,000
100128,000 (max Llama 3.1)
How long the AI's reply typically is. A short answer ≈ 100–300
tokens; a full report ≈ 1,000–4,000.
500
Step 3 of 5 — Concurrent users
How many concurrent users will use the AI at peak?
Size for your busiest moment, not your average. Your true peak is
derived from your average requests per second.
If you don't plan for peak use, each prompt can cost you more.
How many people send a request at the same moment during your
busiest hour.
10
Optional but useful: we use this to work out your peak load and
your cost per prompt.
Enter your requests per second to see your estimated peak.
These details come from the model itself and feed the memory
formula. The values are pre-filled from the model's public specs.
If your model gives a KV head count directly, enter it here.
Same as attention heads for standard models; 1 for the simplest
design.
Step 4 of 5 — Reply speed
How fast does the AI need to respond?
Two speeds affect your hardware. Tighter limits mean the system
handles fewer people at once.
Time to first word (TTFT) — how
long until the first word shows up. Total time for the reply (TTLT) — from sending the prompt to getting the whole response.
⚠️ Tighter speed limits cut how much the system can do at once. These
are planning targets — check your GPU's real benchmarks before you
buy.
Don't guess your hardware needs.
Calculators give you targets, but real-world performance varies. Get
certainty before you buy.