GPU memory estimator
Estimate weights, KV cache, and full-training states before scheduling a run.
Study guide & lesson outline
LEARNING CONTRACTUnderstand → implement → investigate
Goal: explain the mechanism in this lesson, reproduce one example and test a failure case. Reading alone does not mark mastery.
- Understand: state assumptions, shapes/units and a counterexample.
- Implement: reproduce a numerical example and inspect boundaries.
- Investigate: compare one alternative under a matched budget.
01Configure
02Add your input
Choose settings, load an example, or paste your own input.
03Inspect results
Choose settings, load an example, or paste your own input.
The display may be truncated; the exported JSON report contains all computed results.
How to interpret this
Single-device planning estimate. Inference uses ideal packed weight storage and a dense KV cache; quantization metadata, allocator behavior, kernels, and activations add overhead. Full training uses a conventional 18 bytes/parameter mixed-precision AdamW baseline, plus your activation/runtime allowances. This is not a prediction for every framework, LoRA, sharding, offload, or measured peak VRAM.
GiB = 2³⁰ bytes. Training states = parameters × 18 bytes. KV = 2 × layers × KV heads × head dimension × sequence × batch × bytes. Safety margin multiplies the subtotal, including allowances.
Method references
Assumptions, implementation & acceptance
The 18-byte training baseline is 2 working-weight + 4 master-weight + 4 gradient + 8 Adam-moment bytes per parameter. This is one dtype layout, not a universal framework constant or measured peak.
Reviewed counting units, statistical assumptions and numerical examples. Provider accounting and actual GPU allocation still require external measurement.
Before running: predict a boundary case
The 18-byte training baseline is 2 working-weight + 4 master-weight + 4 gradient + 8 Adam-moment bytes per parameter. This is one dtype layout, not a universal framework constant or measured peak.
Hint: isolate the assumption
Keep the baseline input fixed, change one parameter or boundary, and compare both the numerical result and the claimed units. A passing example verifies this fixture only.
Minimal executable example
Save the example and linked modules in one folder. Use Node.js 22 or newer, then run node vram-calculator.mjs. npm install gpt-tokenizer@4.1.0
import {encode as rawEncode} from 'gpt-tokenizer/encoding/cl100k_base';
import {promptDiff,inspectDataset,chunkDocument,compareScores,estimateVRAM,samplingLab,speculativeLab} from './research-core.mjs';
const encode=s=>rawEncode(s,{disallowedSpecial:new Set()});
const result=estimateVRAM({mode:'inference',params:7,weightBits:16,layers:32,kvHeads:8,headDim:128,sequence:4096,batch:1,kvBytes:2,runtime:2,activation:4,margin:15,capacity:24});
console.log(result.components.cache); // 0.5 GiBTest and transfer
The 18-byte training baseline is 2 working-weight + 4 master-weight + 4 gradient + 8 Adam-moment bytes per parameter. This is one dtype layout, not a universal framework constant or measured peak.
Record input, code version, expected result and observed result in the notebook below. Then construct a counterexample that violates the assumption above and explain why the original conclusion no longer follows.
Download complete kit (.tar.gz) · Complete reference kit and regression checksPredict, test and keep evidence
Before revealing an answer, predict one result. After the exercise, record the discrepancy and a transfer test. Saved locally across language editions.
Open notebook / export / clear →The mathematical review scope is recorded per lesson. It does not establish empirical paper reproduction; report suspected errors in your notes. Review scope