Estimador de memoria GPU
Desglosa pesos, caché KV y estados de entrenamiento antes de ejecutar.
Study guide & lesson outline
LEARNING CONTRACTUnderstand → implement → investigate
Goal: explain the mechanism in this lesson, reproduce one example and test a failure case. Reading alone does not mark mastery.
- Understand: state assumptions, shapes/units and a counterexample.
- Implement: reproduce a numerical example and inspect boundaries.
- Investigate: compare one alternative under a matched budget.
01Configurar
02Añadir entrada
Configura los parámetros, carga un ejemplo o pega tus datos.
03Revisar resultados
Configura los parámetros, carga un ejemplo o pega tus datos.
La vista puede estar truncada; el JSON exportado incluye todos los resultados calculados.
Cómo interpretar los resultados
Estimación de un dispositivo. La inferencia usa almacenamiento ideal de pesos y caché KV densa; metadatos de cuantización, kernels, activaciones y asignadores añaden coste. El entrenamiento completo usa 18 bytes/parámetro como referencia AdamW de precisión mixta, más tus reservas. No modela todos los frameworks, LoRA, particionado, offload ni picos medidos.
GiB = 2³⁰ bytes. Estados = parámetros × 18 bytes. KV = 2 × capas × cabezas KV × dimensión × secuencia × lote × bytes. El margen multiplica el subtotal con reservas.
Referencias del método
Assumptions, implementation & acceptance
The 18-byte training baseline is 2 working-weight + 4 master-weight + 4 gradient + 8 Adam-moment bytes per parameter. This is one dtype layout, not a universal framework constant or measured peak.
Reviewed counting units, statistical assumptions and numerical examples. Provider accounting and actual GPU allocation still require external measurement.
Before running: predict a boundary case
The 18-byte training baseline is 2 working-weight + 4 master-weight + 4 gradient + 8 Adam-moment bytes per parameter. This is one dtype layout, not a universal framework constant or measured peak.
Hint: isolate the assumption
Keep the baseline input fixed, change one parameter or boundary, and compare both the numerical result and the claimed units. A passing example verifies this fixture only.
Minimal executable example
Save the example and linked modules in one folder. Use Node.js 22 or newer, then run node vram-calculator.mjs. npm install gpt-tokenizer@4.1.0
import {encode as rawEncode} from 'gpt-tokenizer/encoding/cl100k_base';
import {promptDiff,inspectDataset,chunkDocument,compareScores,estimateVRAM,samplingLab,speculativeLab} from './research-core.mjs';
const encode=s=>rawEncode(s,{disallowedSpecial:new Set()});
const result=estimateVRAM({mode:'inference',params:7,weightBits:16,layers:32,kvHeads:8,headDim:128,sequence:4096,batch:1,kvBytes:2,runtime:2,activation:4,margin:15,capacity:24});
console.log(result.components.cache); // 0.5 GiBTest and transfer
The 18-byte training baseline is 2 working-weight + 4 master-weight + 4 gradient + 8 Adam-moment bytes per parameter. This is one dtype layout, not a universal framework constant or measured peak.
Record input, code version, expected result and observed result in the notebook below. Then construct a counterexample that violates the assumption above and explain why the original conclusion no longer follows.
Download complete kit (.tar.gz) · Complete reference kit and regression checksPredict, test and keep evidence
Before revealing an answer, predict one result. After the exercise, record the discrepancy and a transfer test. Saved locally across language editions.
Open notebook / export / clear →The mathematical review scope is recorded per lesson. It does not establish empirical paper reproduction; report suspected errors in your notes. Review scope