Token LabSearch

← Back to tools
Local only · No model calls

Less debugging. More discovery.

Seven local tools for AI engineers, researchers, and PhD students. Check data, compare experiments, and test sampling or decoding hypotheses with explicit assumptions.

Tokenizer · Context budget · Payload inspector →

01

Prompt diff & token impact

Compare two prompt versions line by line and measure their token and UTF-8 byte changes.

Open tool →
02

JSONL dataset inspector

Find malformed records, repeated samples, long inputs, and exact train/validation overlap.

Open tool →
03

RAG chunking lab

Explore Unicode-safe chunk boundaries, overlap, and exact text-token counts.

Open tool →
04

Paired evaluation comparison

Compare two systems on the same examples with a reproducible bootstrap interval.

Open tool →
05

GPU memory estimator

Estimate weights, KV cache, and full-training states before scheduling a run.

Open tool →
06

Sampling lab: logits → tokens

Test temperature, top-k and nucleus sampling against a reproducible local distribution.

Open tool →
07

Speculative decoding lab

Explore when a draft model helps—or hurts—under explicit acceptance and latency assumptions.

Open tool →