샘플링 실험: logits → 토큰
재현 가능한 로컬 분포에서 temperature, top-k, top-p를 테스트합니다.
Study guide & lesson outline
LEARNING CONTRACTUnderstand → implement → investigate
Goal: explain the mechanism in this lesson, reproduce one example and test a failure case. Reading alone does not mark mastery.
- Understand: state assumptions, shapes/units and a counterexample.
- Implement: reproduce a numerical example and inspect boundaries.
- Investigate: compare one alternative under a matched budget.
01설정
02입력 추가
설정 후 예제를 불러오거나 입력을 붙여 넣으세요.
03결과 확인
설정 후 예제를 불러오거나 입력을 붙여 넣으세요.
화면은 일부만 표시할 수 있으며 JSON 보고서는 전체 계산 결과를 포함합니다.
결과 해석
CSV는 token,logit이며 라벨은 고유하고 비어 있지 않아야 합니다. logit은 −10,000~10,000의 유한 수입니다. 온도, top-k, top-p 순서로 적용하고 필터마다 재정규화합니다. 온도 0은 탐욕 선택이며 동점은 입력 순서입니다. 고정 시드 범주형 샘플러로, 실제 모델이 아닙니다. 최대 100후보와 10,000회.
방법 참고 자료
Assumptions, implementation & acceptance
Filtering order is temperature, top-k, then top-p and renormalization. Entropy is in bits. Temperature zero is explicitly greedy; stochastic draws depend on the seed.
Reviewed counting units, statistical assumptions and numerical examples. Provider accounting and actual GPU allocation still require external measurement.
Before running: predict a boundary case
Filtering order is temperature, top-k, then top-p and renormalization. Entropy is in bits. Temperature zero is explicitly greedy; stochastic draws depend on the seed.
Hint: isolate the assumption
Keep the baseline input fixed, change one parameter or boundary, and compare both the numerical result and the claimed units. A passing example verifies this fixture only.
Minimal executable example
Save the example and linked modules in one folder. Use Node.js 22 or newer, then run node sampling-lab.mjs. npm install gpt-tokenizer@4.1.0
import {encode as rawEncode} from 'gpt-tokenizer/encoding/cl100k_base';
import {promptDiff,inspectDataset,chunkDocument,compareScores,estimateVRAM,samplingLab,speculativeLab} from './research-core.mjs';
const encode=s=>rawEncode(s,{disallowedSpecial:new Set()});
const result=samplingLab('token,logit\na,0\nb,0',1,0,1,10000,42);
console.log(result.entropy); // 1 bit; observed frequencies only approximate 0.5Test and transfer
Filtering order is temperature, top-k, then top-p and renormalization. Entropy is in bits. Temperature zero is explicitly greedy; stochastic draws depend on the seed.
Record input, code version, expected result and observed result in the notebook below. Then construct a counterexample that violates the assumption above and explain why the original conclusion no longer follows.
Download complete kit (.tar.gz) · Complete reference kit and regression checksPredict, test and keep evidence
Before revealing an answer, predict one result. After the exercise, record the discrepancy and a transfer test. Saved locally across language editions.
Open notebook / export / clear →The mathematical review scope is recorded per lesson. It does not establish empirical paper reproduction; report suspected errors in your notes. Review scope