サンプリング実験：logitsからトークンへ
temperature、top-k、top-pを再現可能なローカル分布で試します。
Study guide & lesson outline
LEARNING CONTRACTUnderstand → implement → investigate
Goal: explain the mechanism in this lesson, reproduce one example and test a failure case. Reading alone does not mark mastery.
- Understand: state assumptions, shapes/units and a counterexample.
- Implement: reproduce a numerical example and inspect boundaries.
- Investigate: compare one alternative under a matched budget.
01設定
02入力を追加
設定して例を読み込むか、データを貼り付けてください。
03結果を確認
設定して例を読み込むか、データを貼り付けてください。
表示を省略する場合がありますが、JSONレポートは全計算結果を含みます。
結果の読み方
CSVはtoken,logit。ラベルは一意で空でなく、logitは−10,000〜10,000の有限値です。温度、top-k、top-pの順に適用し、各段階で再正規化します。温度0は貪欲選択で、同点は入力順です。固定シードのカテゴリカル標本化であり、実モデルではありません。最大100候補、10,000回。
手法の参考資料
Assumptions, implementation & acceptance
Filtering order is temperature, top-k, then top-p and renormalization. Entropy is in bits. Temperature zero is explicitly greedy; stochastic draws depend on the seed.
Reviewed counting units, statistical assumptions and numerical examples. Provider accounting and actual GPU allocation still require external measurement.
Before running: predict a boundary case
Filtering order is temperature, top-k, then top-p and renormalization. Entropy is in bits. Temperature zero is explicitly greedy; stochastic draws depend on the seed.
Hint: isolate the assumption
Keep the baseline input fixed, change one parameter or boundary, and compare both the numerical result and the claimed units. A passing example verifies this fixture only.
Minimal executable example
Save the example and linked modules in one folder. Use Node.js 22 or newer, then run node sampling-lab.mjs. npm install gpt-tokenizer@4.1.0
import {encode as rawEncode} from 'gpt-tokenizer/encoding/cl100k_base';
import {promptDiff,inspectDataset,chunkDocument,compareScores,estimateVRAM,samplingLab,speculativeLab} from './research-core.mjs';
const encode=s=>rawEncode(s,{disallowedSpecial:new Set()});
const result=samplingLab('token,logit\na,0\nb,0',1,0,1,10000,42);
console.log(result.entropy); // 1 bit; observed frequencies only approximate 0.5Test and transfer
Filtering order is temperature, top-k, then top-p and renormalization. Entropy is in bits. Temperature zero is explicitly greedy; stochastic draws depend on the seed.
Record input, code version, expected result and observed result in the notebook below. Then construct a counterexample that violates the assumption above and explain why the original conclusion no longer follows.
Download complete kit (.tar.gz) · Complete reference kit and regression checksPredict, test and keep evidence
Before revealing an answer, predict one result. After the exercise, record the discrepancy and a transfer test. Saved locally across language editions.
Open notebook / export / clear →The mathematical review scope is recorded per lesson. It does not establish empirical paper reproduction; report suspected errors in your notes. Review scope