六つの深い実験で、目的関数の導出、実装検証、失敗の分析、研究の説明を学びます。
Check gradients numerically, inject a reduction bug, and distinguish instability from a wrong derivative.
Understand entropy, cross-entropy, both KL directions and support mismatch.
Train low-rank factors, compare an analytical capacity bound, and inspect initialization failure.
Analyze paired run means, uncertainty and a predeclared practical improvement threshold.
Inspect InfoNCE, temperature and the exact gradient applied to a possible false negative.
Record a falsifiable hypothesis, inspect controls and export a reviewable experiment report.
Read for a research question, not just a collection of links. These are official abstract and PDF links; reading guidance is our editorial learning path.
PDF links checked 2026-10-25. Unversioned arXiv URLs resolve to the current revision; record an explicit version for a reproducible study.
Browse papers by topic →arXiv:1412.6980 · 2014
Kingma & Ba · Year denotes first arXiv submission
Read: Algorithm 1; moment estimates and bias correction.
Reading checkpoint: Explain why zero-initialized moments need correction. The gradient debugger uses SGD, not Adam.
@misc{arxiv14126980,
title = {Adam: A Method for Stochastic Optimization},
year = {2014},
eprint = {1412.6980},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/1412.6980}
}arXiv:1711.05101 · 2017
Loshchilov & Hutter · Year denotes first arXiv submission
Read: Comparison of L2 regularization and decoupled decay under adaptive updates.
Reading checkpoint: Write the two updates side by side; identify where the preconditioner acts. This is a reading extension, not an implemented AdamW benchmark.
@misc{arxiv171105101,
title = {Decoupled Weight Decay Regularization},
year = {2017},
eprint = {1711.05101},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/1711.05101}
}arXiv:1706.04599 · 2017
Guo et al. · Year denotes first arXiv submission
Read: Reliability diagrams, calibration metrics, temperature scaling.
Reading checkpoint: Distinguish class ranking from confidence quality; fit temperature on validation data, never the final test set.
@misc{arxiv170604599,
title = {On Calibration of Modern Neural Networks},
year = {2017},
eprint = {1706.04599},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/1706.04599}
}arXiv:2106.09685 · 2021
Hu et al. · Year denotes first arXiv submission
Read: Low-rank parameterization, initialization and rank investigations.
Reading checkpoint: Derive parameter count and the first update when B=0; distinguish the toy matrix experiment from language-model fine-tuning.
@misc{arxiv210609685,
title = {LoRA: Low-Rank Adaptation of Large Language Models},
year = {2021},
eprint = {2106.09685},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2106.09685}
}arXiv:2305.14314 · 2023
Dettmers et al. · Year denotes first arXiv submission
Read: Frozen quantized backbone, NF4, double quantization and paged optimizers.
Reading checkpoint: Make separate memory accounts for weights, adapters, optimizer states and activations. The site does not execute QLoRA.
@misc{arxiv230514314,
title = {QLoRA: Efficient Finetuning of Quantized LLMs},
year = {2023},
eprint = {2305.14314},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2305.14314}
}arXiv:1807.03748 · 2018
van den Oord, Li & Vinyals · Year denotes first arXiv submission
Read: Section 2.3: InfoNCE; Appendix A.1: mutual-information argument.
Reading checkpoint: Track the negative-sampling assumptions before interpreting a mutual-information bound.
@misc{arxiv180703748,
title = {Representation Learning with Contrastive Predictive Coding},
year = {2018},
eprint = {1807.03748},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/1807.03748}
}arXiv:2002.05709 · 2020
Chen et al. · Year denotes first arXiv submission
Read: Framework, augmentation ablations, projection head and contrastive objective.
Reading checkpoint: List which augmentation invariances are useful or harmful for your domain. A lower training loss is not the only evaluation.
@misc{arxiv200205709,
title = {A Simple Framework for Contrastive Learning of Visual Representations},
year = {2020},
eprint = {2002.05709},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2002.05709}
}arXiv:1709.06560 · 2017
Henderson et al. · Year denotes first arXiv submission
Read: Sources of experimental variability and reporting recommendations.
Reading checkpoint: Separate randomness from implementation and tuning differences. RL findings motivate controls; do not blindly transfer its numerical results to LLMs.
@misc{arxiv170906560,
title = {Deep Reinforcement Learning that Matters},
year = {2017},
eprint = {1709.06560},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/1709.06560}
}arXiv:1909.03004 · 2019
Dodge et al. · Year denotes first arXiv submission
Read: Expected validation performance under a computational budget; model selection.
Reading checkpoint: Report search space, trial count and selection rule alongside the final score.
@misc{arxiv190903004,
title = {Show Your Work: Improved Reporting of Experimental Results},
year = {2019},
eprint = {1909.03004},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/1909.03004}
}arXiv:2305.18290 · 2023
Rafailov et al. · Year denotes first arXiv submission
Read: KL-constrained reward objective, policy reparameterization and preference loss.
Reading checkpoint: Identify the reference-policy log ratio and β. The current information lab teaches KL, not a DPO trainer.
@misc{arxiv230518290,
title = {Direct Preference Optimization: Your Language Model is Secretly a Reward Model},
year = {2023},
eprint = {2305.18290},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2305.18290}
}arXiv:1706.03762 · 2017
Vaswani et al. · Year denotes first arXiv submission
Read: Scaled dot-product attention, masking, multi-head attention and positional encodings.
Reading checkpoint: Derive dimensions and masking before interpreting the attention heatmap.
@misc{arxiv170603762,
title = {Attention Is All You Need},
year = {2017},
eprint = {1706.03762},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/1706.03762}
}arXiv:2205.14135 · 2022
Dao et al. · Year denotes first arXiv submission
Read: Tiling, online softmax, recomputation and IO complexity.
Reading checkpoint: Explain why reducing HBM traffic can accelerate exact attention; the browser does not benchmark GPU kernels.
@misc{arxiv220514135,
title = {FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness},
year = {2022},
eprint = {2205.14135},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2205.14135}
}arXiv:2210.03629 · 2022
Yao et al. · Year denotes first arXiv submission
Read: Interleaved reasoning/action trajectories and comparisons with alternative prompting.
Reading checkpoint: Compare agent success under matched tool access and action budgets. Mock tool traces are not evidence about an LLM policy.
@misc{arxiv221003629,
title = {ReAct: Synergizing Reasoning and Acting in Language Models},
year = {2022},
eprint = {2210.03629},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2210.03629}
}arXiv:2306.05685 · 2023
Zheng et al. · Year denotes first arXiv submission
Read: Judge biases, agreement with humans and benchmark construction.
Reading checkpoint: Use order-swapped comparisons and label audits; consistency alone does not establish correctness.
@misc{arxiv230605685,
title = {Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena},
year = {2023},
eprint = {2306.05685},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2306.05685}
}arXiv:1707.06347 · 2017
Schulman et al. · Year denotes first arXiv submission
Read: Read the method, experimental protocol and limitations together.
Reading checkpoint: Which controlled experiment could falsify the capability claim?
@misc{arxiv170706347,
title = {Proximal Policy Optimization Algorithms},
year = {2017},
eprint = {1707.06347},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/1707.06347}
}arXiv:2203.02155 · 2022
Ouyang et al. · Year denotes first arXiv submission
Read: Read the method, experimental protocol and limitations together.
Reading checkpoint: Which controlled experiment could falsify the capability claim?
@misc{arxiv220302155,
title = {Training language models to follow instructions with human feedback},
year = {2022},
eprint = {2203.02155},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2203.02155}
}arXiv:2501.12948 · 2025
DeepSeek-AI · Year denotes first arXiv submission
Read: Read the method, experimental protocol and limitations together.
Reading checkpoint: Which controlled experiment could falsify the capability claim?
@misc{arxiv250112948,
title = {DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning},
year = {2025},
eprint = {2501.12948},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2501.12948v1}
}ローカル計算と合成データです。有料 API や実モデルのベンチマークは使用しません。
技術本文は英語と中国語に対応しています。この言語では英語の技術本文を表示します。