SimCLR: augmentation defines what similarity means
Follow two views through a shared encoder and projection head, then audit whether the invariance is valid.
A Simple Framework for Contrastive Learning of Visual Representations · Chen et al. · arXiv:2002.05709
Original editorial analysis by Token Lab · Source checked 2026-10-25 · Paper claims and synthetic examples are separated.
Detailed technical articles are available in English and Chinese. This locale shows the English article with localized navigation.
01 · What the paper establishes
SimCLR studies augmentation composition, a nonlinear projection head and contrastive training at scale. The paper reports representation quality through downstream evaluation, including linear evaluation, rather than treating training loss alone as success.
Primary source: paper abstract ↗Study guide & lesson outline
LEARNING CONTRACTUnderstand → implement → investigate
Goal: explain the mechanism in this lesson, reproduce one example and test a failure case. Reading alone does not mark mastery.
- Understand: state assumptions, shapes/units and a counterexample.
- Implement: reproduce a numerical example and inspect boundaries.
- Investigate: compare one alternative under a matched budget.
02 · The mechanism
Text / 原始公式
x → augmentation₁ → f → h₁ → g → z₁ x → augmentation₂ → f → h₂ → g → z₂ Lᵢⱼ = −log[exp(sim(zᵢ,zⱼ)/τ) / Σₖ≠ᵢ exp(sim(zᵢ,zₖ)/τ)] N original examples → 2N views → 2N−2 negatives/anchor
The positive-pair rule tells the representation which changes to ignore. Two crops may preserve object identity, but in another domain a crop can remove the only relevant signal. The projection head separates the contrastive objective’s representation from the encoder features used downstream. That separation should be tested, not assumed to help every dataset.
Notation and intuition are teaching explanations. Consult the original method for complete assumptions and implementation details.
Mechanism map & worked derivation
| Symbol | Meaning |
|---|---|
| f, g | Shared encoder and projection head |
| h, z | Encoder representation and projected representation |
| 2N−2 | Negatives per anchor under the standard paired batch |
- Generate two independently augmented views for each source example. Construct positive indices from source identity, not from incidental batch adjacency.
- For
there are six views. Each anchor excludes itself, has one positive and four negatives. Verify that every anchor has exactly one intended positive. - Evaluate encoder features h separately from projection z. A contrastive objective can remove nuisance information at z that is still useful in h for another downstream task.
Write a domain-specific augmentation validity table. Include at least one plausible-looking transformation that changes the label.
Change one variable. Explain what moves.
Original teaching calculation, not a reproduction of the paper’s benchmark. Bars are relative within this panel; numeric values and units are authoritative.
04 · Work through an example
Original counting example:
05 · Design your experiment
Design an augmentation validity table for your domain before running experiments. Compare no projection head versus a nonlinear head under a matched compute budget. Evaluate h and z separately with separate trained linear classifiers on frozen features, with matched splits and tuning budgets. Include a label-preserving and a deliberately destructive augmentation, recording retrieval failures and class-specific accuracy.
State one hypothesis; save the configuration and raw results; identify a competing explanation; report the narrowest supported conclusion. Use the linked lab as a local starting point, not a full paper reproduction.
Open related lesson / lab →06 · Limits & reviewer questions
Editorial caution: batch size changes negative count, gradient noise and often optimizer behavior simultaneously. An ablation that changes all three does not isolate a single cause. Duplicate or semantically related examples may become false negatives even when they have different file names.
Should a medical-image augmentation automatically copy the natural-image recipe?
No. First identify which transformations preserve the downstream label and clinically relevant signal. A transformation can be visually plausible while erasing the evidence needed for the task.
07 · Read the original
Framework and augmentation/projection-head ablations, then the linear-evaluation protocol.
Year is the first arXiv submission year. Unversioned links follow revisions; record the exact version in your reproduction report. DeepSeek-R1 here is explicitly pinned to v1.
Continue this topic
CPC: learn by identifying the right future
Treat contrastive prediction as a classification problem and inspect what negative sampling assumes.
van den Oord, Li & Vinyals · arXiv:1807.03748