06 · OpenVLA: vision, language and action interfaces
Trace a language-conditioned robot policy through action encoding, normalization and embodiment.
Prerequisites: Transformers, imitation learning and robot frames.
Technical posts currently have English and Chinese editions. This locale displays the English article with localized navigation.
Understand the mechanism
OpenVLA is an open vision-language-action model trained from robot demonstrations. Its action-token interface makes the bridge between a language-model backbone and physical controls explicit. A language instruction alone does not define the actuator convention. Record whether outputs are absolute poses, deltas, velocities or joint targets, which frame they use, and how gripper values are interpreted. Transfer across robots requires matching these interfaces as well as learning useful visual features.
Text / 原始公式
L = −Σⱼ log pθ(bⱼ | image, instruction, b<ⱼ)
â = a_min + (b + 1/2)(a_max − a_min)/K
b ∈ {0,…,K−1} [uniform-bin teaching example]Study guide & lesson outline
LEARNING CONTRACTUnderstand → implement → investigate
Goal: explain the mechanism in this lesson, reproduce one example and test a failure case. Reading alone does not mark mastery.
- Understand: state assumptions, shapes/units and a counterexample.
- Implement: reproduce a numerical example and inspect boundaries.
- Investigate: compare one alternative under a matched budget.
Mechanism map & worked derivation
| Symbol | Meaning |
|---|---|
| b, K | Action bin index and number of bins in the toy codec |
| a_min, a_max | Declared action range; real preprocessing must be versioned |
- For a teaching codec with range [−1,1],
and , decoding the bin center gives 0.00390625. - Its bin width is 0.0078125 and maximum within-bin error is 0.00390625, ignoring clipping. That number has the units of the encoded action.
- A 1 cm translation delta at 10 Hz differs from a 1 cm/s velocity command. Write an explicit adapter contract before comparing policies.
Before fine-tuning, build round-trip codec tests, inspect saturation and verify action units on recorded data. Evaluate held-out objects, instructions and scenes separately. A codec test is not a model evaluation.
Original interview exercises & analysis
Answer aloud before revealing the analysis. Add assumptions, a counterexample and a test you would run.
Does language pretraining imply physical grounding?
No. Grounding depends on demonstrations, observations, embodiment and evaluation. Use instruction counterfactuals and object swaps to test whether language is used.
What breaks during cross-robot transfer?
Camera geometry, action frames, normalization, gripper semantics and timing can all shift. Audit each before attributing failure solely to model size.
Read the primary sources
Original Token Lab explanations and synthetic calculations; not measured robot performance. Source links checked 2026-10-25. Unversioned arXiv links may change; record a version for reproduction.