Token LabSearch

← Physical AI
PHYSICAL AI / 6 OF 8

06 · OpenVLA: vision, language and action interfaces

Trace a language-conditioned robot policy through action encoding, normalization and embodiment.

Prerequisites: Transformers, imitation learning and robot frames.

Understand the mechanism

OpenVLA is an open vision-language-action model trained from robot demonstrations. Its action-token interface makes the bridge between a language-model backbone and physical controls explicit. A language instruction alone does not define the actuator convention. Record whether outputs are absolute poses, deltas, velocities or joint targets, which frame they use, and how gripper values are interpreted. Transfer across robots requires matching these interfaces as well as learning useful visual features.

Text / 原始公式
L = −Σⱼ log pθ(bⱼ | image, instruction, b<ⱼ)
â = a_min + (b + 1/2)(a_max − a_min)/K
b ∈ {0,…,K−1}  [uniform-bin teaching example]
Study guide & lesson outlineLEARNING CONTRACT

Understand → implement → investigate

Goal: explain the mechanism in this lesson, reproduce one example and test a failure case. Reading alone does not mark mastery.

Prerequisite route →Related implementation practice →
  • Understand: state assumptions, shapes/units and a counterexample.
  • Implement: reproduce a numerical example and inspect boundaries.
  • Investigate: compare one alternative under a matched budget.

Mechanism map & worked derivation

Image + instruction + state contract → Vision-language representation → Action tokens → decoded values → Robot adapter + feedback
Original schematic; read the steps from top to bottom. SVG ↓
Symbols and assumptions
SymbolMeaning
b, KAction bin index and number of bins in the toy codec
a_min, a_maxDeclared action range; real preprocessing must be versioned
  1. For a teaching codec with range [−1,1], and , decoding the bin center gives 0.00390625.
  2. Its bin width is 0.0078125 and maximum within-bin error is 0.00390625, ignoring clipping. That number has the units of the encoded action.
  3. A 1 cm translation delta at 10 Hz differs from a 1 cm/s velocity command. Write an explicit adapter contract before comparing policies.
Experiment checkpoint

Before fine-tuning, build round-trip codec tests, inspect saturation and verify action units on recorded data. Evaluate held-out objects, instructions and scenes separately. A codec test is not a model evaluation.

Original interview exercises & analysis

Answer aloud before revealing the analysis. Add assumptions, a counterexample and a test you would run.

Does language pretraining imply physical grounding?

No. Grounding depends on demonstrations, observations, embodiment and evaluation. Use instruction counterfactuals and object swaps to test whether language is used.

What breaks during cross-robot transfer?

Camera geometry, action frames, normalization, gripper semantics and timing can all shift. Audit each before attributing failure solely to model size.

Read the primary sources

OpenVLA ↗ · arXiv PDF ↗

Original Token Lab explanations and synthetic calculations; not measured robot performance. Source links checked 2026-10-25. Unversioned arXiv links may change; record a version for reproduction.

MATHEMATICAL REVIEW · 2026-10-26

Assumptions, implementation & acceptance

Action bins quantize normalized coordinates. Half-bin error assumes nearest-bin rounding away from boundary/clipping effects. Token prediction alone does not guarantee calibration, control stability or safety.

Formula conditions, units and worked arithmetic reviewed. The implementation isolates one numerical mechanism; it does not train a policy or control real hardware.

Before running: predict a boundary case

For values within bounds nearest-bin absolute error is at most half a step.

Hint: isolate the assumption

Keep the baseline input fixed, change one parameter or boundary, and compare both the numerical result and the claimed units. A passing example verifies this fixture only.

Minimal executable example

Save the example and linked modules in one folder. Use Node.js 22 or newer, then run node vla-openvla.mjs. No third-party runtime dependencies.

const solve=()=>{
const lo=-1,hi=1,bins=5,value=.3;
const step=(hi-lo)/(bins-1), index=Math.round((value-lo)/step);
return [index,lo+index*step,step/2];
};
const actual=solve();
const expected=[3,0.5,0.25];
if(JSON.stringify(actual)===JSON.stringify(expected)){console.log('PASS',actual)}else{
const a=[actual].flat(), b=[expected].flat();
if(a.length!==b.length||a.some((v,i)=>!Number.isFinite(v)||Math.abs(v-b[i])>1e-10))throw Error('Numerical mismatch');
console.log('PASS within 1e-10',actual);
}
Download this example
Reference output — reveal after predicting
[
  3,
  0.5,
  0.25
]

Test and transfer

For values within bounds nearest-bin absolute error is at most half a step.

Record input, code version, expected result and observed result in the notebook below. Then construct a counterexample that violates the assumption above and explain why the original conclusion no longer follows.

Download complete kit (.tar.gz) · Complete reference kit and regression checks

Predict, test and keep evidence

Before revealing an answer, predict one result. After the exercise, record the discrepancy and a transfer test. Saved locally across language editions.

Open notebook / export / clear →

The mathematical review scope is recorded per lesson. It does not establish empirical paper reproduction; report suspected errors in your notes. Review scope