Token LabSearch

← Physical AI
PHYSICAL AI / 2 OF 8

02 · Perception, state estimation and time

Understand why a sharp image can still produce a stale, uncertain control state.

Prerequisites: Frames, probability and covariance.

Understand the mechanism

The physical state is partially observed. Cameras, encoders and force sensors have different rates and delays. Keep acquisition time separate from arrival time, and estimate state at the command time. A Kalman-style correction balances predicted uncertainty against measurement uncertainty; correlated errors invalidate a naive independent fusion. Occlusion is missing information, not a zero measurement. Preserve uncertainty and a validity flag through the policy interface.

Text / 原始公式
K = P / (P + R)
x̂_new = x̂_prior + K(z − x̂_prior)
P_new = (1 − K)P
position lag ≈ velocity × sensor age
Study guide & lesson outlineLEARNING CONTRACT

Understand → implement → investigate

Goal: explain the mechanism in this lesson, reproduce one example and test a failure case. Reading alone does not mark mastery.

Prerequisite route →Related implementation practice →
  • Understand: state assumptions, shapes/units and a counterexample.
  • Implement: reproduce a numerical example and inspect boundaries.
  • Investigate: compare one alternative under a matched budget.

Mechanism map & worked derivation

Timestamped sensors → Predict state to present → Fuse valid measurements → State + uncertainty + age
Original schematic; read the steps from top to bottom. SVG ↓
Symbols and assumptions
SymbolMeaning
P, RScalar prior and measurement variances
z, KMeasurement and Kalman gain
  1. Prior position is 1.0 m with m²; sensor reports 1.2 m with m².
  2. The corrected position is 1.16 m and variance 0.008 m². This scalar example assumes unbiased independent Gaussian noise.
  3. At 0.5 m/s, an 80 ms old observation is displaced by approximately 4 cm under constant velocity. Better classification accuracy cannot remove this lag.
Experiment checkpoint

Create a synthetic constant-velocity trace with seeded noise, dropped frames and delay. Compare raw versus filtered RMSE and lag. Hold out the delay pattern, not random adjacent frames.

Original interview exercises & analysis

Answer aloud before revealing the analysis. Add assumptions, a counterexample and a test you would run.

Why can fusion be overconfident?

Two estimates may share the same camera or calibration error. Treating them as independent double-counts evidence; track correlation or conservatively fuse.

What should happen during occlusion?

Predict with growing uncertainty and apply an age/uncertainty gate. Define when to pause or request a new view; do not invent observations.

Read the primary sources

Underactuated Robotics — Russ Tedrake ↗

Original Token Lab explanations and synthetic calculations; not measured robot performance. Source links checked 2026-10-25. Unversioned arXiv links may change; record a version for reproduction.

MATHEMATICAL REVIEW · 2026-10-26

Assumptions, implementation & acceptance

The scalar Kalman example assumes direct measurement , independent zero-mean noises and positive innovation variance. Covariances use squared position units.

Formula conditions, units and worked arithmetic reviewed. The implementation isolates one numerical mechanism; it does not train a policy or control real hardware.

Before running: predict a boundary case

Posterior lies between prior and measurement; posterior variance is nonnegative and smaller.

Hint: isolate the assumption

Keep the baseline input fixed, change one parameter or boundary, and compare both the numerical result and the claimed units. A passing example verifies this fixture only.

Minimal executable example

Save the example and linked modules in one folder. Use Node.js 22 or newer, then run node perception-and-state.mjs. No third-party runtime dependencies.

const solve=()=>{
const prior=1, P=4, measurement=3, R=1;
const K=P/(P+R);return [K,prior+K*(measurement-prior),(1-K)*P];
};
const actual=solve();
const expected=[0.8,2.6,0.8];
if(JSON.stringify(actual)===JSON.stringify(expected)){console.log('PASS',actual)}else{
const a=[actual].flat(), b=[expected].flat();
if(a.length!==b.length||a.some((v,i)=>!Number.isFinite(v)||Math.abs(v-b[i])>1e-10))throw Error('Numerical mismatch');
console.log('PASS within 1e-10',actual);
}
Download this example
Reference output — reveal after predicting
[
  0.8,
  2.6,
  0.8
]

Test and transfer

Posterior lies between prior and measurement; posterior variance is nonnegative and smaller.

Record input, code version, expected result and observed result in the notebook below. Then construct a counterexample that violates the assumption above and explain why the original conclusion no longer follows.

Download complete kit (.tar.gz) · Complete reference kit and regression checks

Predict, test and keep evidence

Before revealing an answer, predict one result. After the exercise, record the discrepancy and a transfer test. Saved locally across language editions.

Open notebook / export / clear →

The mathematical review scope is recorded per lesson. It does not establish empirical paper reproduction; report suspected errors in your notes. Review scope