Does RL actually improve capability?

Assumptions, implementation & acceptance

s/√n assumes IID independent experimental units with finite variance. A normal 1.96 interval is approximate and fragile at small n; compare a Student-t interval. Episodes from one training seed are not independent training runs.

Formula conditions, units and worked arithmetic reviewed. The implementation isolates one numerical mechanism; it does not train a policy or control real hardware.

Before running: predict a boundary case

Sample variance is 2.5 and SE is sqrt(0.5); episodes cannot replace independent seeds.

Minimal executable example

Save the example and linked modules in one folder. Use Node.js 22 or newer, then run node rl-evaluation.mjs. No third-party runtime dependencies.

const solve=()=>{
const differences=[1,2,3,4,5], n=differences.length;
const mean=differences.reduce((a,b)=>a+b)/n;
const variance=differences.reduce((s,x)=>s+(x-mean)**2,0)/(n-1);
return [mean,Math.sqrt(variance/n)];
};
const actual=solve();
const expected=[3,0.7071067811865476];
if(JSON.stringify(actual)===JSON.stringify(expected)){console.log('PASS',actual)}else{
const a=[actual].flat(), b=[expected].flat();
if(a.length!==b.length||a.some((v,i)=>!Number.isFinite(v)||Math.abs(v-b[i])>1e-10))throw Error('Numerical mismatch');
console.log('PASS within 1e-10',actual);
}

[
  3,
  0.7071067811865476
]

Test and transfer

Sample variance is 2.5 and SE is sqrt(0.5); episodes cannot replace independent seeds.

Record input, code version, expected result and observed result in the notebook below. Then construct a counterexample that violates the assumption above and explain why the original conclusion no longer follows.