Document extraction: JSON shape is not factual correctness

1 · Where you encounter this

A common document-assistant task is extracting fields into a table. OpenAI’s API separately offers Structured Outputs for schema-constrained responses. That API capability is not a claim that every ChatGPT table uses the same implementation. Here you supply a candidate JSON object; the local code validates it without calling a model.

Source checked 2026-10-25. Product behavior is publicly documented; the code below is an independent teaching implementation.

2 · The engineering decision

Use two independent gates. Shape validation rejects missing, unexpected or incorrectly typed fields. Evidence validation checks whether quoted text appears in the source and supports the declared amount. Neither gate alone solves semantic ambiguity: a document can contain subtotal, tax and total, all valid numbers. Keep rejection visible instead of silently coercing a string to a number or substituting zero.

1. The supplied candidate has the four allowed keys and a finite nonnegative amount.
2. The invoice ID occurs in the source; the quote USD 42.50 also occurs.
3. Formatting 42.5 to two decimals gives 42.50, which matches the evidence.
4. Change amount to 425: shapeValid stays true but accepted becomes false. This separates formatting from grounding.

4 · Complete source code

Requires a modern browser or Node.js 18+. Download the module, save this runner beside it and execute node demo.mjs. No npm dependencies.

// Validate a candidate output supplied by a person or model. No model call.
export const example = {
  source: 'Invoice A-17. Total: USD 42.50.',
  candidate: {invoice_id: 'A-17', currency: 'USD', amount: 42.5, evidence: 'USD 42.50'}
};
export function run({source, candidate}) {
  const errors = [];
  if (typeof source !== 'string' || source.length > 20000) throw Error('Invalid source');
  if (!candidate || typeof candidate !== 'object' || Array.isArray(candidate)) throw Error('candidate must be an object');
  const keys = ['invoice_id', 'currency', 'amount', 'evidence'];
  if (Object.keys(candidate).some(key => !keys.includes(key))) errors.push('Unexpected field');
  for (const key of ['invoice_id', 'currency', 'evidence']) {
    if (typeof candidate[key] !== 'string' || !candidate[key].trim()) errors.push(`Invalid ${key}`);
  }
  if (!Number.isFinite(candidate.amount) || candidate.amount < 0) errors.push('Invalid amount');
  if (!['USD', 'EUR', 'CNY'].includes(candidate.currency)) errors.push('Unsupported currency');
  const shapeValid = errors.length === 0;
  if (shapeValid) {
    if (!source.includes(candidate.invoice_id)) errors.push('Invoice ID absent from source');
    if (!source.includes(candidate.evidence)) errors.push('Evidence quote absent from source');
    const expected = `${candidate.currency} ${candidate.amount.toFixed(2)}`;
    if (!candidate.evidence.includes(expected)) errors.push('Amount/currency not supported by quote');
  }
  return {shapeValid, accepted: errors.length === 0, errors,
    limitation: 'Exact-text checks only; conflicting totals and invoice context need further validation.'};
}

import {run, example} from './validated-extraction.mjs';
console.log(JSON.stringify(run(example), null, 2));

5 · Failure cases and production gaps

Exact substring checks are teaching constraints, not invoice understanding. Multiple invoices, contradictory amounts, localization and decimal precision require stronger handling. Financial production systems usually represent amounts as decimal or integer minor units. Handle refusal, incomplete output and parse failure separately when integrating a real model.