File Q&A: build an evidence-first document assistant

1 · Where you encounter this

ChatGPT publicly supports uploading files for analysis, summarization and extraction. Try a non-sensitive policy document and ask for the refund deadline with the supporting passage. This tutorial implements a small lexical retrieval component, not ChatGPT’s internal file-processing architecture.

Source checked 2026-10-25. Product behavior is publicly documented; the code below is an independent teaching implementation.

2 · The engineering decision

The practical problem is provenance: a fluent answer is not enough if the evidence cannot be traced. We split on blank lines, preserve paragraph IDs, score overlap with query terms and return exact quotes. No hit produces abstention. The score is lexical overlap, not confidence, truth or semantic relevance. This deliberately simple baseline makes failure visible before introducing embeddings or an LLM.

1. Inspect the query terms: refund and deadline.
2. P1 contains refund, so overlap is 1/2 = 0.5. P2 and P3 score zero.
3. Return P1 verbatim, including its ID. Do not fabricate a synthesized answer.
4. Change the query to reimbursement window: the baseline abstains despite related meaning. This motivates semantic retrieval.

4 · Complete source code

Requires a modern browser or Node.js 18+. Download the module, save this runner beside it and execute node demo.mjs. No npm dependencies.

// Local lexical retrieval baseline. No LLM, embeddings, PDF parser or network.
export const example = {
  query: 'refund deadline',
  document: 'Refund policy: request a refund within 30 days.\n\nShipping policy: delivery takes 5 days.\n\nSupport policy: contact support for damaged items.'
};
const terms = text => new Set(text.toLowerCase().match(/[\p{L}\p{N}]+/gu) || []);
export function run({query, document}) {
  if (typeof query !== 'string' || typeof document !== 'string') throw Error('query and document must be strings');
  if (document.length > 20000 || query.length > 500) throw Error('Input limit exceeded');
  const q = terms(query);
  const evidence = document.split(/\n\s*\n/).map((text, index) => {
    const tokens = terms(text);
    const matches = [...q].filter(term => tokens.has(term));
    return {id: `P${index + 1}`, text, matches, score: matches.length / Math.max(1, q.size)};
  }).filter(item => item.score > 0)
    .sort((a, b) => b.score - a.score || a.id.localeCompare(b.id)).slice(0, 2);
  return {
    status: evidence.length ? 'evidence_found' : 'abstain',
    evidence,
    // Evidence quotes, not a generated or semantically verified answer.
    extract: evidence.map(item => `[${item.id}] ${item.text}`).join('\n')
  };
}

import {run, example} from './document-evidence.mjs';
console.log(JSON.stringify(run(example), null, 2));

5 · Failure cases and production gaps

Keyword matches can be irrelevant or contradictory. Paragraph IDs are only stable within this input; production citations need document versions and offsets. Whitespace tokenization is weak for unsegmented Chinese. This demo accepts plain text, not PDFs or OCR. Document instructions are data and must not become system instructions in a later generation stage.