Token Lab검색

← 모든 실험
하이브리드 검색과 재정렬 / 3 OF 5

벡터 검색만

Only vector direction determines the ranking.

로컬 계산 / 시뮬레이션

외부 API 없는 로컬 교육용 실험입니다. 합성 데이터는 운영 환경 측정이 아닙니다.

Study guide & lesson outlineLEARNING CONTRACT

Understand → implement → investigate

Goal: explain the mechanism in this lesson, reproduce one example and test a failure case. Reading alone does not mark mastery.

Prerequisite route →Related implementation practice →

02 · 예측 및 설정

기술 설명, 매개변수 이름과 참고 답변은 현재 영어입니다.

03 · 근거 확인

실행하면 결과가 표시됩니다.

방법과 참고 문헌

Lexical score counts matching query terms (not BM25). Dense score is cosine on supplied 2D vectors, not a live embedding model. RRF combines ranks. Optional reranking uses known relevance labels as an oracle upper-bound, not a trained reranker. Empty query ties preserve document order.

Text / 原始公式
score=w/(k+lexicalRank)+(1−w)/(k+denseRank); DCG=Σ(2^rel−1)/log₂(rank+1)

An oracle reranker leaks labels intentionally; never report it as learned model performance.

Reciprocal Rank Fusion · SIGIR 2009 ↗

연습 문제

retrieval-3 / 01What does “Dense only” test?

Only vector direction determines the ranking.

Original teaching answer / local derivation. Method reference: Reciprocal Rank Fusion · SIGIR 2009 ↗

실험 열기 →
retrieval-3 / 02Which computation produces the result in “Dense only”?

;

Original teaching answer / local derivation. Method reference: Reciprocal Rank Fusion · SIGIR 2009 ↗

실험 열기 →
retrieval-3 / 03What is the baseline configuration for “Dense only”?

{"query":"KV cache memory","documents":"[\n {\n \"id\": \"D1\",\n \"text\": \"KV cache stores keys and values for generation\",\n \"vector\": [\n 1,\n 0.1\n ],\n \"relevance\": 3\n },\n {\n \"id\": \"D2\",\n \"text\": \"Cache restaurant menu and opening hours\",\n \"vector\": [\n 0.1,\n 1\n ],\n \"relevance\": 0\n },\n {\n \"id\": \"D3\",\n \"text\": \"Grouped query attention reduces memory\",\n \"vector\": [\n 0.95,\n 0.2\n ],\n \"relevance\": 2\n },\n {\n \"id\": \"D4\",\n \"text\": \"KV cache memory footprint depends on sequence length\",\n \"vector\": [\n 0.8,\n 0.3\n ],\n \"relevance\": 3\n },\n {\n \"id\": \"D5\",\n \"text\": \"Tokenization splits input text\",\n \"vector\": [\n 0.4,\n 0.8\n ],\n \"relevance\": 0\n }\n]","x":1,"y":0.1,"weight":0,"k":60,"retrieve":5,"top":3,"rerank":0}

Original teaching answer / local derivation. Method reference: Reciprocal Rank Fusion · SIGIR 2009 ↗

실험 열기 →
retrieval-3 / 04What numerical result should you check for “Dense only”?

{"Recall@k":1,"nDCG@k":0.9594535145926796}

Original teaching answer / local derivation. Method reference: Reciprocal Rank Fusion · SIGIR 2009 ↗

실험 열기 →
retrieval-3 / 05How would you run a controlled comparison for “Dense only”?

Keep the supplied fixture and other settings fixed; change weight. Compare Recall@k and nDCG@k while varying candidate pool and fusion weight.

Original teaching answer / local derivation. Method reference: Reciprocal Rank Fusion · SIGIR 2009 ↗

실험 열기 →
retrieval-3 / 06What failure could invalidate a conclusion from “Dense only”?

An oracle reranker leaks labels intentionally; never report it as learned model performance.

Original teaching answer / local derivation. Method reference: Reciprocal Rank Fusion · SIGIR 2009 ↗

실험 열기 →
retrieval-3 / 07Is “Dense only” a benchmark of a deployed language model?

No. Lexical score counts matching query terms (not BM25). Dense score is cosine on supplied 2D vectors, not a live embedding model. RRF combines ranks. Optional reranking uses known relevance labels as an oracle upper-bound, not a trained reranker. Empty query ties preserve document order.

Original teaching answer / local derivation. Method reference: Reciprocal Rank Fusion · SIGIR 2009 ↗

실험 열기 →
retrieval-3 / 08Which evidence should you report for “Dense only”?

Compare Recall@k and nDCG@k while varying candidate pool and fusion weight. Export inputs, engine, result and method; distinguish synthetic fixtures from observed production data.

Original teaching answer / local derivation. Method reference: Reciprocal Rank Fusion · SIGIR 2009 ↗

실험 열기 →
retrieval-3 / 09How can another researcher reproduce “Dense only”?

Open retrieval-3, reset the preset, retain the exact inputs and run. The calculation is deterministic for identical inputs.

Original teaching answer / local derivation. Method reference: Reciprocal Rank Fusion · SIGIR 2009 ↗

실험 열기 →
retrieval-3 / 10What is the next validation step after “Dense only”?

An oracle reranker leaks labels intentionally; never report it as learned model performance. Test the hypothesis on a representative held-out workload and document a baseline, success metric and stopping rule before claiming deployment readiness.

Original teaching answer / local derivation. Method reference: Reciprocal Rank Fusion · SIGIR 2009 ↗

실험 열기 →
MATHEMATICAL REVIEW · 2026-10-26

Assumptions, implementation & acceptance

Weighted reciprocal-rank fusion uses ranks starting at 1. Supplied relevance is an oracle reranker, not a trained model. Zero-vector cosine uses a software sentinel; labels must not enter a real retrieval index.

Engine formulas, scenario presets and numerical safeguards reviewed. Seeded synthetic inputs are not a production benchmark.

Query → lexical and vector ranks → Weighted reciprocal-rank fusion → Optional relevance oracle rerank → Top-k → recall and nDCG
Mechanism map: follow the transformations from top to bottom.

Before running: predict a boundary case

Only vector direction determines the ranking.

Hint: isolate the assumption

Keep the baseline input fixed, change one parameter or boundary, and compare both the numerical result and the claimed units. A passing example verifies this fixture only.

Minimal executable example

Save the example and linked modules in one folder. Use Node.js 22 or newer, then run node retrieval-3.mjs. No third-party runtime dependencies.

import {calculate,network} from './academy-core.mjs';
const params={
  "query": "KV cache memory",
  "documents": "[\n  {\n    \"id\": \"D1\",\n    \"text\": \"KV cache stores keys and values for generation\",\n    \"vector\": [\n      1,\n      0.1\n    ],\n    \"relevance\": 3\n  },\n  {\n    \"id\": \"D2\",\n    \"text\": \"Cache restaurant menu and opening hours\",\n    \"vector\": [\n      0.1,\n      1\n    ],\n    \"relevance\": 0\n  },\n  {\n    \"id\": \"D3\",\n    \"text\": \"Grouped query attention reduces memory\",\n    \"vector\": [\n      0.95,\n      0.2\n    ],\n    \"relevance\": 2\n  },\n  {\n    \"id\": \"D4\",\n    \"text\": \"KV cache memory footprint depends on sequence length\",\n    \"vector\": [\n      0.8,\n      0.3\n    ],\n    \"relevance\": 3\n  },\n  {\n    \"id\": \"D5\",\n    \"text\": \"Tokenization splits input text\",\n    \"vector\": [\n      0.4,\n      0.8\n    ],\n    \"relevance\": 0\n  }\n]",
  "x": 1,
  "y": 0.1,
  "weight": 0,
  "k": 60,
  "retrieve": 5,
  "top": 3,
  "rerank": 0
};
const result=calculate('retrieval',params);
console.log(result.metrics);
Download this exampleacademy-core.mjs

Test and transfer

Only vector direction determines the ranking.

Record input, code version, expected result and observed result in the notebook below. Then construct a counterexample that violates the assumption above and explain why the original conclusion no longer follows.

Download complete kit (.tar.gz) · Complete reference kit and regression checks

Predict, test and keep evidence

Before revealing an answer, predict one result. After the exercise, record the discrepancy and a transfer test. Saved locally across language editions.

Open notebook / export / clear →

The mathematical review scope is recorded per lesson. It does not establish empirical paper reproduction; report suspected errors in your notes. Review scope