Information Retrieval

Represent queries and documents as vectors and identify their dimensions.

Compute dot-product similarity between two representation vectors.

Compute cosine similarity and explain the effect of vector normalization.

Interpret a probability distribution over candidate documents.

Compute softmax probabilities from a small set of retrieval scores.

Calculate cross-entropy loss for a single positive document.

Explain how gradient descent updates a model parameter from a loss gradient.

Express an information need as a query, document corpus, and relevance relation.

Explain how an inverted index supports efficient term-based retrieval.

Calculate TF-IDF weights for terms in a small document collection.

Compute a BM25 score for a query-document pair.

Explain how BM25 parameters control term saturation and document-length normalization.

Distinguish lexical mismatch from semantic mismatch in retrieval examples.

Identify which classical retrieval assumptions motivate neural retrieval methods.

Calculate precision and recall at a specified rank cutoff.

Calculate reciprocal rank for a ranked result list.

Calculate average precision for individual queries and mean average precision across a small query set.

Calculate discounted cumulative gain and normalized DCG at a specified cutoff.

Choose an evaluation metric that matches a stated search objective.

Explain how pooling and incomplete relevance judgments can bias offline evaluation.

Interpret whether a reported metric difference is practically and statistically meaningful using a paired significance test.

Distinguish sparse lexical representations from dense semantic representations.

Apply subword tokenization to a simple text example.

Explain the purpose of token, position, and special-token representations in a transformer encoder.

Calculate scaled dot-product attention for a small numerical example.

Explain how self-attention produces context-sensitive token representations.

Compare pooled sequence representations with token-level representations for retrieval.

Trace how a transformer encoder converts text into a retrieval representation.

Diagram the query and document encoding paths of a bi-encoder retriever and identify where similarity is computed.

Explain how contrastive learning separates relevant and non-relevant documents.

Calculate an in-batch contrastive loss for a small retrieval example.

Distinguish random, in-batch, and hard negatives by their effects on retriever training.

Explain why exact nearest-neighbor search becomes costly and how approximate search introduces a recall-latency trade-off.

Compare graph-based and partition-based approximate-nearest-neighbor indexes, including the effect of vector quantization, across recall, latency, memory, and update cost.

Diagnose common dense-retrieval failures from example result lists.

Explain how learned sparse retrieval expands and reweights lexical features.

Interpret a learned sparse representation for a query or document.

Explain how sparsity regularization affects effectiveness and index cost.

Compute a late-interaction MaxSim score for a small query-document example.

Explain how ColBERT-style retrieval preserves token-level matching.

Compare learned sparse, dense bi-encoder, and late-interaction models by representation and scoring.

Select an architecture for a stated effectiveness, latency, and storage constraint.

Trace how a cross-encoder jointly scores a query-document pair.

Compare bi-encoder and cross-encoder scoring in retrieval quality and computational cost.

Distinguish pointwise, pairwise, and listwise objectives used to train ranking models.

Construct a two-stage retrieval pipeline from candidate-generator and reranker components.

Calculate the latency cost of reranking a specified number of candidates.

Explain how knowledge distillation transfers ranking behavior to a smaller model.

Diagnose whether poor results originate in candidate recall or reranker ordering.

Identify complementary lexical and semantic evidence in search examples.

Normalize lexical and dense scores using a specified normalization method.

Combine ranked lists using reciprocal rank fusion.

Combine normalized retrieval scores using weighted linear fusion.

Tune a fusion weight from a small validation result table.

Explain how query characteristics can guide adaptive retrieval or fusion.

Design an ablation that isolates the contribution of each retrieval component.

Distinguish explicit judgments, click logs, and synthetic queries as supervision sources.

Identify position bias and selection bias in implicit-feedback data.

Construct positive and negative training examples from relevance judgments or behavioral data while avoiding obvious false negatives.

Compare full fine-tuning, parameter-efficient tuning, and zero-shot transfer for retrieval.

Explain how synthetic query generation can support domain adaptation.

Recognize data leakage between retrieval training and evaluation sets.

Extract the research question, baseline, intervention, and supporting evidence from an IR paper.

Trace information flow through ingestion, retrieval, context construction, and generation in a RAG pipeline.

Compare fixed-size, structure-aware, and semantic chunking strategies.

Predict how chunk size and overlap affect retrieval and generation behavior.

Design metadata fields and filters for a stated RAG corpus.

Explain how query rewriting and multi-query retrieval can improve candidate coverage.

Construct a context-selection policy under a fixed token budget.

Distinguish retrieval failure, context-use failure, and unsupported generation.

Define separate evaluation questions for retrieval quality, answer quality, and evidence faithfulness.

Choose metrics for evaluating retrieval and answer quality within a RAG system.

Construct a small test set containing answerable, unanswerable, and adversarial queries.

Assess whether an answer is supported by its retrieved evidence.

Design an ablation for chunking, retrieval, reranking, or context selection.

Identify prompt-injection risks in retrieved documents.

Select layered mitigations for untrusted or restricted retrieved content, including provenance checks and document-level access controls.

Translate a product search requirement into measurable relevance, latency, cost, and freshness targets.

Estimate the storage and latency implications of sparse, dense, and late-interaction indexes.

Select a retrieval architecture for a specified corpus and query workload.

Design a staged evaluation plan combining offline relevance tests, an online experiment, and operational guardrails.

Assess whether reported effectiveness-efficiency evidence supports an IR paper's stated conclusion.

Apply a structured framework to compare the architecture, supervision, benchmarks, and computational efficiency of modern IR papers.

Produce a concise, evidence-based design proposal for an end-to-end hybrid retrieval and RAG system.