AI Engineer interview questions

A RAG system indexes source content, retrieves relevant evidence for a query, and gives that evidence to a generator.

• The ingestion path parses documents, splits them into chunks, creates embeddings, and stores vectors with text and metadata.
• At query time, the system transforms the user query into a search representation and retrieves candidate chunks from the index.
• An optional reranker orders the candidates more precisely, after which context assembly selects and formats the best evidence.
• The language model receives the query and assembled context, generates an answer, and can attach citations back to the stored sources.

Chunk size and overlap balance semantic completeness, retrieval precision, and context cost.

• Smaller chunks usually match focused questions more precisely, but they can split an explanation and remove context needed to interpret a fact.
• Larger chunks preserve more local context, but their embeddings may represent several topics and they consume more of the model's context window.
• Overlap protects information near chunk boundaries, but too much overlap creates duplicate results, a larger index, and repeated prompt content.
• Choose token-aware values by document type, then compare recall and answer quality on a representative evaluation set.

Use structure-aware chunking when document boundaries carry meaning, and semantic chunking when topic transitions are more informative than formatting.

• Structure-aware chunking follows headings, paragraphs, lists, tables, or code blocks, preserving hierarchy and source traceability.
• Semantic chunking groups nearby sentences by meaning and splits around topic shifts, which suits transcripts or loosely formatted prose.
• Structural units can be uneven, so large sections may need token-based subdivision and tiny sections may need merging.
• Semantic chunking costs more, is less deterministic, and can separate content that belongs to one structural unit, so a hybrid of both is often practical.

Parent-child retrieval searches precise small chunks but supplies their broader parent sections to the model.

• Each child chunk is embedded with a parent identifier, while the larger parent text is stored separately.
• Search runs over children because their focused embeddings usually match specific queries better than large sections do.
• The system deduplicates matched parent identifiers and fetches the corresponding parents, or selected neighboring children, for context assembly.
• This combines retrieval precision with readable context, but oversized parents can add unrelated text and consume too many prompt tokens.

Dense retrieval matches meaning, sparse retrieval matches terms, and hybrid retrieval combines both signals.

• Dense retrieval compares learned embeddings, so it handles paraphrases and conceptual similarity but may miss rare names or exact identifiers.
• Sparse retrieval, such as BM25, rewards shared tokens and term rarity, making it strong for product codes, names, and exact technical phrases.
• Hybrid retrieval runs both methods and combines their rankings with a technique such as reciprocal rank fusion or calibrated score weighting.
• Select the approach on labeled queries, with hybrid retrieval often fitting corpora that mix natural-language concepts with exact terminology.

They improve retrieval in different ways by clarifying one query, broadening its vocabulary, or splitting it into simpler searches.

• Rewriting produces a clearer standalone query, for example by resolving pronouns or conversational context while preserving the original intent.
• Expansion creates related formulations, synonyms, or likely terms so relevant documents using different wording can be found.
• Decomposition turns a compound or multi-hop question into subquestions whose evidence can be retrieved and combined.
• Each method can introduce intent drift and extra cost, so retain the original query, preserve its constraints, and evaluate the merged results.

Pre-filtering restricts the searchable candidates first, while post-filtering removes results after similarity search.

• Pre-filtering enforces hard constraints such as tenant, language, document type, or date before irrelevant vectors can enter the result set.
• Very selective pre-filters can reduce ANN recall or efficiency if the vector index does not handle filtered search well.
• Post-filtering is simpler for flexible criteria, but it can return fewer than top-k valid results and waste retrieval capacity on discarded candidates.
• Use supported pre-filtering for authorization and other mandatory constraints, while post-filtering or an expanded candidate set can serve soft preferences.

Choose the smallest top-k that meets the required recall while staying within the latency and downstream processing budget.

• A small top-k is faster and cheaper, but a relevant passage just below the cutoff cannot be recovered by later stages.
• A larger top-k improves the chance of finding relevant evidence, but increases result transfer, reranking work, and context selection cost.
• Top-k does not replace ANN search-breadth settings such as efSearch or nprobe, which also control how thoroughly the index explores candidates.
• Measure recall@k and latency on labeled queries, and keep retrieval top-k larger than the final number of chunks passed to the model.

A bi-encoder retrieves candidates efficiently, while a cross-encoder reranks a smaller set with richer query-document interactions.

• A bi-encoder encodes queries and documents independently, allowing document vectors to be precomputed and searched with an ANN index.
• A cross-encoder processes each query-document pair jointly, capturing token-level relationships more accurately but requiring a separate model pass per pair.
• A common pipeline retrieves tens or hundreds of candidates with a bi-encoder and reranks them to a much smaller final set with a cross-encoder.
• Reranking cannot recover a relevant document omitted by the first stage, so evaluate retriever recall before judging reranker quality.

Multi-stage retrieval narrows a broad candidate pool with increasingly precise methods, while MMR reduces redundancy among the final results.

• An initial sparse, dense, or hybrid search retrieves enough candidates to favor recall over precision.
• Metadata checks, deduplication, and a stronger reranker then reduce that pool before context assembly.
• MMR selects each next chunk by balancing its relevance to the query against its similarity to already selected chunks.
• More diversity can improve topic coverage but admit weaker matches, so tune the relevance-diversity balance against coverage and answer quality.

Text embeddings encode learned features that help place related inputs near one another, not a complete or literal representation of meaning.

• Their relative geometry is shaped by the training data and objective and can capture semantic or syntactic associations, while individual dimensions usually have no stable human-readable meaning.
• A high score indicates proximity in that learned space, but it does not establish entailment, factual agreement, or identical intent.
• Negation, numbers, rare entities, truncation, and domain shift can make apparently similar vectors hide important differences.
• Retrieval quality should be measured on labeled task examples, with exact filters or a reranker handling distinctions that embeddings miss.

I would benchmark candidate models on representative queries and documents while balancing relevance against dimension, throughput, and total cost.

• The training data and multilingual evaluation should cover the domain terminology, languages, and cross-language retrieval patterns the task requires.
• Higher dimension can preserve more signal, but it increases vector storage, memory bandwidth, and index work without guaranteeing better task quality.
• Maximum input length, tokenization, batching throughput, licensing, and per-token or compute cost affect whether a model fits the constraints.
• Recall@k or nDCG on a labeled set should drive the final choice, followed by an end-to-end check with the intended retrieval pipeline.

Asymmetric embeddings encode queries and documents for different roles while mapping both into a shared space where relevant pairs score highly.

• The query encoder or instruction emphasizes a short information need, while the document side represents richer evidence that can satisfy it.
• The two sides may use shared weights with different prefixes or separate encoders trained together with a contrastive objective.
• They are useful when query and document distributions differ, such as a short question retrieving a long passage.
• Each side must use the model's prescribed query and document mode because symmetric encoding can reduce retrieval quality.

These metrics rank vectors differently unless vector norms are controlled, so the search metric must match the model's training objective and normalization.

• Cosine similarity compares direction and ignores magnitude by dividing the dot product by both vector norms.
• Dot product includes both direction and magnitude, which matters when the model has learned useful information in vector length.
• Euclidean distance measures straight-line separation, with smaller values indicating closer vectors.
• For unit-normalized vectors, dot product equals cosine similarity and squared Euclidean distance equals 2 minus 2 times cosine similarity.

I would fine-tune only after a measured baseline shows recurring domain distinctions that a general model fails to capture and suitable training pairs are available.

• Error analysis should first rule out simpler causes such as poor chunking, weak labels, or a mismatched similarity metric.
• Training examples should pair realistic queries with relevant items and include informative negatives under a contrastive or ranking loss.
• The train and validation split should separate related documents, entities, or users so near-duplicates cannot inflate retrieval scores.
• I would compare against the original model on domain recall and broader validation data, then keep the tuned model only if the gain justifies its added cost.

Both provide non-relevant examples that shape the embedding boundary, but hard negatives are deliberately confusing while in-batch negatives are reused for efficiency.

• Hard negatives are close to the query yet incorrect, and they can be mined with lexical search, an earlier model, or human judgments.
• In-batch training treats the positive documents for other queries in the same batch as negatives, creating many comparisons without extra encoding.
• False negatives must be masked because another query's positive may also be relevant to the current query, especially in duplicate or multi-answer data.
• Diverse batches, an appropriate loss temperature, and a mix of easy and hard negatives usually produce more stable learning than only extreme negatives.

HNSW organizes vectors as a multilayer proximity graph and performs an approximate greedy search from sparse upper layers to a dense base layer.

• Each inserted vector receives a random maximum layer and connects to nearby neighbors, with M limiting roughly how many links each node keeps.
• Search descends through upper layers to find a promising region, then explores a candidate queue on the base layer.
• Larger M and efConstruction generally improve graph connectivity and recall, but increase memory, index build time, and insertion work.
• Larger efSearch examines more candidates and usually improves recall at the cost of query latency, and it must be at least as large as the requested result count.

IVF limits search to selected coarse clusters, while product quantization stores compact codes and estimates distances instead of comparing every full vector exactly.

• IVF trains centroids, assigns each vector to an inverted list, and probes only the nearest nprobe lists for a query.
• Product quantization splits vectors or residuals into subvectors and represents each part by the index of a learned codeword.
• Probing more lists improves IVF recall but adds latency, while stronger compression saves memory and bandwidth but increases distance error.
• More centroids, subquantizers, or code bits can improve accuracy at additional training, storage, or lookup cost; exact reranking is possible only if full-precision vectors are retained or fetched, and it cannot recover candidates missing from the shortlist.

Mutable vector stores commonly represent an update as a new insertion and hide old entries with tombstones until compaction rebuilds cleaner index structures.

• An update must replace or invalidate the previous vector for the same logical ID so only the newest live version is returned.
• A delete can mark an entry with a tombstone immediately even when removing it from graph links or compressed segments in place is expensive.
• Searches filter tombstoned records, but accumulated dead entries consume space and can increase candidate work or reduce graph quality.
• Compaction rewrites segments or rebuilds the index from live records, reclaiming space while preserving ID mappings and a consistent view of the data.

Sharding partitions vectors across nodes to increase capacity, while replication keeps copies of each shard to improve availability and read throughput.

• Hashing a high-cardinality key tends to balance data, while tenant-based sharding localizes ownership but can create skew or hot shards when tenants differ greatly in size.
• A global nearest-neighbor query often requests local top results from every required shard and merges them into a final top-k; selective shard routing reduces fan-out but can reduce recall if routing is wrong.
• Replicas must share compatible vectors and index settings, while the chosen consistency policy determines when recent writes become visible on each copy.
• More shards add routing and merge overhead, and more replicas add storage and write work, so both counts should follow capacity, latency, and failure requirements.