LLM Engineer interview questions

Ingestion prepares a searchable knowledge representation, while query-time components retrieve and assemble evidence for one request.

• Ingestion parses sources, normalizes metadata, chunks content, creates embeddings, and updates indexes with stable document identities.
• Query-time work transforms the question, applies access filters, retrieves and reranks candidates, then packs context for generation.
• Versioning the source, chunker, and embedding model makes index contents reproducible and supports controlled reindexing.

Fixed-size chunking follows token or character counts, while structure-aware chunking preserves document units such as headings, paragraphs, tables, or code blocks.

• Fixed windows are simple and predictable but may cut a semantic unit at an arbitrary point.
• Structure-aware chunks often improve interpretability and citation quality when the source has meaningful boundaries.
• Both approaches still need size limits because one section can exceed the embedding or generation budget.

Semantic chunking places boundaries where the topic or meaning changes rather than at only fixed lengths.

• It may compare adjacent sentence embeddings or use a model to identify coherent sections.
• Better coherence can improve retrieval, but the method costs more and may produce uneven chunk sizes.
• Thresholds and resulting chunks must be evaluated on domain questions instead of assumed superior to simpler splitting.

Parent-child retrieval indexes small child chunks for precise matching but returns a larger parent section for generation context.

• Small children improve the chance that a focused query matches the relevant passage.
• The parent restores surrounding definitions and references that a tiny chunk may omit.
• Parent identifiers and offsets must remain stable so retrieved children map to the correct source section.

Multi-vector retrieval stores several representations for one source item so different aspects of it can match a query.

• Vectors may represent the original text, summaries, headings, extracted questions, or separate fields.
• Any matching representation can retrieve the same source, improving recall for varied query phrasing.
• More vectors increase index size and duplicate matches, so results need grouping and source-level scoring.

Overlap should preserve information crossing chunk boundaries without filling the index and prompt with near-duplicates.

• Narrative prose may need sentence-level continuity, while code and tables need boundaries based on their own structure.
• Excess overlap inflates storage, biases ranking toward repeated text, and wastes context tokens.
• The setting should be selected with retrieval and answer evaluations across representative document types.

Dense retrieval embeds queries and passages into a compatible vector space and ranks passages by vector similarity.

• Dual-encoder models compute query and passage vectors separately, enabling document embeddings to be indexed in advance.
• Dense retrieval captures semantic similarity and paraphrases but may miss exact identifiers or rare terms.
• Retrieval quality depends on the embedding model, input formatting, corpus, metric, and index configuration.

Sparse retrieval remains strong when exact lexical evidence is more informative than semantic similarity.

• It handles product codes, names, error strings, legal phrases, and rare domain terms well.
• BM25 weights term frequency and rarity without requiring an embedding model or vector index.
• Its misses on paraphrases complement dense retrieval, which is why the two are often combined.

Hybrid search retrieves from both systems and fuses their rankings or calibrated scores into one candidate list.

• Reciprocal Rank Fusion combines rank positions without assuming BM25 and vector scores share a scale.
• Weighted score fusion requires normalization and validation because raw scores have different meanings.
• Fusion weights and candidate depths should be tuned against relevance labels for the target query distribution.

Metadata filtering should restrict candidates as early as the retrieval system can enforce required access and domain constraints correctly.

• Pre-filtering avoids retrieving unauthorized tenants, languages, dates, or document classes.
• Post-filtering can leave too few results because forbidden candidates already occupied the nearest-neighbor slots.
• Filter selectivity and index support affect recall and latency, so the chosen vector store behavior must be verified.

A cross-encoder scores a query and candidate together, allowing token-level interaction that separate embeddings cannot model.

• It is usually more accurate for fine relevance distinctions than a dual encoder.
• Joint encoding is too expensive for the full corpus, so it reranks a limited first-stage candidate set.
• The first retriever must still have high recall because a reranker cannot recover a missing document.

Late-interaction models keep multiple token-level vectors and compare them at query time instead of compressing all meaning into one vector.

• This preserves finer-grained matching signals for terms and passages.
• It can offer a quality and cost point between dual encoders and full cross-encoders.
• The trade-off is larger indexes and more expensive scoring than single-vector retrieval.

Query rewriting converts the user's input into a clearer retrieval query while preserving the original information need.

• It can resolve conversational references, expand abbreviations, normalize terminology, or remove irrelevant phrasing.
• The original question should remain available for generation because a rewrite may lose intent or constraints.
• Rewrite quality needs evaluation since a fluent but incorrect rewrite can make every later stage fail.

Multi-query retrieval generates several alternative searches for one information need and merges their candidates.

• Variants can cover synonyms, different aspects, or lexical and semantic formulations.
• The method can raise recall when one query formulation misses relevant evidence.
• It adds model and retrieval cost and may introduce topic drift, so deduplication and evaluation are required.

HyDE asks a model to generate a hypothetical relevant document and uses its embedding to retrieve real documents.

• The hypothetical text can resemble the language and detail of an answer more closely than a short question.
• Retrieval uses the generated representation, not the hypothetical claims as factual evidence.
• HyDE can help difficult queries but can also steer retrieval toward a model's unsupported assumptions.

Query decomposition is useful when one question contains several facts or reasoning steps that require different evidence.

• Each subquery retrieves a focused set of passages instead of forcing one embedding to represent the whole request.
• Results must be joined under the original question so relationships and constraints are not lost.
• Unnecessary decomposition increases latency and can compound errors across generated subquestions.

Contextual compression extracts or selects the parts of retrieved content most relevant to the current query before generation.

• It reduces token use and distraction when source chunks contain substantial irrelevant material.
• Compression may use rules, sentence selection, or another model.
• The compressor can remove qualifiers or evidence, so faithfulness must be evaluated against the original source.

Retrieval embeddings are commonly trained with contrastive objectives that pull relevant pairs together and push irrelevant pairs apart.

• Positive pairs may be query-document, title-passage, duplicate, or semantically equivalent examples.
• Hard negatives teach the model to separate plausible but wrong candidates from true matches.
• The training pair distribution determines which notion of similarity the model learns.

Asymmetric embedding models use different instructions, prefixes, or encoders because queries and documents play different retrieval roles.

• A query is often short and expresses an information need, while a passage contains the answer in a different style.
• Correct prefixes align both inputs with the training objective and shared vector space.
• Using the document format for queries can reduce quality even when vector dimensions still match.

Normalization changes vector magnitude and determines when cosine similarity, dot product, or Euclidean distance produce equivalent rankings.

• For unit-normalized vectors, cosine similarity equals dot product, and Euclidean distance is monotonically related.
• Without normalization, dot product also reflects vector magnitude.
• The index metric must match the embedding model's recommendation and the way vectors were stored.