Machine Learning Engineer interview questions

I would start with LightGBM because this is medium-width tabular data and the latency budget rewards a strong, cheap baseline.

• Use a time-based split, native missing-value handling, and categorical encodings fitted only on training folds; report PR-AUC and recall at the retention-team capacity.
• A 1,000-tree model can often score in 1 to 3 ms on CPU, leaving room for feature lookup inside the 10 ms budget.
• I would test a neural model only if error analysis shows learnable high-order interactions or embeddings improve PR-AUC by enough to justify higher latency and tuning cost.

I would use two stages: retrieve a few hundred candidates cheaply, then rerank them with richer user and item context.

• Precompute item embeddings and use an ANN index such as ScaNN or FAISS to retrieve about 500 candidates in 20 ms.
• A compact ranking model scores those 500 in under 45 ms, with 20 ms for features and 35 ms reserved for network and tail variance.
• Evaluate retrieval recall@500, ranking NDCG@20, catalog coverage, and online click or conversion lift because a better reranker cannot recover missing candidates.

I would start with a MobileNetV3 or EfficientNet-Lite variant and optimize it on the target phone rather than shrink a server model blindly.

• Train at the smallest resolution that preserves the required classes, then export only operators supported by Core ML or TensorFlow Lite.
• Apply INT8 quantization with a representative calibration set and measure model size, top-1 accuracy, p95 latency, memory, and battery use on low-end devices.
• If INT8 loses more than 1 point, use quantization-aware training or distill from the larger teacher before increasing the architecture size.

I would benchmark a TF-IDF linear model and a small distilled transformer before choosing full BERT.

• A linear baseline gives transparent per-label errors and can process well above 1 million tickets per day within 15 ms on CPU.
• DistilBERT or MiniLM is justified only if macro-F1 on rare intent labels improves materially, for example from 0.71 to at least 0.76.
• Full BERT is a poor default if it needs GPU serving for a gain the business cannot use; I would compare cost per million tickets as well as F1.

I would use a global model because shared seasonal and product signals give sparse series more statistical strength.

• Backtest with rolling origins over the latest 12 weeks and report weighted quantile loss by volume and cold-start cohort.
• Product ID embeddings, price, promotion, calendar, and stock status let one LightGBM, DeepAR, or Temporal Fusion Transformer share patterns across series.
• I would retain simple seasonal-naive forecasts as a fallback and reject the global model if it cannot beat them on low-history products and aggregate inventory cost.

I would let the fast model answer confident cases and send only uncertain examples to the slow model.

• Calibrate the fast model and choose two confidence thresholds on validation data so about 20% of traffic escalates.
• The expected model time is 2 ms plus 0.20 times 45 ms, or 11 ms, before routing overhead.
• Evaluate end-to-end accuracy, escalation by cohort, p99 latency, and cost; distillation may be simpler if the slow path creates unstable tails.

I would ship the ensemble only if the extra 4 recall points cover its cost and the models make genuinely different errors.

• Measure pairwise error correlation and recall at the same false-positive review capacity, not three headline AUC values.
• Benchmark parallel and sequential execution against the p99 budget and calculate dollars per additional prevented fraud case.
• If the ensemble is valuable but too expensive, distill its soft outputs into one student and require recall within 1 point of 82%.

I would test a shared encoder with task-specific heads because the small tasks may benefit from the 10-million-label task.

• Sample tasks explicitly so the largest dataset does not dominate every update, and tune loss weights from gradient scale and validation behavior.
• Compare each task with its single-task baseline, especially for negative transfer on labels whose semantics conflict.
• I would keep separate models if two tasks lose more than their agreed 1-point F1 guardrail or require incompatible serving cadences.

I would use supervised learning only where failure labels cover the modes we need to catch, and anomaly detection for novel or unlabeled deviations.

• Build a time-based test set with all confirmed failures and representative normal operation, then report recall at a fixed daily alert budget rather than accuracy.
• Gradient-boosted trees can learn known failure modes with cost-sensitive loss, while an autoencoder or isolation forest proposes unfamiliar cases for review.
• A hybrid route sends high supervised risk or high anomaly score to inspection, but thresholds must be calibrated to the team's capacity, such as 200 alerts per day.

I would rank by incremental treatment effect because high purchase probability does not mean the discount caused the purchase.

• Randomize an eligible holdout and train an uplift model on treatment, outcome, and pre-treatment features with no post-offer leakage.
• Evaluate uplift or Qini curves and incremental profit in the top 5%, including discount cost, rather than ROC-AUC.
• If overlap is weak or the experiment is too small, use the randomized policy directly instead of presenting an unstable observational estimate as causal.

The effective batch is 4,096 examples, so I would validate optimization at that batch before accepting the throughput gain.

• The calculation is 32 GPUs times 64 examples times 2 accumulated microbatches per optimizer step.
• DDP averages gradients across ranks, while loss normalization, clipping, and the learning-rate schedule must advance per optimizer step, not per microbatch.
• I would test linear learning-rate scaling with warm-up, then compare time to target validation score because 32x hardware rarely gives 32x useful convergence.

Plain DDP is insufficient because each rank would hold roughly 208 GB of mixed-precision parameters, gradients, and AdamW state before activations.

• DDP replicates that state on every GPU, so activation checkpointing alone cannot fit it into 80 GB.
• PyTorch FSDP FULL_SHARD or ZeRO-3 distributes parameters, gradients, and optimizer state across 64 ranks.
• I would wrap transformer blocks, use BF16, and benchmark all-gather overhead and tokens per GPU-second rather than choosing maximum sharding without measurement.

I would keep communication-heavy tensor parallelism within each eight-GPU NVLink node and use data or pipeline parallelism across nodes.

• An 8-way tensor-parallel group avoids sending every layer's collectives over the slower inter-node fabric.
• Pipeline stages are added only if one node cannot hold its layer group, with enough microbatches to keep the pipeline bubble below 10%.
• I would compare candidate layouts by peak memory, tokens per second, exposed collective time, and validation parity on the same global batch.

I would accumulate eight microbatches because 32 times 8 times 8 equals the required global batch of 2,048.

• Scale each microbatch loss by one eighth before backward or average the accumulated gradients exactly once.
• Gradient clipping, optimizer step, EMA, and learning-rate schedule run after the eighth microbatch, while DDP synchronization is skipped for the first seven.
• I would verify BatchNorm or stochastic-layer behavior because accumulation matches gradient size but not every operation of a physical 2,048-example batch.

I would treat linear scaling as a starting hypothesis and retune warm-up and total optimizer steps against validation quality.

• A 16x batch suggests up to a 16x learning-rate trial for SGD, but Adam often needs a smaller increase and explicit stability sweeps.
• Epoch-based schedules now contain 16x fewer optimizer steps, so warm-up, decay, EMA, and checkpoint intervals must be expressed in updates or tokens.
• Compare time and GPU-hours to the same validation target, because faster epochs are not useful if the large batch loses generalization or needs more data passes.

I would choose BF16 because its FP32-like exponent range removes most loss-scaling failures while retaining tensor-core throughput.

• Keep optimizer statistics, selected reductions, and numerically sensitive normalization in FP32.
• Compare step time, peak memory, non-finite gradients, and validation loss over at least one full schedule, not a 100-step speed sample.
• FP16 remains reasonable only if dynamic loss scaling removes the 0.4% failures without skipped updates or quality loss.

I would reduce exposed communication and improve overlap before adding more GPUs.

• Profile gradient bucket readiness and NCCL link throughput to distinguish poor overlap, tiny collectives, topology placement, and a slow rank.
• Test larger microbatches, gradient accumulation, BF16 collectives, and tuned bucket sizes while overlapping reduction with backward.
• Accept a change only if tokens per GPU-second and time to target quality improve, with collective time ideally below 20% of the step.

I would reshape and cache the input path so storage sustains at least 50 GB/s with headroom before scaling compute further.

• Pack examples into immutable sequential shards of 1 to 4 GB instead of millions of small files, with enough shards for independent rank sampling.
• Prefetch shards to node-local NVMe and use parallel decoding, pinned memory, and asynchronous host-to-device copies.
• Measure remote throughput, cache hit rate, decode time, and dataloader wait, requiring wait below 10% of step time on the final dataset.

I would write a sharded checkpoint asynchronously and publish it only after every required shard is durable.

• Save model, optimizer, scheduler, scaler, RNG, sampler position, and completed global step with checksums under one generation.
• Each rank stages to local NVMe and uploads multipart objects to S3 while training continues from an immutable state snapshot.
• A manifest becomes visible only after all shards finish, and restore is tested on the supported world sizes with subsequent loss and sample order checked.

I would use an early-stopping search with a hard GPU-hour budget rather than fully train a fixed number of trials.

• Start with 40 to 80 random or Bayesian trials over conditional ranges and evaluate them at comparable token or step milestones.
• Ray Tune ASHA stops weak trials and promotes the best, while concurrency stays below the storage and dataloader saturation point.
• Reserve about 20% of the budget for repeated seeds and final retraining, then select once on an untouched test set.