AI Research Engineer interview questions

I would scope the papers as deltas from one shared checkpoint and compare every method with an unchanged continuation from that checkpoint.

• At a measured 40% MFU against an H100's roughly 0.99-PFLOP/s dense BF16 peak, one H100-hour supplies about 1.42e18 FLOPs; dense training is about 6ND total, or 6N per token.
• Building the shared 7B checkpoint on 100B tokens costs about 2,950 hours, then three method continuations plus one unchanged control at 50B tokens each cost about 5,900 hours.
• The remaining roughly 3,150 hours fund 1B and 3B mechanism ablations, evaluation, and retries; a paper needing a 300B-token run must supply a checkpoint or be reported as a partial reproduction.
• I would report paired method-minus-control deltas at matched tokens and core FLOPs and call a claim reproduced only when its preregistered effect survives a fresh confirmation within the 12,000-hour ledger.

I would require every proposal to clear a cheap falsification gate before it can receive one of the 6 target-scale slots.

• Each proposal declares one causal claim, one primary eval, a +0.7-point minimum useful effect, a failure condition, and exact token and FLOP controls.
• At measured 40% MFU, a 5B-token screen on 1B and 7B costs under 170 H100-hours per idea, so all 18 can be screened for roughly 3,100 hours without pretending they are 70B evidence.
• Each selected 8,000-hour 70B run is capped near 27B dense-equivalent tokens before overhead, and slots are ranked by prediction-adjusted information gain with 2 reserved for high-uncertainty ideas.
• A branch advances only if fresh proxy evidence predicts the target, the 70B result exceeds +0.7 with its corrected interval above zero, and no guardrail regresses over 0.3 point.

I would allocate compute in stages and derive token caps from measured cost at each context length so weak methods never reach 128K.

• I would spend 4,000 H100-hours on one 7B baseline, packing checks, and exact layer-shape FLOP and MFU measurements at 8K, 32K, and 128K.
• The 3 methods then receive 2,000 hours at 8K and 3,000 at 32K, exactly 15,000 hours total, with token caps set by measured valid tokens per H100-hour rather than assumed equal exposure.
• Only methods gaining at least 5 retrieval points at 32K with under 1 point of 8K regression split the remaining 21,000 hours for fresh-seed 128K confirmation.
• I would report both iso-token and iso-FLOP views and select a winner only if its cluster-aware 95% interval is positive and its quality per H100-hour beats baseline by at least 20%.

I would preregister the causal hypotheses, analysis plan, and compute-dependent stopping rules before launching the first large run.

• I would name 2 primary outcomes, their exact dataset versions, decoding settings, aggregation, and a minimum effect of 1.5 points.
• The 24 comparisons would use a declared hierarchy or Holm correction with family-wise alpha 0.05 rather than selective uncorrected wins.
• I would lock 3 seeds, exclusion criteria, missing-run handling, and a 10% holdout that remains unopened until architecture selection ends.
• I would accept a claim only if the corrected test passes, the effect exceeds 1.5 points, and no preregistered safety slice falls by more than 1 point.

I would use prespecified group-sequential looks rather than checking the dashboard after every run.

• I would schedule looks after 10%, 25%, 50%, 75%, and 100% of compute with alpha-spending boundaries such as O'Brien-Fleming.
• Futility stops when the conditional power of reaching the 0.8-point target falls below 15%, while early success requires the adjusted upper boundary.
• Model sizes and seeds arrive in balanced blocks so an early look never confounds scale with one lucky seed or data shard.
• I would stop and claim success only at the adjusted boundary, stop for futility below 15%, and otherwise spend the full 60,000 GPU-hours.

I would publish the result as a bounded test of the claimed effect, not as proof that the idea can never work.

• I would release configs, seeds, data hashes, checkpoint lineage, and a discrepancy table against all 3 original implementations.
• Equivalence testing would ask whether effects outside a preregistered -0.7 to +0.7 band can be rejected, alongside the ordinary confidence interval.
• I would include sensitivity analyses over 3 model sizes and 2 data mixtures to identify where the null result does and does not transfer.
• I would call the negative result conclusive only if equivalence passes and all 6 conditions exclude a gain larger than 0.7 points.

I would fit a compute-optimal frontier on proxies and make the final parameter-token choice from the fixed core-FLOP ledger.

• Under the dense approximation C about 6ND, the full 2.4e25 FLOPs imply about 57.1T tokens at 70B or 10.0T at 400B, with intermediate candidates computed from D equals C divided by 6N.
• I would fit loss on 1B, 3B, 7B, and 13B runs over at least 4 token-to-parameter ratios, charging all pilot FLOPs to the same ledger if 2.4e25 is the program cap.
• Exact layer-shape accounting would add attention, embeddings, and any recompute separately, while a measured MFU converts the chosen FLOPs to wall-clock without changing the scientific budget.
• I would spend at most 5% on pilots and confirmation, then choose the smallest candidate inside the predicted-loss interval unless a sealed downstream set shows at least a 1-point gain.

I would run separate iso-token and iso-FLOP panels because one comparison cannot hold data exposure and active compute fixed at once.

• At 2T tokens, the dense 6ND term is about 6 times 180B times 2T, or 2.16e24 FLOPs, while the 50B-active MoE term is about 6.0e23 before architecture-specific corrections, a roughly 3.6-fold difference.
• The iso-token panel gives both models the same ordered 2T-token corpus and reports that compute difference rather than calling the arms matched.
• The iso-FLOP panel starts with dense at 2T tokens and MoE at about 7.2T, then adjusts the MoE token cap using exact shared-attention, active-expert, and router accounting while reporting auxiliary-loss and recompute FLOPs separately.
• Three seeds for both nominal panels consume about 2.12e25 of the 6ND-style model term, leaving roughly 8.8e24 of the 3e25 cap for exact corrections, executed overhead, evaluation, and retries; token caps fall if that reserve is insufficient.

I would screen normalized model shapes on proxies, then spend the remaining pilot budget on a direct 120B bridge.

• At measured 40% MFU, a 10B-token run costs about 84 H100-hours at 2B and 337 hours at 8B under 6ND; 4 shapes, 3 seeds, and both sizes total about 5,050 hours.
• Shapes would match parameter count and training FLOPs while varying layers and hidden width, with head dimension, MLP ratio, tokenizer, and data order controlled rather than forcing an impossible identical head count.
• I would use about 5,100 hours for one 5B-token 120B run of the selected shape and one matched control, leaving roughly 1,850 hours for profiling, evaluation, and retries.
• I would choose the shape only if the paired 120B bridge confirms the proxy loss direction and measured step time, activation memory, and TP traffic stay on the predicted Pareto frontier.

I would use proxy screening and short continuations from one 70B checkpoint because 50,000 hours cannot fund independent long-context pretraining at all 3 lengths.

• At measured 40% MFU, the dense 70B term alone costs about 420B FLOPs per token, so 50,000 H100-hours cover at most about 169B aggregate tokens before the extra attention cost at 64K and 256K.
• I would spend 10,000 hours screening RoPE and sparse-attention choices on 7B, reserve 35,000 for matched continuations of the top 2 methods from one 70B checkpoint, and hold 5,000 for evaluation and retries.
• Each length gets both an iso-token result and an iso-FLOP result using measured layer-shape FLOPs, because quadratic attention makes equal token counts increasingly compute-unequal.
• I would advance only a method whose sealed 256K retrieval and reasoning improve by at least 8 points while 8K quality loses under 0.5 point and the ledger remains below 50,000 hours.

I would screen 6 fusion arms on 7B and 13B proxies and reserve target-scale compute for only one or two 100B confirmations.

• The 6 arms are early, intermediate, and late fusion crossed with 2 matched projector or cross-attention parameter budgets, all using the same frozen multimodal sample stream.
• At measured 40% MFU, 2 seeds of each 7B arm at 5B tokens cost about 1,770 H100-hours total, and one 10B-token run for each 13B arm costs about 3,290 hours before vision overhead.
• I would reserve about 8,450 hours for 2 leading 100B designs at 5B tokens and 2 fresh seeds each, leaving roughly 6,500 of the 20,000 hours for encoders, evaluation, profiling, and failures.
• A fusion design advances only if multimodal quality gains at least 2 points, text-only quality loses under 0.3, and the 100B confirmations agree without any arm receiving hundreds of billions of tokens.

I would cap routing research before the 30,000-hour commitment and test specialization, capacity, and network cost together.

• At measured 40% MFU, 3 routers with 2 seeds on 7B at 5B tokens cost about 885 H100-hours, and the top 2 with 2 seeds on 30B at 3B tokens cost about 1,520 hours.
• The roughly 4,000-hour pilot cap also covers domain-shift replays and all-to-all profiling, with active FLOPs, shared attention, router FLOPs, and dropped tokens reported separately.
• I would compare top-8, expert-choice, and auxiliary-loss-free routing on the same token stream while measuring load CV, routing entropy, specialization, overflow, and link-level traffic.
• I would commit the 30,000-hour run only if a router beats a dense compute control by 0.7 point, keeps overflow below 0.1%, and holds load and traffic gates on fresh seeds.

I would gate scale-up with paired candidate and control continuations from identical checkpoints at each model size.

• Two seed blocks at 70B require four runs of 10B tokens, about 11,800 H100-hours total at measured 40% MFU.
• The paired 70B deltas must land inside the proxy prediction interval, reduce loss by at least 0.4%, and improve sealed downstream quality by at least 0.7 point.
• Only then would I spend about 25,300 hours on four 300B runs of 5B tokens, one candidate and one control for each of two seeds, leaving roughly 7,900 hours for profiling, evaluation, and failures.
• I would claim transfer only if both paired 300B deltas remain positive and their mean is at least 60% of the preregistered extrapolation at matched core FLOPs.

I would estimate a quality-compute frontier across sampling, verification, and search rather than optimize a single decoding recipe.

• I would compare best-of-N at N of 1, 4, 16, and 64, verifier reranking, and tree search using the same 1e9-token budget.
• Each method would report AIME and GPQA-Diamond accuracy against generated tokens, verifier FLOPs, diversity, and calibration over 5 seeds.
• I would reserve 20% of tokens for a hidden test set so search parameters are not tuned on the reported questions.
• I would adopt a method only if it gains at least 8 AIME points and retains at least 80% of that gain when token budget is halved.

I would begin with TP 8, PP 8, CP 1, and DP 64, with EP 8 partitioning the DP dimension rather than multiplying the world size.

• The world size is TP times PP times CP times DP, so 8 times 8 times 1 times 64 equals 4,096; EP 8 leaves expert data parallelism of 64 divided by 8, or 8.
• TP 8 stays inside each 8-GPU NVLink node, PP spans 8 stage groups, and EP all-to-all uses topology-selected groups of 8 corresponding ranks across nodes.
• Sequence parallelism reuses the TP group, but context parallelism is a separate world-size factor; choosing CP 4 would reduce DP to 16 and expert-DP to 2 under the same 4,096 GPUs.
• I would accept the layout only after layerwise byte accounting and 256 to 1,024 GPU pilots predict under 72 GB peak, under 25% communication, and over 80% scaling efficiency at 4,096 GPUs.

I would start with TP 8, PP 2, and DP 128 so each collective follows the shards that actually share data.

• TP 8 occupies one NVLink node, PP pairs 2 nodes, and the product 8 times 2 times 128 equals all 2,048 GPUs.
• TP collectives remain inside NVLink, pipeline sends use NIC-local rank pairs, and each DP collective joins the corresponding TP and PP shard across 128 replicas over InfiniBand.
• I would not perform an intra-node DP reduction first because the 8 local TP ranks own different tensor shards; cross-node DP reduce-scatter would instead use rail-aware rings or trees and measured overlap.
• Microbenchmarks by message size and a staged 256, 1,024, and 2,048 GPU run must hold collective time below 20%, p99 link tails within 10% of median, and full-scale efficiency above 82%.

I would compare complete feasible training paths rather than treat FSDP, ZeRO-3, and Megatron-LM as interchangeable features.

• Candidate paths are composable PyTorch TP 8 plus FSDP2 FULL_SHARD over DP 128, DeepSpeed ZeRO-3 plus TP 8 where the exact graph is supported, and Megatron Core TP 8 plus PP 2 plus DP 64 with its distributed optimizer.
• Each path uses the same token stream, global batch, precision, and mathematical kernels where available, while topology differences and core FLOPs are reported rather than hidden.
• A 22B one-step parity suite precedes 175B pilots that measure peak and transient all-gathers, MFU, collective overlap, checkpoint resharding, and loss over 2,000 steps.
• I would choose only a path below 72 GB per rank, within 0.1% loss of the reference, above 45% MFU, and able to restore into a changed DP size without custom checkpoint surgery.

I would separate global persistent bytes, local sharding, transient materialization, and activations in one executable memory model.

• Globally, BF16 parameters are about 440 GB, BF16 gradients 440 GB, FP32 Adam moments 1.76 TB, and an optional FP32 master copy 880 GB, for 3.52 TB if all 4 categories exist.
• Under TP 8, PP 4, and DP 64, full state sharding divides persistent state across all 2,048 ranks, while a distributed optimizer that replicates BF16 parameters keeps about 13.75 GB of parameters per model shard and shards only eligible state over DP.
• I would add the largest per-layer all-gather, reduce-scatter buckets, activations from exact microbatch and sequence shapes, attention workspaces, NCCL buffers, fragmentation, and at least 10% headroom.
• Launch requires an instrumented shape-faithful proxy to match predicted peaks within 5% and every target rank to remain below 72 GB through the first optimizer step and checkpoint save.

I would select recompute by measured GB saved per added millisecond and require enough headroom for the actual 32K length distribution.

• The current peak is about 98 GB, so the policy must save at least 26 GB to reach a 72 GB operating target, not merely cross below the nominal 80 GB limit.
• I would compare selective attention, MLP, and full-block non-reentrant checkpoint regions on identical packed batches, recording saved tensors, extra FLOPs, step time, and gradient parity.
• If microbatch already equals 1 and selective recompute cannot reach 72 GB under a 20% time cost, I would add context parallelism or reduce the trained context instead of claiming a nonexistent batch reduction.
• I would keep the cheapest policy that passes 1,000 steps below 72 GB on every rank, matches gradients within tolerance, and survives p99 sequence lengths without allocator retries.

I would use measured stage costs and an interleaved schedule because plain 1F1B cannot meet an 8% bubble target at modest microbatch counts.

• A feasible world layout is TP 8, PP 16, and DP 8, whose product is 1,024 GPUs, with 6 physical layers assigned to each pipeline stage by time rather than count alone.
• With 2 virtual chunks per stage and 128 microbatches, the ideal interleaved bubble term is roughly 15 divided by 256, or 5.9%, leaving limited room for imbalance.
• I would sweep 96 to 160 microbatches while holding global tokens constant, then include embeddings, loss, recompute, sends, receives, and activation residency in the schedule model.
• The chosen schedule must measure below 8% bubbles for 2,000 steps, keep every stage within 3% of median time, stay under 72 GB, and beat plain 1F1B throughput by at least 7%.