Computer Vision Engineer interview questions

A residual block learns a correction to its input, which gives gradients a short identity path through a deep network.

• If the residual branch initially contributes little, the block behaves close to identity instead of forcing every layer to relearn the full representation.
• During backpropagation, the skip path carries a gradient term that does not pass through every weight layer, reducing degradation in very deep models.
• Addition requires compatible spatial and channel dimensions, so a strided or 1 by 1 projection is used when the block changes shape.
• This improves optimization, but it does not by itself prevent overfitting or guarantee that arbitrary extra depth helps.

It splits spatial filtering from channel mixing, replacing one dense convolution with a depthwise convolution and a 1 by 1 pointwise convolution.

• A standard K by K convolution costs roughly K squared times input channels times output channels per pixel.
• The separated form costs K squared times input channels plus input channels times output channels, which is much smaller when channel counts are high.
• The reduced arithmetic and parameters suit MobileNet-style edge models, but low theoretical FLOPs may not yield low latency if the device kernels or memory access are inefficient.
• Separating the operations constrains cross-channel interaction, so accuracy can fall unless width, expansion blocks, or training are adjusted.

An FPN combines semantically strong coarse features with spatially precise fine features to produce useful maps at several resolutions.

• A top-down path upsamples deeper maps while lateral 1 by 1 convolutions align channels from the backbone.
• Elementwise fusion gives high-resolution levels stronger semantics than using early backbone features alone.
• Detector heads assign small objects to fine levels and large objects to coarse levels, limiting the scale range each head must handle.
• Detectron2 and MMDetection expose pyramid levels and assignment rules, which must match image resize policy and expected object sizes.

U-Net skip connections restore localization detail by passing encoder features directly to decoder stages at the same resolution.

• Downsampling builds context but discards precise edges and small structures that upsampling alone cannot reconstruct reliably.
• Concatenation lets the decoder combine low-level geometry with deep semantic features, unlike residual addition whose main purpose is optimization.
• Encoder and decoder shapes must align, so padding, cropping, and input divisibility need explicit handling in PyTorch.
• Excessively strong shallow features can encourage texture shortcuts, so augmentation and decoder design still matter.

A dilated convolution spaces kernel samples apart, enlarging the receptive field without adding pooling or proportionally more parameters.

• DeepLab keeps a smaller output stride and uses atrous spatial pyramid pooling with several dilation rates to capture different context scales.
• Large rates can create gridding artifacts because neighboring outputs sample disjoint input patterns.
• Output stride, dilation, and crop size must be coordinated, since a rate useful on a large feature map may be ineffective on a small one.
• The approach improves dense localization but costs more memory than aggressively downsampled backbones.

I choose from the measured accuracy and latency envelope: one-stage models predict densely in one pass, while two-stage models refine a selected set of proposals.

• YOLO-style one-stage detectors usually offer simpler, faster inference for real-time streams and edge deployment.
• Faster R-CNN can spend more computation on proposal features and often remains attractive for small objects or high-accuracy offline inspection.
• The boundary is not absolute because backbone, image size, TensorRT support, and batching can outweigh the architectural category.
• I benchmark end-to-end preprocessing, postprocessing, memory, and target-class AP rather than compare headline model FLOPs.

Anchor-free detectors predict objects from points or centers instead of scoring a predefined set of boxes at every location.

• Anchor-based models require scales, aspect ratios, and positive IoU thresholds that can mismatch unusual object geometry.
• Anchor-free heads commonly predict class heatmaps plus distances to box sides, reducing anchor hyperparameters and output volume.
• They still need a label assignment rule, such as center sampling or dynamic matching, so they are not free of design choices.
• MMDetection ablations should keep backbone, schedule, resize, and NMS fixed before attributing a gain to the head type.

Faster R-CNN uses a shared backbone, a region proposal network, and a per-proposal head for classification and box refinement.

• The RPN scores anchors and regresses proposal offsets, then proposal NMS keeps a manageable candidate set.
• The original architecture used RoI Pooling; common modern FPN implementations use configurable RoI Align to sample fixed-size proposal features without coordinate quantization.
• Training combines RPN objectness and box losses with second-stage classification and regression losses using matched ground truth.
• At inference, refined boxes are decoded, low scores are removed, and class-wise NMS produces final detections.

Mask R-CNN adds a parallel per-instance mask head to Faster R-CNN and relies on accurately aligned RoI features.

• RoI Align samples floating-point coordinates rather than rounding proposal boundaries, which is important for pixel-level masks.
• The mask branch predicts a small binary mask per class or selected class independently of the box regression branch.
• Training uses a mask loss only for positive proposals, alongside the existing RPN, classification, and box losses.
• At inference, each predicted mask is resized into its detected box, so box errors still limit final mask quality.

A modern YOLO detector uses a backbone, multi-scale feature fusion, and dense heads that jointly learn classes, localization, and often objectness.

• Training resizes and augments images, assigns ground-truth boxes to head locations, and balances classification and IoU-based box losses across pyramid levels.
• The raw head output must be decoded from grid-relative values into image coordinates with the exact stride and preprocessing convention used by that implementation.
• Inference applies confidence filtering and NMS after decoding, and those steps can dominate latency when candidate counts are large.
• Exporting to ONNX or TensorRT requires verifying letterbox reversal, coordinate order, score composition, and postprocessing against the PyTorch output.

A ViT divides an image into fixed-size patches, projects each flattened patch into an embedding, adds positional information, and applies transformer blocks.

• For an H by W image and P by P patches, self-attention receives roughly HW divided by P squared tokens.
• Each token can attend globally to every other token, while a CNN builds its receptive field through stacked local kernels.
• Attention cost grows quadratically with token count, so halving patch size sharply increases memory and compute.
• Dense prediction models reshape or tap token features at several depths because a single class token is insufficient for localization.

A CNN usually provides stronger locality and translation priors, while a ViT offers flexible global interaction but often needs stronger pretraining and deployment support.

• On a small labeled dataset, a pretrained CNN can be easier to fine-tune because its inductive bias reduces the amount of structure it must learn.
• A well-pretrained ViT can outperform it, especially at scale, but attention memory rises quickly with high-resolution inputs.
• GPU throughput does not guarantee camera-stream latency because batch size one, operator fusion, and TensorRT kernel availability matter.
• I compare target-device p50 and p99 latency, memory, and per-class quality after identical preprocessing rather than select by parameter count.

I start with frozen DINOv2 features as a baseline, then fine-tune progressively if the target domain and task require adaptation.

• Self-supervised pretraining learns transferable visual structure from unlabeled images, which can reduce labeled-data needs for retrieval, classification, or dense tasks.
• A linear probe tests representation quality without allowing a large head to hide weak features.
• For a specialized domain such as thermal imagery, unfreezing later blocks with a lower learning rate can adapt semantics while limiting catastrophic forgetting.
• Input normalization, patch resolution, license constraints, and target-device cost must be checked before adopting the checkpoint.

I train the student on ground truth while also matching informative outputs or features from a stronger frozen teacher.

• Temperature-scaled soft class probabilities expose similarities between classes that one-hot labels discard.
• Detection or segmentation distillation may match feature maps, logits, boxes, or masks, but spatial dimensions and foreground weighting must be aligned.
• The distillation weight and temperature are validation hyperparameters because an overconfident or domain-mismatched teacher can transfer errors.
• Success is measured on target hardware: a smaller student is useful only if TensorRT latency or memory improves at acceptable task quality.

Focal loss downweights easy, already-correct examples so numerous background locations do not dominate the classification gradient.

• It multiplies cross-entropy by a factor based on one minus the predicted probability of the true class, controlled by gamma.
• An alpha term can additionally rebalance positive and negative classes, but it serves a different purpose from the focusing term.
• Larger gamma concentrates learning on hard examples, yet noisy labels can then receive excessive weight.
• I inspect positive and negative loss contributions in PyTorch instead of assuming a conventional gamma suits every assignment strategy.

I use overlap-based loss when foreground pixels are sparse and pixelwise cross-entropy would be dominated by background.

• Dice optimizes a soft overlap ratio and gives the small foreground region more influence than its raw pixel count.
• Tversky weights false positives and false negatives separately, so I can penalize missed lesions more than extra mask area.
• The smoothing constant and treatment of empty masks affect gradients and must be explicit.
• Combining Dice or Tversky with cross-entropy often preserves stable pixelwise learning while improving region overlap.

They extend overlap-based regression to provide useful geometry signals in cases where plain IoU is weak or zero.

• IoU directly optimizes overlap but has no gradient with respect to separation when predicted and target boxes do not intersect.
• GIoU adds a penalty from the smallest enclosing box, giving a signal to disjoint boxes but sometimes converging slowly.
• DIoU directly penalizes center distance, while CIoU also includes an aspect-ratio term.
• I validate the loss with the detector's box parameterization and assignment because a better standalone formula does not guarantee higher COCO mAP.

Label assignment decides which predictions receive positive, negative, or ignored supervision, so it shapes the detector's gradient as much as the loss does.

• Fixed IoU thresholds are simple but can yield too few positives for small or unusual objects.
• Center sampling limits positives to plausible locations, while dynamic schemes use current classification and localization costs to select matches.
• One-to-one Hungarian matching, used by DETR-style models, assigns each object to one query and removes duplicate supervision.
• Assignment must be inspected per object size and class because unstable or excessive positives can overwhelm later NMS tuning.

I collect high-scoring false positives from realistic data and feed a controlled subset back into training as explicit negatives.

• For a detector, I run the current model over unlabeled or negative scenes and review crops such as reflections, logos, or background textures it confuses with the target.
• Sampling focuses on informative errors without letting millions of near-identical negatives dominate each batch.
• Deduplication and human verification are necessary because hidden true objects mislabeled as negatives damage recall.
• I keep a fixed challenge set to confirm mining improves the failure mode without reducing performance on ordinary positives.

They resolve overlapping predictions differently: NMS suppresses boxes, Soft-NMS decays scores, and WBF combines coordinates.

• Greedy NMS keeps the highest-scoring box and removes lower-scoring boxes above an IoU threshold, which is fast but can drop nearby objects.
• Soft-NMS reduces overlapping scores instead of deleting them, preserving candidates in crowded scenes at additional postprocessing cost.
• WBF averages coordinates using confidence weights and is most useful for well-calibrated model ensembles, not as a drop-in replacement for every single detector.
• Thresholds are tuned per class and density using end-to-end metrics, and ONNX or TensorRT export must preserve the chosen semantics.