DevOps Engineer interview questions

A CI pipeline should put the fastest and cheapest feedback first, then increase cost and risk gradually.

• Run linting and static checks first, unit tests next, then build the container and test that built image in integration tests.
• Run security scans only after cheaper checks pass, and publish or deploy only after every quality gate succeeds.
• This ordering prevents a formatting error from waiting behind a ten-minute build and avoids spending resources on already-broken commits.
• On pull requests, stop before publishing; on merge to main, publish the artifact and run deployment stages.

CI builds are slow because clean runners lack the persistent layer cache available on a developer machine.

• Give BuildKit a persistent source through registry-backed cache-from and cache-to, or mount the CI platform's cache storage into the build.
• Use BuildKit cache mounts for package-manager directories such as pip or npm so downloads survive builds without becoming image layers.
• Copy dependency manifests and install dependencies before copying application code, preserving expensive layers when only source files change.
• Verify cache hits in CI logs, because remote caching cannot compensate for a Dockerfile whose early layers change on every commit.

Cloud access in CI should use short-lived, narrowly scoped identities instead of stored access keys.

• With OIDC federation, the CI provider issues an identity token per job, the cloud validates it, and the job assumes only the role it needs.
• No static cloud secret then exists to leak through logs or forks, and there is no long-lived credential to rotate manually.
• Long-lived keys tend to remain unrotated and accumulate broad permissions, increasing both the chance and impact of compromise.
• If OIDC is unavailable, use short-lived tokens from a secrets manager, masked variables, least privilege, and scheduled rotation.

Build once, deploy many means promoting one tested, immutable artifact unchanged through every environment.

• Build and test one container image, then promote the exact same digest through development, staging, and production.
• Inject environment-specific configuration at runtime rather than baking it into separate images.
• Rebuilding per environment can change resolved dependencies, build inputs, or the base image, so production receives a binary that staging never tested.
• Tie immutable tags to the commit and never use mutable latest for promotion, preserving provenance and reliable rollback.

Duplicate GitHub Actions pipelines should become versioned shared workflows with thin service-specific callers.

• Put workflow_call definitions with typed inputs and secrets in a central repository, leaving each service repository to pass only its parameters.
• Use composite actions for smaller reusable step sequences, such as configuring cloud credentials or building an image.
• Reference released tags rather than main so breaking changes can be adopted deliberately instead of affecting all repositories at once.
• Document an escape hatch for services with legitimate exceptions without returning the whole organization to copied workflows.

Pipeline optimization starts with measured bottlenecks, not assumed ones.

• Collect per-stage and per-step timings across a few hundred runs to locate time spent on dependency installation, image builds, and serial tests.
• Cache dependencies and build layers, then parallelize independent jobs and shard test suites.
• Move slow full end-to-end suites out of the pull-request critical path into post-merge or nightly runs when their risk profile allows it.
• Set a duration budget and alert on regressions so performance remains owned instead of drifting back to forty minutes.

Trunk-based development shifts CI/CD toward fast, reliable validation of small changes merged frequently into main.

• The pipeline must be quick and trustworthy enough to gate every merge, while incomplete functionality stays behind feature flags rather than branches.
• Long-lived branches delay integration, accumulate conflicts, and turn staging into a queue of large, partially tested merges.
• Trunk-based development and feature flags support continuous deployment by exposing integration problems early and reducing batch size.
• GitFlow-style branches favor release trains and larger batches; this can suit regulated release schedules but reduces deployment frequency.

Manual approval belongs only where a person contributes judgment that automated gates cannot provide.

• Use it for high-risk production changes, irreversible operations such as data migrations, or compliance requirements for a second reviewer.
• Implement approval through protected environments with a clearly defined approver group and enough context for an informed decision.
• Requiring a click for every routine deployment creates latency, encourages rubber-stamping, and batches changes into riskier releases.
• Gate routine releases with automated tests and canary metrics, reserving human review for cases someone will genuinely inspect.

A reliable rollback redeploys the previous known-good artifact without requiring a new build.

• Keep immutable, versioned images in the registry and make deployment tooling able to select any retained version through a tested one-command path.
• Rehearse rollback procedures, because an untested runbook is likely to fail during an incident.
• Git revert alone waits for build and test while production is failing, and it is unusable if the pipeline itself is broken.
• Use backward-compatible schema changes and expand-contract migrations so data written by the new version does not prevent the old artifact from returning.

A monorepo pipeline should rebuild and deploy only affected services while retaining a full-build safety net.

• Use path filters so each service reacts to changes in its own directory and in shared paths it consumes.
• Model shared-library dependencies with Nx, Bazel, Turborepo, or a maintained script that computes affected services from the diff.
• Run a complete build nightly because path and dependency-graph logic can miss relationships or contain bugs.
• Keep deployment independent per service so one team's change cannot release another team's unfinished work.

Self-hosted runners are justified by network, hardware, caching, or high-volume cost requirements that hosted runners cannot meet well.

• Choose them for private-network access, GPUs or large memory, persistent caches, or workloads where hosted per-minute pricing is uneconomical.
• The team then owns patching, capacity, scaling, monitoring, and regular replacement of the runner fleet.
• Prefer ephemeral or frequently recycled runners so one build cannot leave state for the next.
• Treat forked pull-request code as remote code execution: isolate its network and credentials, using hosted runners by default and locked-down ephemeral self-hosted runners only where needed.

GitOps makes Git the source of desired cluster state and lets an in-cluster controller reconcile reality to it.

• Manifests or Helm values live in Git, and a deployment is a merged pull request that changes an image tag for Argo CD or Flux to pull.
• CI no longer needs cluster credentials because the controller inside the cluster fetches and applies the change.
• Reconciliation detects manual drift and can report or revert it, making desired state continuously enforced rather than applied once.
• Rollback is a Git revert in the state repository, while Git history provides the audit trail of who changed what.

A build pipeline should make every input traceable and prevent unverified artifacts from reaching deployment.

• Pin third-party CI actions by commit SHA, base images by digest, and dependencies through lock files instead of mutable references.
• Generate an SBOM during the build and scan dependencies and images for known vulnerabilities, blocking critical findings by policy.
• Sign built images so deployment targets can verify that the artifact is exactly what CI produced.
• Never pass secrets through build arguments or layers, because they remain recoverable through image history.

Database migrations in continuous deployment must remain compatible with both old and new application instances during rollout.

• Run migrations as a separate init job or pipeline stage before deploying application code.
• Use expand-contract: add a nullable column, deploy code that writes both formats, backfill data, switch reads, and remove the old column in a later release.
• Keep each intermediate schema valid while old and new instances use it simultaneously.
• Do not ship a migration and code that immediately depends on it in the same deployment, preserving safe rollout and rollback.

One artifact can serve every environment only when all varying configuration stays outside the image.

• Inject environment variables, mounted configuration files, or configuration-service values at deployment time according to twelve-factor principles.
• Keep a shared base plus small Kustomize overlays or Helm value files so environment differences remain visible in review.
• Deliver secrets separately through a secrets manager or sealed secrets, never as plain values committed to the repository.
• Validate required configuration at startup so a missing or invalid value fails during deployment rather than during a later production request.

A Jenkins-to-GitHub-Actions migration preserves delivery concepts but should redesign the execution model rather than translate jobs line by line.

• Jenkins stages map to jobs, shared libraries to reusable workflows or composite actions, agents to runners, and credentials to environment secrets or preferably OIDC.
• Replace a stateful Jenkins server and years of plugin configuration with declarative repository-owned YAML and ephemeral runners.
• Remove snowflake job settings and unnecessary plugins instead of reproducing their accumulated behavior in the new platform.
• Inventory jobs that assume persistent workspaces, plugins with no Actions equivalent, and undocumented behavior hidden on the Jenkins server.

Infrastructure code should pass layered static, plan, integration, and staging checks before production.

• Start with terraform fmt and validate, tflint, and policy-as-code rules that reject risks such as public S3 buckets or 0.0.0.0/0 ingress.
• Run terraform plan on every pull request and review the plan diff as the primary artifact, including the proposed blast radius rather than only HCL style.
• For modules with meaningful logic, create them in an ephemeral sandbox account, assert behavior, and destroy the resources after the test.
• Apply the same change to staging infrastructure as the final safety layer before promoting it to production.

CI notifications become actionable when each failure reaches its owner with enough context to start triage immediately.

• Send pull-request failures directly to the author and main-branch service failures to the owning team's channel.
• Do not broadcast failures to people who cannot act on them, which prevents the wall-of-red channel everyone eventually mutes.
• Include the failed stage, a useful error excerpt, a log link, and the difference from the last green run in every notification.
• Track product test failures, infrastructure flakes, and environment outages as separate signal classes so routine noise does not teach teams to ignore red status.

A multi-stage Dockerfile separates compilation from execution so production receives only the files required at runtime.

• The first stage uses the full toolchain image, installs dependencies, and compiles the binary or bundles the application.
• The final stage starts from a minimal distroless, Alpine, or slim base and copies only the built artifact and runtime dependencies.
• Compilers, development headers, package managers, and source code stay out of production, reducing image size and attack surface.
• Stable dependency installation remains in early cached layers while the tiny final stage rebuilds and ships quickly.

A 1.8 GB image should be reduced by addressing the largest structural sources first and then preventing regression.

• Inspect the base image first; replacing a full OS or large language image with slim or distroless usually gives the biggest low-effort saving.
• Use docker history or dive to find build tools, apt or pip caches, copied source trees, and missing .dockerignore entries such as .git or node_modules.
• Use multi-stage builds to exclude build tools and remove package caches in the same RUN instruction that creates them.
• Add an image-size limit to CI after the reduction so silent bloat cannot return.