Automation Engineer interview questions
100 real questions with model answers and explanations for Senior Automation Engineer candidates.
See a Automation Engineer resume example →Practice with flashcards
Spaced repetition · Hunter Pass
Questions
I would provide a thin control plane and versioned SDKs rather than force every repository into one framework.
- Repositories publish a standard test manifest with suite type, owner, resource class, timeout, and expected duration.
- The control plane schedules isolated runners, streams normalized results, and keeps p95 queue time below two minutes.
- Teams retain Playwright, pytest, JUnit, or Go test adapters, while the platform owns authentication, artifacts, policy, and observability.
Why interviewers ask this: The interviewer evaluates whether the candidate can standardize platform contracts without creating a framework monoculture.
I would classify tests by boundary and failure ownership, then attach execution budgets to each class.
- Pull requests get unit and component tests under five minutes, service integration tests under ten, and only critical end-to-end journeys under fifteen.
- Contract tests verify service boundaries without full environments, while nightly suites cover broader browser and data combinations.
- Each test declares one taxonomy value in its manifest, and quarterly reports expose teams that move cheap assertions into expensive layers.
Why interviewers ask this: A strong answer turns the test pyramid into enforceable latency and ownership rules rather than a diagram.
I would use surviving mutations as diagnostic evidence about weak assertions, not make the aggregate mutation score a release gate.
- I would group survivors by pricing risk and mutation operator, then review equivalent or unreachable mutations so tool noise does not drive test work.
- For high-risk survivors, I would add behavior-focused unit, contract, or property tests and confirm that they fail for the changed rule without duplicating implementation details.
- I would track killed high-risk mutations, escaped pricing defects, and test runtime by rule area, while avoiding a universal score target that teams can game.
Why interviewers ask this: The interviewer checks whether mutation testing guides risk-based improvements without rewarding metric manipulation or low-value tests.
I would support a small protocol and certify adapters, not maintain a universal framework.
- Every adapter must emit the same test ID, timestamps, status, retry attempt, owner, and artifact references.
- The platform team supports one golden adapter per language and gives community adapters a compatibility suite with a six-month support window.
- Unsupported framework internals remain team-owned, which preserves choice without turning six platform engineers into library maintainers.
Why interviewers ask this: A strong answer defines a durable interoperability boundary and an explicit support model for polyglot teams.
I would require deterministic discovery, machine-readable execution, and a stable result protocol before certification.
- Discovery must list immutable test IDs, tags, owner, timeout, resource needs, and historical grouping without running test bodies.
- Execution must honor shard filters, cancellation within 30 seconds, and separate test failures from runner failures.
- A conformance suite injects timeouts, crashes, duplicate events, and malformed output, and the plugin must pass it on every release.
Why interviewers ask this: The interviewer evaluates whether plugin compatibility covers lifecycle and failure semantics, not only happy-path result parsing.
I would preregister the dependency experiment, use a matched control, and keep artifact persistence off the terminal-result critical path.
- Steady-state hypothesis: at 12,000 jobs per hour, at least 99.95% of accepted jobs outside the fault cohort reach a terminal result within 60 seconds, no more than 0.1 percentage point below the control.
- In one availability zone, two non-overlapping cohorts each capped at 2.5% of global traffic run separately: one adds exactly 300 ms of latency, while the other injects 3% explicit errors without added latency; a routing guard keeps combined exposure at or below 5%.
- The artifact client has a 500 ms per-call timeout and opens its circuit breaker when at least 20 of the last 50 calls time out or fail, marking evidence pending for bounded asynchronous retry; automation aborts if unaffected completion breaches the hypothesis, impact exceeds one zone or 5% globally, or result p95 exceeds 60 seconds.
- Cleanup removes the proxy rule, waits for healthy probes to close the breaker, drains retries, and reconciles accepted jobs, terminal results, artifact references and checksums, and missing evidence before closure.
Why interviewers ask this: The interviewer evaluates a hypothesis-driven experiment on an external dependency, zone-scoped blast radius, timeout and circuit-breaker behavior, automatic aborts, cleanup, and complete evidence reconciliation.
I would give each run namespaced, generated data with explicit lifecycle rather than maintain a mutable shared fixture database.
- A data API creates tenant-scoped records from versioned factories and returns identifiers instead of fixed emails or account numbers.
- Workers use unique run IDs and idempotent cleanup, with a 24-hour TTL as protection when cancellation skips teardown.
- Production-derived fixtures are tokenized before storage, and schema compatibility tests run whenever a factory or service contract changes.
Why interviewers ask this: The interviewer checks isolation, reproducibility, privacy, and cleanup at parallel enterprise scale.
I would use semantic versions, repository compatibility telemetry, and staged automated upgrades.
- Supported majors run side by side through a declared migration window, and security fixes are backported according to the published support policy.
- Ten repositories, then 10%, 50%, and the full fleet are exposure stages only; each promotion requires a minimum execution count and an acceptable upper confidence bound on SDK-attributable regressions, with zero tolerance for declared severe outcomes.
- The old major remains supported while any supported repository still uses it unless that repository completes migration or has an approved, time-bound exception.
Why interviewers ask this: A strong answer combines package versioning with statistically defensible fleet evidence and a deprecation policy that does not abandon supported consumers.
I would standardize a lossless execution envelope and keep domain-specific evidence as typed attachments.
- The common model contains run, suite, shard, test, attempt, status, timing, owner, environment, code revision, and failure category.
- Screenshots, traces, HAR files, device logs, and load-test histograms stay in object storage behind immutable references.
- Raw adapter output is retained for 14 days so normalization bugs can be repaired without rerunning expensive suites.
Why interviewers ask this: The interviewer evaluates whether normalization enables shared analytics without discarding evidence unique to each test domain.
I would define user-visible SLOs for admission, scheduling, execution fidelity, and result availability.
- The API targets 99.9% successful job admission and p95 admission latency below one second over 28 days.
- Queue p95 stays below two minutes for standard jobs, and 99.95% of accepted test events appear in final results within 60 seconds.
- Runner availability is segmented by resource class so scarce GPU or device capacity cannot hide degradation of the normal pool.
Why interviewers ask this: A strong answer measures the platform from team workflows and separates control-plane health from test failures.
I would use a durable regional work queue with an idempotent global control plane and data-local workers.
- Discovery creates immutable shard specifications, while leases with heartbeats allow abandoned work to return to the queue safely.
- Regional schedulers place shards near dependencies and upload append-only events to a globally replicated result store.
- The control plane never synchronously coordinates individual tests, limiting cross-region latency and reducing its failure blast radius.
Why interviewers ask this: The interviewer checks decomposition, lease semantics, regional locality, and removal of a central throughput bottleneck.
I would schedule with weighted fair queues and explicit per-tenant concurrency rather than global FIFO.
- Each team receives a baseline share, while unused slots can be borrowed and reclaimed as other queues become active.
- Pull-request jobs outrank nightly jobs, but aging raises long-waiting work so low priority does not mean never.
- Limits use runner-minutes rather than job count because one 32-core, 60-minute job is not equal to a two-minute unit suite.
Why interviewers ask this: A strong answer handles fairness, priority, starvation, and unequal resource consumption in one scheduling policy.
I would reject the sub-15-minute target under the current granularity because one indivisible test already takes 20 minutes.
- That test must be split, optimized, moved out of the blocking lane, or the SLA raised above its measured p90 or p99 before sharding can meet the target.
- For the remaining tests, I would key duration history by test ID and environment and use longest-processing-time bin packing with conservative estimates for unknown tests.
- I would validate shard predictions against setup time and duration variance, reserving headroom instead of planning to the deadline.
Why interviewers ask this: The interviewer evaluates whether the candidate identifies an impossible constraint before applying a sharding algorithm and validates the revised plan with measured tail latency.
I would use single-use virtual machines or hardened microVMs for untrusted code, not a shared Docker daemon.
- Jobs receive short-lived credentials scoped to synthetic test accounts and cannot reach production networks or cloud metadata endpoints.
- Egress is allowlisted through a proxy, workspace disks are destroyed after each job, and caches are content-addressed and read-only.
- Trusted release jobs run in separate accounts and node pools, accepting higher cost to prevent credential and kernel crossover.
Why interviewers ask this: A strong answer treats test code as arbitrary code and chooses isolation based on credential and kernel risk.
I would bind idempotent events to run, shard, test, attempt, and fencing epoch, then reduce them with explicit state transitions.
- Discovery persists the expected test IDs and count, while event sequence ranges expose gaps and globally unique event IDs remove duplicates.
- The reducer accepts one terminal state for the current attempt and epoch, records contradictions, and rejects every stale-epoch write.
- Finalization occurs only when every expected test has an accepted terminal state or is explicitly classified as missing at the deadline; a shard-terminal event alone never proves completeness.
Why interviewers ask this: The interviewer checks expected-result accounting, sequence-gap detection, fencing, terminal-state semantics, and honest treatment of missing tests.
I would persist an immutable replay manifest with the original attempt and state which dependencies were not captured rather than promise bit-for-bit reproduction.
- The manifest pins source revision, image digest, package artifacts and lockfiles, runner toolchain, configuration, feature flags, and fixture and schema versions.
- It records random seeds, clock and timezone, input data hashes, and sanitized external request and response contracts with immutable artifact references.
- Replay runs in an isolated network from those references and reports missing or mismatched state; uncaptured third-party behavior is diagnostic evidence, not an equivalent replay.
Why interviewers ask this: The interviewer evaluates hermetic replay inputs, immutable provenance, captured nondeterminism, and explicit limits on replay fidelity.
I would cancel superseded branch runs cooperatively, then enforce termination after a short grace period.
- The control plane marks the run cancelled, stops new shard leases, and sends cancellation tokens to active workers.
- Tests get 30 seconds to upload diagnostics and clean resources before the runner process or Pod is terminated.
- Merge-queue and release runs are exempt, while reusable caches publish only after successful atomic completion to avoid poisoning later jobs.
Why interviewers ask this: The interviewer evaluates cancellation propagation, cleanup, cache integrity, and exceptions for authoritative runs.
The mean demand is 600 concurrent runners, so I would start near 800 and validate it against burst arrival data.
- The calculation is 9,000 times four divided by 60, with roughly 33% headroom for peaks, node startup, and maintenance.
- Autoscaling follows queue age and requested CPU and memory, not job count alone, with a two-minute p95 queue target.
- Reserved baseline capacity handles normal load, while spot workers absorb interruptible nightly suites to control cost.
Why interviewers ask this: A strong answer combines correct concurrency math with multidimensional resources, burst headroom, and purchasing strategy.
I would optimize immutable inputs near runners while preserving clean execution environments.
- Frequently used signed images are pre-pulled on warm nodes, and images are slimmed so the p95 pull stays below 30 seconds.
- Maven, npm, and browser binaries use content-addressed regional caches keyed by lockfile and toolchain, never writable shared directories.
- Cache hit rate and bytes transferred prove the gain, while a cold-cache canary verifies jobs remain reproducible without cached state.
Why interviewers ask this: The interviewer checks whether startup optimization preserves isolation, integrity, and cold-path reproducibility.
I would make residency a hard scheduling constraint and fail over only within the permitted geography.
- Job manifests label data class and allowed regions, and admission rejects a suite whose dependencies violate those labels.
- EU queues can burst across two European regions but never to US runners, even if that increases completion time.
- Capacity alerts fire before queue p95 exceeds five minutes, and the business explicitly funds EU reserve rather than relying on illegal spillover.
Why interviewers ask this: A strong answer treats residency as a placement invariant and makes the capacity cost visible.
Locked questions
- 21
A repository has 40,000 tests and must give merge feedback within ten minutes. Which checks would block a pull request?
feedbackcode-reviewtesting - 22
A monorepo contains 180 services and shared libraries. How would you build the changed-service graph used for test selection?
monorepo - 23
A test-impact model selects 12% of tests, but leadership wants proof that it is safe before making it a merge gate. How would you validate it?
validationtesting - 24
Thirty services deploy independently and integration environments are unreliable. How would contract tests become a release gate?
contractdeployment - 25
Define automated canary analysis for a service with a 0.3% baseline error rate and 20,000 requests per minute.
deployment-strategies - 26
You must create a release-confidence score for 70 services. Which inputs and decision boundaries would you use?
- 27
How would you define quality SLOs for a product that releases 300 times per week across 45 services?
slo - 28
A merge queue handles 300 pull requests per day, but retesting every queue position takes 35 minutes. How would you redesign it?
code-reviewdata-structures - 29
The same quality gate is applied to a documentation site and a payment service. How would you introduce risk tiers across 120 repositories?
documentationquality-gates - 30
An auditor asks why release 7.4.2 was allowed into production six months ago. What evidence should the platform retain?
- 31
Design ephemeral environments for 200 pull requests per day, each needing 12 services and a PostgreSQL database within eight minutes.
databasepostgresdesign - 32
Preview environments take 25 minutes to provision, but product teams need them in under seven. How would you reduce the critical path?
schedulingcommunication - 33
Forty teams share three Kubernetes clusters for test workloads. How would you enforce multi-tenancy?
kubernetes - 34
A load-test team regularly consumes all CPU and delays functional tests for 20 teams. How would you prevent noisy-neighbor failures?
testing - 35
Teams have copied a 2,000-line Terraform preview-environment stack into 60 repositories. How would you restructure it?
terraform - 36
The company requires test execution in AWS and Azure, but the platform team has only eight engineers. What would you abstract?
- 37
Ephemeral test infrastructure costs 420,000 dollars per month and must be reduced by 30% without increasing queue p95 above three minutes. What would you do?
data-structures - 38
Preview environments need realistic customer workflows but cannot expose production PII or long-lived secrets. How would you provision them?
secretspii - 39
How would you detect flaky tests statistically across 15 million executions per month without labeling every rare failure as flaky?
flaky - 40
A suite has 700 flaky tests and quarantining all of them would remove 18% of critical coverage. What quarantine policy would you use?
coverageflaky - 41
A vendor proposes self-healing selectors that automatically rewrite failed browser tests. Where would you allow and forbid healing?
procurementtesting - 42
A 12-minute end-to-end test fails only in CI, and logs cannot show where time was spent. How would you instrument the path?
e2e - 43
Design a quality telemetry schema that can compare 300 repositories without misleading teams with different test volumes.
schemadesign - 44
A predictive quality model reports 92% accuracy when only 3% of releases have defects. Is that sufficient to gate releases?
defects - 45
Your release-risk model performs well offline but degrades after teams change their test strategy. How would you detect invalidity?
test-strategy - 46
Forty teams receive a quality dashboard, but none changes behavior. Which operational views would you replace it with?
- 47
How would you secure the software supply chain for a test platform executing code from 250 repositories?
supply-chain - 48
Only 6 of 35 teams adopt a new test manifest standard after three months. With six platform engineers, how would you change the rollout?
decision-making - 49
You must migrate 9,000 Selenium tests in 80 repositories to Playwright within nine months without freezing feature delivery. What plan would you use?
seleniumplaywright - 50
A proposed test platform needs 1.2 million dollars and eight engineers for one year. How would you justify or reject it for 40 teams?
- 51
A network partition leaves 180 Kubernetes test runners active in two regions, and 7% of jobs appear owned by both schedulers. What do you do?
partitioningkubernetes - 52
A webhook retry bug turns the normal 900-job CI queue into 74,000 jobs in 12 minutes while runners continue completing 1,400 jobs per minute. How do you contain the storm?
resiliencewebhooksdata-structures - 53
Your platform reports 312 duplicate completions among 48,000 test jobs after workers retry timed-out result uploads. How do you determine the valid result?
resilience - 54
After a message-broker failover, 1,860 of 60,000 test jobs have no result, although runner logs show 1,420 finished. What is your recovery plan?
recovery - 55
The scheduler predicts that every pull-request suite will finish within 15 minutes, but setup-time variance causes 22% of runs to miss that promise. How would you model and validate completion forecasts?
dispersionpromisesvalidation - 56
Runner CPU averages 46%, yet job start p99 rises from 90 seconds to 28 minutes for Android builds. What do you inspect before buying capacity?
capacity - 57
Spot termination removes 240 of 600 runners in 4 minutes and 3,700 tests restart from the beginning. How do you reduce recovery time without accepting partial results?
testing - 58
Artifact downloads add 17 minutes to a 24-minute suite after 900 runners simultaneously fetch a 6 GB image. Registry egress is capped at 50 Gbps. What do you change?
artifactsregistries - 59
A priority policy lets release jobs jump the queue, but 640 nightly jobs have waited over 9 hours and their data expires after 12 hours. How do you prevent starvation?
data-structures - 60
An autoscaler reacts to CPU, oscillates between 300 and 1,200 runners every 15 minutes, and queue p99 reaches 41 minutes. How do you stabilize it?
reactdata-structuresscaling - 61
Flaky failures jump from 2.4% to 18.7% across 36 repositories after a runner image release. What do you do in the first hour?
flaky - 62
A cleanup defect leaves 2.8 million test orders, and 23% of billing tests now read polluted customer histories. How do you recover safely?
defects - 63
Parallel tests accidentally reuse tenant tokens, and logs show 47 cross-tenant reads among 1.2 million API calls. What is your incident response?
incidentsapitokens - 64
Staging passes 4,600 tests, but production fails because Java is 21.0.6 in CI and 17.0.12 in production, with 84 other package differences. How do you prevent another drift escape?
iac - 65
An automated quarantine service removes 1,140 tests after two failures each, cutting the required suite by 38% overnight. What do you do?
testing - 66
A self-healing locator changes 286 failed checkout tests from the Pay button to a similarly named Save payment method button, producing green runs. What do you change?
locators - 67
A test fails 18 times in 600 runs on one browser and 1 time in 620 on another. How do you decide whether the browser difference is actionable?
- 68
A shared test-data creation API serves 6,000 setup calls per minute, but p99 rises to 40 seconds during releases and consumes half the pull-request budget. How would you redesign and validate it?
validationapi - 69
Exactly 9% of tests fail only around midnight UTC after the fleet moved from NTP to a faulty local time source with 11-second skew. How do you prove and fix it?
testing - 70
A self-healing environment controller restarts databases whenever a smoke test fails, causing 14 restarts and 52 minutes of downtime in one day. How do you redesign the response?
databasesmoke - 71
A CI gate blocks 19% of releases because coverage moves from 82.0% to 81.9%, while escaped defects are unchanged. What gate do you replace it with?
coveragedefects - 72
An API provider must make a field required, but 7 of 43 consumers deploy only monthly and the release is in 10 days. How do you roll out the contract?
deploymentapi - 73
Test impact analysis selects 74 of 8,200 tests, but a production defect appears in an unselected tax calculation used through runtime reflection. What changes?
defects - 74
A 5% canary is green for 30 minutes, but it received only 46 payment attempts versus 1,100 expected and excluded iOS users. Do you promote?
deployment-strategies - 75
After an Appium image update, 68% of 320 Android devices remain offline and the mobile release window closes in 3 hours. What do you do?
mobile - 76
A release gate passes because a migration job exits successfully after processing only 68 million of 80 million rows. What automation change prevents this?
migrationsconcurrency - 77
You plan a chaos experiment for a regional result store while 20% of CI traffic is live. How do you test failover without turning resilience validation into an outage?
experimentsvalidation - 78
Impact analysis skips all browser tests for a CSS-token change, but 14% of checkout sessions use a high-contrast theme assembled at runtime. How do you adjust selection?
cssa11ytokens - 79
A canary analyzer declares a regression because conversion is 4.1% versus 4.5% after 800 sessions, but assignment placed twice as many returning users in control. What do you do?
sessionsdeployment-strategies - 80
Appium sessions take 14 minutes to start on 52 iPhones after parallel runs heat devices above 43 C and battery health protection throttles charging. How do you recover the fleet?
throttlesessionsmobile - 81
A CI log exposes a production API token for 23 minutes and it was downloaded by 14 users. What do you do first?
tokensapi - 82
A compromised test-report package sends environment variables to an external host in 186 CI jobs before detection. How do you respond?
config - 83
Secret scanning reports 9,400 findings after a regex update, but only 31 are confirmed credentials and release pipelines are blocked. How do you restore signal safely?
secretsregexci-cd - 84
An RPA bot creates 126 duplicate supplier payments totaling $480,000 after a timeout hides the confirmation page. What do you do?
resilience - 85
A credential-rotation bot locks 3,800 employee accounts because it retries an old password five times per account. How do you recover?
passwords - 86
A procurement bot selects the wrong table row after a UI redesign and approves 74 invoices above the $25,000 limit. What controls do you add?
procurement - 87
OCR reads 1,250.00 as 125,000 on 38 invoices, creating $4.7 million in proposed payments. How do you redesign the automation?
- 88
A Terraform plan proposes deleting 86 production databases after a module rename changed resource addresses. The maintenance window is 45 minutes. What do you do?
databaseterraform - 89
A Kubernetes cleanup job matches the label app=test and deletes 312 production pods because the label exists in both namespaces. How do you contain and prevent recurrence?
kubernetes - 90
An unattended RPA process closes 9,600 customer cases after a CRM pagination change repeats page 1 eighty times. How do you repair the incident?
incidentspaginationconcurrency - 91
A load test claims 18,000 transactions per second, but production traffic is 35% writes and the script sends 98% cached reads. What conclusion do you publish?
transactionscachingload-testing - 92
At 6,000 requests per second, a load tool reports p99 of 220 ms, but it pauses request generation while responses are slow. Production p99 is 3.1 seconds. What is wrong?
- 93
A test platform meets 99.9% monthly availability but still has two 20-minute pull-request outages every week. What multi-window alerting policy would you derive from its SLO?
sloalerting - 94
Synthetic checkout monitors pass, but their dedicated accounts receive feature flags and fraud rules that differ from real users. How would you validate and govern synthetic representativeness?
monitoringfeature-flagsvalidation - 95
Test observability spend rises from $38,000 to $146,000 per month after every Selenium command becomes a span. What do you cut?
seleniumobservability - 96
A schema migration has copied 62 million of 90 million rows when error rate reaches 8%; old and new applications are both live. How do you roll back?
schemamigrationsrollback - 97
2 hours before release, 11 of 2,200 tests fail: 8 are known flakes, 2 expose invoice rounding errors, and 1 test environment is unreachable. The launch supports a signed contract. What do you recommend?
test-environments - 98
A junior engineer changes a shared wait from 5 to 30 seconds, and suite p95 grows from 24 to 71 minutes across 60 repositories. How do you mentor after the failure?
mentoring - 99
A release manager wants to ignore a canary alert because only 6 of 12,000 sessions lost saved drafts, while rollback takes 18 minutes. What decision do you make?
rollbackalertingsessions - 100
After a reviewer approved a script that deleted 18 shared test environments, a mid-level engineer is reluctant to approve any automation change. How do you rebuild judgment?
test-environments