Machine Learning Engineer Interview Questions (2026): Scenarios and What They Listen For

Before anything else, work out which of four jobs you are actually interviewing for. The titles are used interchangeably in job descriptions, and the interviews behind them are not the same interview.

• A machine learning engineer owns training and serving a model in production. The deliverable is a running system with a latency budget, a retraining path and a rollback plan.
• A research scientist owns a novel result. The deliverable is a finding, and the screen usually involves a publication record and a defence of your own work.
• A data scientist owns analysis and inference for a decision. The deliverable is a conclusion someone acts on, and the interview leans on experiment design, causality and communication.
• An applied AI engineer usually integrates a model somebody else trained: retrieval, agents, evaluation harnesses and the product surface around them.

Say this plainly because it costs people offers: candidates apply to the wrong one of these four and get screened out for a mismatch they never saw. An engineer with three shipped production models is rejected by a research posting for having no publications. A researcher with strong papers is rejected by an ML engineering posting for never having owned a serving path. Read the responsibilities section, not the title, and then pick your door: machine learning engineer roles, research roles, or applied AI engineering roles.

Scenario questions

These are the bulk of a real loop and the part that decides the outcome. None has a clean answer. The interviewer is grading how you narrow an ambiguous failure.

"Your model looks great offline and performs badly in production. Walk me through what you check."

What the interviewer is listening for: an ordered search, not a list of possibilities. Strong answers check the boring causes first: is the production input distribution the same as the evaluation set, are the features computed by the same code, is the label you optimised the outcome the product actually cares about, and is the offline metric measuring the decision threshold you deployed. Weak answers jump straight to the model architecture.

"Describe training and serving skew and a time you caused it."

What the interviewer is listening for: whether you have lived it. The answer they want names a concrete mechanism: a feature computed with a full-table aggregate in training and a partial one at request time, a fill value that differs between the batch job and the service, a timestamp that leaks future information offline. Admitting you caused one scores higher than describing it in the abstract.

"An upstream data pipeline changed silently. Your metrics did not move for two weeks, then fell off. What now?"

What the interviewer is listening for: detection before repair. They want to hear about input monitoring that is independent of model output: schema checks, null rates, category cardinality, distribution comparisons against a fixed reference window. Then the containment question, which is whether you roll back the model or freeze the feature, and finally the contract with the upstream team so it cannot recur silently.

"A product manager wants better answers from a language model. Do you fine-tune, prompt, or retrieve?"

What the interviewer is listening for: that you diagnose before you choose. Missing knowledge is a retrieval problem. Wrong format, tone or task framing is a prompting problem. A behaviour the model cannot be steered into, at a volume that justifies the maintenance, is a fine-tuning problem. The candidate who names the cheapest reversible option first, and says what evidence would push them up the ladder, wins this question.

"Three months after launch, the model is worse. Nothing was deployed. Explain."

What the interviewer is listening for: a real vocabulary for drift. Covariate shift, label shift and concept drift are different problems with different fixes, and the case candidates forget is the feedback loop: the model changed user behaviour, which changed the data, which degraded the model. Then, how you would know before a customer does.

"Cut inference cost in half without hurting the product."

What the interviewer is listening for: ordering by cost of change. Batching, caching and right-sizing hardware come before quantisation, distillation or a smaller model, and a cascade routing easy cases to a cheap model comes before retraining anything. The signal is whether you name the quality metric you hold constant.

"You discover your evaluation set leaked into training. What do you do?"

What the interviewer is listening for: honesty under pressure and a recovery plan. Every reported number is now unreliable, so the first move is to say so to the people who acted on it. Then a clean split, ideally by time or by entity rather than at random, a re-measurement, and a check for the same leak in sibling projects. Candidates who try to salvage the old numbers fail this question.

Knowledge questions

Shorter and checkable, used to confirm the scenarios were not rehearsed. Expect a follow-up that pushes on the edge of your answer.

• What is overfitting, and how do regularisation, early stopping and more data each attack it differently?
• How would you detect distribution shift in production without labels?
• When is batch inference the right call and when do you need online serving?
• Give me a case where a validation split lies to you. Random splits on time series and on grouped entities are the two answers they expect.
• What makes an evaluation decision-grade rather than merely favourable?
• Data parallel against model parallel training: what breaks first as you scale, and why?
• What do you actually lose to quantisation, and how would you measure the loss rather than assume it?

Evaluation is the one to over-prepare. It is the shared language between this seat and model evaluation roles, and it is where the follow-up questions get hardest.

Judgment questions

Usually asked by the hiring manager. There is no correct answer, only a defensible one.

• When should you not use machine learning? The answer they want includes rules, heuristics and a lookup table. No labels, no feedback signal, a hard requirement to explain every decision, or a problem a deterministic rule solves are all good reasons to decline.
• How do you decide a model is good enough to ship? Strong answers set the bar before training against the current baseline, including doing nothing, and name the worst acceptable outcome for the worst affected user segment rather than the average.
• More data or a better model? They are listening for a way to find out cheaply, such as a learning curve on subsampled data, an error analysis over a stratified sample, and an estimate of what labelling would cost.

Questions you should ask them

Almost nobody writes this section, and it is the half of the interview you control. These answers tell you whether the team really runs models in production.

• Who owns retraining, and is it scheduled, triggered by a metric, or done by hand?
• How long after a prediction does the label arrive? A long delay changes the whole job.
• What was the last model rollback, and what caused it? If there has never been one, ask how a bad model would be caught.
• Has an offline metric ever disagreed with the business metric, and who decided what to do?
• Who computes features at serving time, this team or another one?
• What fraction of my first six months is expected to be pipelines and platform work rather than modelling?
• Which model in production is the one nobody wants to touch?

If the team cannot answer the retraining and rollback questions, the role is closer to building the platform than to modelling on it, which may be exactly what you want. That is the overlap with AI infrastructure roles.

Before you interview

Take one model you have shipped and write out the whole path in your own words: the target, the label and its delay, the features and where they are computed, the offline and online metrics, the serving mode, the monitoring, and the one time it went wrong and what you did. Almost every scenario question above is answerable from that one narrative, and interviewers can tell within a minute whether you are describing a system you ran or one you read about.

Then read live postings rather than guides, because the responsibilities section is where a company tells you which of the four jobs this actually is. Start with open ML engineer roles, the companies hiring, and, early in your career, entry-level AI roles. Pay and disclosure patterns across the board are on the market data page.