AI Engineer Interview Questions (2026): Agents, Retrieval, Evals and What They Listen For

Start by drawing the line that the job titles blur. An AI engineer, also posted as applied AI engineer or forward deployed engineer, builds systems on top of models they did not train: retrieval, agents, tool calling, evaluation harnesses, guardrails and the product surface around all of it. A machine learning engineer owns the model itself, the training pipeline and the serving path.

In a large number of AI engineering roles there is no model training at all. That is not a watered-down version of the ML job. It is a different job, and it is the largest and fastest-growing function on this board. It is also the most common way into AI from ordinary software engineering, and a software engineering background is an advantage here rather than a gap: the work is latency budgets, retries, caching, failure recovery, observability and product judgment, applied to a component that returns a different answer every time.

The postings say the same thing. On 7 September 2026, of 2,852 live AI roles on this board, 877 mentioned agents, more than PyTorch at 608, and the seven platform terms (AWS, GCP, Spark, Kubernetes, SQL, CI/CD and Azure) totalled 2,571 mentions against 1,058 for PyTorch, TensorFlow and JAX combined. Full method and the rest of the ranking are in the skills report. If your experience is systems rather than modelling, apply to AI engineering roles, not to ML engineering roles. Candidates lose offers to that mismatch every week.

Scenario questions

This is the core of the loop and the part that decides it. Every question below is a failure the team has already had.

"Your retrieval system returns answers that are confident and wrong. Debug it."

What the interviewer is listening for: that you split retrieval from generation before touching either. First measure whether the correct passage was in the retrieved set at all. If it was not, the fault is chunking, embedding choice, the query itself or missing content, and no prompt change will fix it. If it was there and the answer still went wrong, the fault is in generation or grounding. Strong candidates also say what the system should do when nothing relevant is found, which is refuse rather than improvise.

"An agent loops on the same tool, and in one run it deleted a record. What do you change?"

What the interviewer is listening for: containment before cleverness. Step budgets, loop and repeat detection, and a hard timeout are the immediate answers. Then the real one: destructive tools need a different tier from read-only tools, with confirmation, scoped credentials, a dry-run mode and an audit trail. Anyone who answers only with a better prompt has not run an agent in production.

"The prompt worked on every example you tried and fails for real users. Why?"

What the interviewer is listening for: awareness that your test inputs were written by you. Real inputs are shorter, ambiguous, multilingual, pasted from elsewhere, adversarial, or about something out of scope. The move they want is to sample actual production inputs, cluster the failures and fix the largest cluster, rather than patching the prompt on each complaint.

"Your model provider deprecates the version you shipped on. What is your plan?"

What the interviewer is listening for: whether you built the exit in advance. A provider abstraction, prompts and model choices in configuration rather than code, and above all an eval suite that can be run against the new version to quantify what changed. Without that suite the migration is a guess, and saying so is the point of the question.

"Costs tripled the month after launch. Bring them down."

What the interviewer is listening for: measurement before optimisation. Find out where the tokens go: retrieved context, chat history, retries, an agent taking too many steps, or one endpoint used far more than expected. Then caching, trimming context, routing easy requests to a smaller model, and capping agent steps. Naming the quality metric you will hold constant while you do it is the mark of someone who has done this before.

"In a customer demo, the model invents a refund policy. What do you do, in the room and after?"

What the interviewer is listening for: composure and ownership. In the room, correct it immediately and plainly, state what the system is grounded on, and do not blame the model. Afterwards, reproduce it, add it to the eval set, and fix the class of failure rather than the instance, which usually means grounding policy answers in retrieved source text and refusing when there is none. This is often the last question and it is scored heavily for judgment.

"Build an eval harness for this feature. There is no ground truth."

What the interviewer is listening for: that you create the ground truth instead of waiting for it. Collect real inputs, hand-label a small set including the failures that prompted the work, write down what good means as criteria rather than a number, then automate: deterministic assertions where you can, a model judge where you cannot, validated against your human labels before you trust it. Say plainly that an unvalidated judge is not evidence. This overlaps directly with model evaluation roles.

Knowledge questions

• What does retrieval actually fix, and what does it not? It supplies missing knowledge and gives you a citation; it does not stop the model contradicting a passage it was given.
• How do you evaluate a system whose output changes between identical runs?
• Chunking trade-offs: what breaks with chunks that are too small, and what breaks with chunks that are too large?
• How do you manage a context window as a conversation grows, and what do you drop first?
• Tool calling: what happens when a tool errors, returns nothing, or is called with invalid arguments?
• Where do guardrails belong, on the input, on the output, or in the tool layer, and what does each placement miss?
• What can you cache in a system like this, and what does streaming buy you that caching does not?

Judgment questions

• Prompting, retrieval or fine-tuning? Missing knowledge points to retrieval, wrong format or task framing points to prompting, and a behaviour you cannot steer into the model at a volume that justifies the maintenance points to fine-tuning. Reach for the cheapest reversible option first and say what evidence would move you up.
• How do you know it is good enough for users? Good answers set a bar before launch, name the worst acceptable failure rather than the average case, and include a staged rollout with a way to turn the feature off.
• Who owns a failure the model caused? You do. The system is the product, and the provider is an implementation detail to everyone outside your team.

Questions you should ask them

Nobody writes this part, and the answers tell you whether the team has shipped this kind of system or is about to learn on you.

• How do you evaluate this feature today, and who looks at the results?
• What is in your eval set, and where did the examples come from?
• What was the worst failure that reached a user, and what changed afterwards?
• What is the latency budget, and what is the monthly spend per active user?
• Are you tied to one model provider, and how quickly could you switch?
• Which parts of the product are agentic, and what can an agent do without a human confirming?
• Who writes the prompts, and where do they live: in code, in configuration, or in a tool nobody reviews?

A team that cannot describe its eval set is running on demos. That is worth knowing before you accept, not after.

Before you interview

Build one small system end to end and keep the receipts. Retrieval over a corpus you care about, or an agent with two or three real tools, plus an eval set of thirty inputs you wrote by hand and a note of what your first version got wrong. That artefact answers most of the scenario questions above, and it is the difference between having opinions about this work and having done it.

Then read the postings. Start with open AI engineering roles, the companies hiring, remote AI roles, and, early in your career, entry-level AI roles. Pay and disclosure patterns across the board are on the market data page.