Articles

Azure Machine Learning Interview Questions: Endpoints, Deployments, and Traffic Routing

Prepare for Azure Machine Learning interview questions on endpoints, deployments, traffic routing, shadow testing, version diagnosis, and safe model rollouts.

Azure Machine Learning interview questions about endpoints, deployments, and traffic routing become easier when you trace one request. The endpoint is the stable entry point; a deployment is the serving configuration behind it, as Microsoft's online endpoint documentation explains. A deployment can be healthy and return the correct prediction during a targeted test while ordinary traffic continues to reach an older model. Your answer should identify which route the client used before changing infrastructure.

This article focuses on managed online endpoints with Azure CLI ML v2.

Evidence boundary: service behavior is grounded in current Microsoft documentation. No verified candidate report establishes that an employer asks these exact scenarios. The diagnostic examples are constructed practice exercises; preparation advice is our inference, not a company interview rubric.

What belongs to an endpoint, and what belongs to a deployment?

An online endpoint is the address and invocation interface through which clients request real-time inference. An online deployment contains the resources and configuration that perform that inference. Microsoft documents this separation so multiple deployments can share an endpoint and support controlled model releases. Official endpoint concepts.

In your sketch, put the endpoint above two deployments. Keep model artifacts, runtime configuration, and prediction routing separate so you can identify which change would resolve the problem.

Configuration	Where it belongs	What to inspect
Endpoint name and authentication mode	Endpoint	Client address and invocation credentials
Traffic and mirror allocations	Endpoint	Request routing policy
Registered model and environment references	Deployment	Artifact and dependency versions
Scoring code, instance type, and instance count	Deployment	Inference behavior and capacity

The endpoint YAML reference defines endpoint settings. Its deployment counterpart supports versioned references such as azureml:risk-model:17 and azureml:risk-env:4, alongside scoring-code and compute settings. Those are illustrative asset names, not resources you already have. Official deployment schema.

Can a deployment with zero traffic still be tested?

Yes. Microsoft's rollout documentation shows invoking a deployment directly while its ordinary traffic allocation is zero. The CLI's --deployment-name option selects that deployment. At the HTTP level, the azureml-model-deployment header can override the endpoint's configured load balancing. Targeted invocation, routing override.

For a workspace with an existing risk-endpoint and a payload matching its scoring contract, compare these illustrative commands:

az ml online-endpoint invoke --name risk-endpoint \
  --deployment-name green --request-file request.json

az ml online-endpoint invoke --name risk-endpoint \
  --request-file request.json

The first tests Green directly. The second exercises the configured default route. Both require the intended workspace/resource-group context and appropriate access. These examples follow the official CLI interface; they were not executed against an Azure subscription for this article.

Our preparation recommendation: always say what your test proves. A successful targeted call supports Green's ability to score that input. It does not establish the default routing map, representative production performance, or correct behavior for every input shape.

Why does a healthy endpoint return the old model?

Consider an original fraud-risk exercise. Blue references model version 17 and environment 4. Green references model version 18 and environment 5. Both pass readiness checks. A targeted Green request produces the expected new output, but the application continues returning version 17.

The endpoint traffic map still assigns Blue 100 percent. Green's creation succeeded; its promotion never happened. This is a routing explanation, so adding replicas to Green would leave the observed response unchanged.

Trace the request before changing configuration. First, confirm the application's scoring address, subscription, workspace, and endpoint identity. Second, check whether its client or gateway adds a deployment-selection header. Third, read the endpoint's traffic and mirror maps. Fourth, inspect the selected deployment's model, environment, and scoring-code references.

Finally, reproduce the same payload through both routes and correlate the requests with deployment logs. In this exercise, add an application-defined model fingerprint to diagnostic output or restricted logs. Azure does not automatically guarantee that your prediction response includes a model_version field; your scoring code must provide the instrumentation you rely on.

If normal calls reach Green but still produce the old result, the investigation moves to artifact loading and preprocessing. Check whether init() loaded the intended model path and whether the scoring script applies the expected feature order. A correct deployment name is evidence about routing, not proof about the file your code loaded.

Make that distinction concrete with a tiny, invented scoring contract. Suppose a linear risk score is 0.01 × amount + 0.8 × recent_failures. For an amount of 100 and two recent failures, the expected score is 2.6. If preprocessing swaps the features, the same coefficients produce 80.02. Either request can return HTTP 200, and a version label can still say 18. The transport succeeded while the scoring contract failed.

Use three separate observations: which deployment handled the request, which artifact it loaded, and whether the input-to-output assertion passed. A model fingerprint addresses the second observation only if it reflects the loaded artifact rather than a hardcoded release label. This toy arithmetic is a local reasoning aid, not an Azure model benchmark or a fraud decision policy.

How do traffic splitting and mirroring differ?

A traffic split chooses which deployment serves a request and supplies its result to the client. Mirroring, or shadow traffic, copies selected ordinary requests to another deployment for observation while excluding that deployment's result from the client response. Microsoft describes both behaviors for online endpoints. Official routing and mirroring concepts.

Compare a 90/10 split with Blue 100 percent plus Green 10 percent mirrored. In the split, some clients receive Green's results. In the shadow configuration, ordinary clients receive Blue's results while Green performs additional inference. The mirror percentage is not another share of client responses. The endpoint schema represents the two policies separately.

The current managed-endpoint rollout documentation limits mirroring to one deployment and at most 50 percent. A deployment receives live traffic or mirrored traffic, not both. Kubernetes online endpoints do not support this mirroring feature. A request explicitly targeting a deployment does not trigger shadow copying. Official mirror restrictions.

In an illustrative window with 1,000 eligible ordinary requests and exactly 100 selected shadow copies, Blue handles 1,000 calls and Green handles 100 additional calls. There are 1,100 inference executions but 1,000 client responses. The counts are stipulated for the exercise; actual managed routing does not promise exact percentages in a small sample.

Which result should each request path return?

Use this original request-path exercise to explain the semantics aloud. Assume the deployments are available and the request succeeds. Percentages describe policy; they are not deterministic assignments to named users.

Request and endpoint policy (%)	Client result	Green activity
Ordinary call; Blue 100, Green 0	Blue	No routed call
Target Green; Blue 100, Green 0	Green	Direct scoring
Ordinary call; Blue 100, mirror Green 10	Blue	Selected copies score silently
Ordinary call; Blue 90, Green 10; mirror disabled	Blue or Green	Some live calls
Target Green after rollback to Blue 100	Green	Override still selects Green

The last row is the useful trap. Changing the ordinary traffic map does not remove an explicit deployment override from a client's request. If that client still targets Green, an endpoint-level rollback alone does not change its selection.

Likewise, a Studio test that selects Green is not evidence of a working 90/10 route. Microsoft's tutorial notes that Studio requires a deployment selection for invocation. Validate ordinary routing with calls that omit the override, then inspect deployment-specific evidence.

How would you promote and roll back Green?

Our recommended answer structure is to separate validation, exposure, and recovery. First, test Green with fixed payloads and expected outputs. Then evaluate representative traffic through a mirror policy. Disable Green's mirrored traffic before assigning it live traffic, consistent with the documented restriction.

For a configured workspace, an illustrative exposure change is:

az ml online-endpoint update --name risk-endpoint \
  --mirror-traffic "green=0"

az ml online-endpoint update --name risk-endpoint \
  --traffic "blue=95 green=5"

Verify the resulting configuration after each completed operation. The commands illustrate separate updates; they do not make promotion atomic. Ensure Green has capacity for the exposure you choose and Blue can still absorb rollback traffic.

Suppose an original observation window contains 1,000 completed requests: 950 to Blue with no errors, and 50 to Green with five errors. Endpoint success is 99.5 percent, but Green's error rate is 10 percent. An aggregate chart can conceal a serious deployment problem.

A rollback decision should reference agreed service and model-quality criteria, not a percentage chosen from this article. Restore Blue's ordinary allocation, check remaining deployment overrides, verify new requests, and preserve Green's evidence for investigation. Retaining a ready Blue deployment during evaluation is our operational recommendation; it is not a guarantee of instantaneous recovery or completion of already-running requests.

What should you monitor before increasing traffic?

Microsoft exposes endpoint request metrics with deployment/status breakdowns and deployment resource metrics with instance breakdowns. Diagnostic tables include traffic, console, and event logs; diagnostic settings determine which logs are collected. Official monitoring documentation.

Compare error rate and latency by deployment over the same time window with comparable inputs. Classify failures and look for queue pressure, model-load problems, or scoring exceptions. Check input shape, payload size, runtime changes, and output schema before deciding that a slower Green needs more replicas.

Readiness indicates readiness to accept traffic; liveness checks container health. Neither is a model-quality evaluation. Probe settings. Use fixed semantic tests, segment-level checks, and delayed ground truth where available. These checks are our recommended evaluation strategy, not automatic guarantees supplied by the probes.

How do you separate capacity, access, and scoring failures?

Microsoft's troubleshooting guidance distinguishes provisioning, container startup, model initialization, and scoring failures. Inspect the failing stage and its logs before changing unrelated settings. A deployment that cannot initialize its model requires a different fix from a healthy model receiving malformed requests. Official troubleshooting guide.

Managed online deployments support Azure Monitor autoscale rules, including metric-based and scheduled policies. Official autoscale guidance. Our inference is that capacity planning should include model warm-up, request concurrency, burst shape, and per-deployment headroom. More instances do not repair a wrong traffic map or incompatible preprocessing.

Also separate the caller's right to invoke the endpoint from the serving deployment's access to required resources. Microsoft documents endpoint authentication as an invocation concern. Client authentication. When an access failure occurs, identify the operation, identity, resource, and observed error before proposing broader permissions.

When would a batch endpoint be the better answer?

If the requirement is scoring a large stored dataset with results available later, discuss batch endpoints. Microsoft's batch endpoint model starts asynchronous jobs and supports a default deployment; it does not use online percentage routing as its deployment-selection mechanism. Official batch endpoint concepts.

Ask whether the caller needs an immediate prediction or can wait for a job. Then discuss input/output locations, completion tracking, and failure handling. Moving an interactive application to batch solely to avoid diagnosing online latency changes its response contract; establish that trade-off before choosing the service.

Practice Azure Machine Learning interview questions

These five verified records exercise related skills. Their company labels and reported prompts are practice evidence, not predictions of your interview or proof that those companies use Azure Machine Learning for the question.

Question	Azure ML follow-up to rehearse
Design an Embedding and Classification API	Bind preprocessing and model versions to a deployment.
Design a GPU inference API	Explain version routing and a reversible rollout.
Design GPU inference request batching	Separate queue delay from model execution and capacity.
Evaluate an ML feature launch	Distinguish technical health from launch readiness.
Build ETA prediction and simulate impact	Choose model-quality and segment guardrails.

For your next rehearsal, sketch one endpoint, two deployments, the ordinary traffic map, and a targeted request. Explain which model answers each call, what evidence could disprove your diagnosis, and how you would reverse the release.

Sources and Further Reading

Research checked October 4, 2026. Examples target managed online endpoints and current Azure CLI ML v2 documentation. Local exercise validation does not replace a deployment test in your Azure environment.