LLM Tool-Calling Interview Questions: Parallel Calls, State, and Error Recovery

An LLM requests two reads and a ticket update. The reads finish in reverse order; the update times out after the server may have committed it. For LLM tool-calling interview questions, a convincing answer must explain both what can run together and what the application knows after each event.

The core rule: parallelize independent work, associate results with call IDs, and track side effects in durable application state. A timeout after dispatch means the outcome may be unknown. It is not permission to repeat the operation with a new identity.

Follow one execution trace from parallel reads through an uncertain write.

Evidence boundary: provider documentation supplies the protocol facts below. The ticket example, state machine, and preparation advice are our design inferences. We found no verified firsthand candidate report establishing that these exact exercises appeared in an employer's interview. Local tests use simulated tools, not live model or ticketing APIs.

What does the model actually execute?

For application-owned function tools, the model proposes a named operation and arguments. Your application validates the request, executes permitted work, and returns an observation. Keep that distinction separate from provider-hosted tools, whose execution is managed by the provider.

Official fact: OpenAI's function-calling flow includes an application execution step followed by tool output. A response can contain multiple calls; setting parallel_tool_calls to false limits the model to zero or one. That option controls emitted calls, not your database transaction boundaries. OpenAI function calling

In our design, three responsibilities remain explicit: the model chooses a proposed action, the scheduler decides when eligible work runs, and the business service decides whether a write is valid. A model-generated sentence saying “completed” cannot replace a service receipt.

Which calls can run in parallel?

Ask about data dependencies, shared resources, and effects. Two reads are candidates for concurrency when neither needs the other's result and their combined load fits the service budget. Reading a ticket and a policy version can run together if the policy identifier is already known.

If the ticket determines which policy to load, the second read must wait. Likewise, two writes touching the same ticket need an explicit conflict policy. Merely using different tool names does not make operations independent.

Official fact: Claude's parallel-tool documentation says the application may execute a batch concurrently or sequentially. It specifically calls out side effects, shared state, and ordering requirements as reasons to prefer sequential execution. Claude parallel tool use

Our design inference: independent reads are not necessarily mutually consistent reads. Two services queried concurrently do not automatically provide a shared snapshot. Preserve version identifiers and revalidate the write against authoritative state. Otherwise, a fast read phase can still produce a stale decision.

Walk through a batch that fails after the write

Assume a user has authorized resolving ticket T42 if its verification checks pass. Our fictional ticket service supports a version precondition, an idempotency key, and authoritative operation-status lookup. These are exercise assumptions, not features promised by every ticketing API.

Event	Observation	Application decision
Model proposes reads A and B	A reads T42; B reads known policy P7	Validate scope, then dispatch both
B finishes first	Policy P7 version 3 permits resolution	Store result under B; do not attach it to A
A finishes second	Ticket is open at version 8; checks pass	Evaluate the proposed transition
Model proposes write C	Resolve T42 using version 8	Authorize; persist operation K; dispatch
C times out	No receipt reached the caller	Mark K's outcome UNKNOWN
Status lookup completes	K committed; receipt R91; ticket version 9	Save receipt and report confirmed success

The key failure is between server commit and receipt delivery. Replaying the whole plan from the beginning could issue the same business action again. Reusing the successful read results without checking their freshness could cause a different error: acting on a ticket someone has since changed.

If the backend rejects version 8 because the current version is 9, refresh the ticket and reconsider the intended action. Do not mechanically replace the expected version with 9 and retry. That would remove the protection without checking why the state changed.

How do you match out-of-order results?

Use identities, not completion positions. Two calls may invoke the same function with different arguments, so function names alone are insufficient.

Official fact: OpenAI Responses pairs function_call_output with its originating call_id. Claude uses tool_result blocks linked by tool_use_id, collecting the batch's results in the next user message before text. Claude also documents error results for calls intentionally not executed. Keep provider adapters separate. OpenAI result handling, Claude result formatting

This original Python helper validates a completed batch of internal result envelopes. It is not a provider SDK implementation:

def reconcile(expected_ids, completed):
    if len(set(expected_ids)) != len(expected_ids):
        raise ValueError("duplicate expected call ID")
    expected = set(expected_ids)
    by_id = {}
    for item in completed:
        cid = item["call_id"]
        if cid not in expected or cid in by_id:
            raise ValueError("unknown or duplicate result")
        by_id[cid] = item
    if set(by_id) != expected:
        raise ValueError("missing result")
    return [by_id[cid] for cid in expected_ids]

For expected IDs ["A", "B"], results arriving as B then A are returned in A/B order without swapping their payloads. A missing result is an incomplete batch, not an empty success. Before this helper, validate envelope shape and bind records to the correct run and turn; IDs are not authorization credentials.

What state must survive a process restart?

Separate the model conversation, execution ledger, and business state. The conversation explains why an action was proposed. The ledger records what the application dispatched and observed. The ticket service owns whether the transition committed.

For operation K, our ledger stores tenant and actor scope, run and turn, provider call ID, tool/schema version, validated arguments, expected ticket version, operation key, attempt count, deadline, status, and any confirmed receipt. A single “last agent message” field cannot represent that contract.

Official fact: OpenAI supports conversation continuation through mechanisms including previous_response_id; when manually managing stateless history, its documentation instructs preserving the response output items, including reasoning items. This is conversation continuity, not an external-write transaction log. OpenAI conversation state

Official fact: LangGraph distinguishes checkpointed thread state from longer-term stores, and warns that in-memory checkpoint storage does not survive process restarts. Choose persistence appropriate to the failure being discussed. LangGraph persistence

Persist the proposed operation before dispatch, then persist the result before allowing dependent work. Still explain the gap: a crash after remote commit but before local result persistence remains ambiguous. Checkpointing alone cannot atomically commit a transaction in another service.

How should the runtime recover an unknown outcome?

Use an explicit state machine. In this exercise, an authorized operation moves from READY to RUNNING. A confirmed receipt produces SUCCEEDED; a definitive rejection produces FAILED. A transport timeout after dispatch produces UNKNOWN, which requires reconciliation.

A provider call ID identifies a protocol request. An application operation key identifies an authorized business action across attempts. A later model turn can produce a new call ID for the same action; do not let that automatically create a second operation. Conversely, two legitimately different actions must not be collapsed simply because their arguments happen to match.

Official guidance: AWS describes caller-provided request identifiers for idempotent APIs and discusses late requests and reuse of an identifier with different intent. The service's contract must define duplicate handling rather than leaving the caller to guess from a timeout. AWS guidance on safe retries

Our design binds K to tenant, action identity, and validated payload. The backend must durably associate that identity with its effect and reject conflicting payload reuse. A client-side dictionary cannot provide this guarantee across crashes or competing workers.

After a timeout, query operation K. If it committed, return its stored receipt. If the service authoritatively confirms it never started, retry under the same key, subject to the original authorization and retry budget. If status remains unknown, pause work that depends on that outcome and report uncertainty.

A temporary “not found” from an eventually consistent lookup is not necessarily that authoritative confirmation. Ask about status visibility, deduplication retention, and late arrivals. If the service cannot resolve uncertainty or safely deduplicate, escalate instead of claiming exactly-once execution.

What should happen when only one parallel call fails?

Preserve successful observations and represent failed calls explicitly. If policy read B times out, do not substitute an empty policy and continue to resolve the ticket. Return a bounded error describing what failed and whether a safe retry remains available. Reuse read A only if its freshness requirements still hold.

Official fact: Python's asyncio.gather preserves input ordering in its result list, while return_exceptions=True collects exceptions as results. With the default setting, an exception can propagate while other submitted tasks continue. These behaviors matter when designing partial-result handling. Python concurrency documentation

Our inference is to choose failure semantics before choosing a concurrency primitive. Cancellation of a local coroutine is not proof that a remote request was rolled back. A late receipt must still be associated with its original operation and reconciled without overwriting a newer attempt's state blindly.

Bound concurrency, per-call timeouts, total run time, model turns, and retry attempts separately. Retry transient reads with bounded backoff; return validation errors for argument repair; stop on authorization failures. Avoid nested unlimited retry loops in the SDK, tool wrapper, and planner.

What checks belong before dispatch?

Treat the proposed call as an untrusted request. Validate the tool allowlist, complete arguments, types, resource ownership, current permission, and any required approval. A schema-valid ticket ID can still identify another tenant's ticket. Derive actor identity from the authenticated session rather than a model-supplied field.

For streamed arguments, buffer until the relevant call is complete and validated before starting a side effect. A prefix that looks like valid JSON is not proof that the model has finished specifying the request. Tool-returned text should remain task data; an instruction embedded in a ticket does not expand the user's authorization.

For the running example, the gate must also check that both prerequisite observations are usable, the ticket still satisfies the transition, and K represents the intended action. These are application design requirements, not guarantees supplied by function schemas.

Which tests make this interview answer credible?

Our local simulator passes 23 checks covering reversed completion order, duplicate and missing results, a foreign-run envelope, a failed policy read, stale versions, a timeout after commit, replay with the same key, conflicting key reuse, and an unresolved outcome that must not trigger another write. It also models restarting the worker while retaining the backend's operation record.

The critical assertion is observable: the committed-timeout scenario produces one ticket transition, and recovery returns the original receipt. Deliberately retrying with a new key demonstrates why a non-idempotent side-effect service can duplicate work. These are fault-injection exercises under stated contracts, not a production durability certification.

For each, add one failure between dispatch and acknowledgment, then explain the recovery evidence.

Question	Practice focus
Design an Enterprise Tool-Using Agent	Separate action proposals, execution policy, and business state.
Write pseudocode for a ReAct-style loop	Preserve tool observations and define stop conditions.
Implement idempotent request handling with idempotency keys	Explain operation identity across replayed attempts.
Debug a Concurrent Job Scheduler	Reason about dependencies, partial completion, and cancellation.
Evaluate an Agentic Decision System End to End	Test terminal outcomes and duplicate effects, not just fluent answers.

For the broader architecture, see our Agentic AI interview guide. When explaining your design, return to the concrete trace: what was proposed, what ran, what committed, and what remains unknown. Choose one ML system design exercise and defend every transition using recorded evidence.

Sources and Further Reading

• OpenAI function calling and result correlation
• OpenAI conversation state
• Claude parallel tool-use execution and formatting
• AWS Builders' Library: making retries safe with idempotent APIs
• LangGraph persistence and checkpoint storage
• Python asyncio tasks and concurrent execution

Checked September 21, 2026. Provider protocols can change. The local exercises simulate execution and recovery; they do not measure model quality or validate a live integration.