cswork / libraryOpen practice workspace

Interview guides · Croncrew Technologies

Croncrew Technologies Software Engineer Interview Guide

Croncrew Technologies Software Engineer interview guide: two reported rounds, Spark, Hadoop and .NET questions, streaming design and a 7-day prep plan.

The 60-second brief

Reports about Croncrew Technologies describe engineering work on infrastructure for platforms that handle large volumes of data. The Software Engineer role appears under three named stack variants: Spark developer, Hadoop developer and .NET developer. Candidates describe the day-to-day as owning features or data pipelines from build to production, working with product managers on requirements, reviewing teammates' code and debugging live issues. Reports do not name a specific product, customer or team, so this guide stays with what they do cover: the stack, the stages and the question types.

This guide is for candidates interviewing as a Software Engineer at Croncrew Technologies, whether your primary stack is Apache Spark, Apache Hadoop or .NET with C#. It covers the two reported stages, the eight reported questions (framework internals, memory management, distributed debugging, streaming design, scalability trade-offs and one bottleneck story) and a seven-day plan. Cloud-native tooling such as Docker, Kubernetes and CI/CD pipelines is reported as a nice-to-have, so it appears only briefly.

Candidates report two stages over roughly 2-4 weeks: an Initial Screening on your technical background, then Deep-Dive Technical Rounds covering coding, framework knowledge and system architecture.

Focus areas

  • Apache Spark
  • Apache Hadoop
  • .NET (C#)

Overview

The reported role is organised by stack. A Spark developer is expected to know distributed data processing in Spark, a Hadoop developer to know MapReduce and the surrounding ecosystem, and a .NET developer to know C# and the runtime. The reported questions follow that split: the difference between RDDs and DataFrames, tuning a MapReduce job, synchronous versus asynchronous programming in .NET, and memory management in large-scale data applications. Pick the stack on your CV as your primary one and prepare it to the level of internals, because candidates report that the deep-dive stage goes beyond syntax into performance tuning, concurrency and parallelism, and, less commonly, garbage collection tuning and custom connector development.

The second group of reported questions is about systems rather than a single framework: how to approach debugging a distributed system failure, how to design a system that processes real-time streaming data with low latency, and how to balance scalability against development speed. The architecture topics candidates list are data integrity in distributed environments, scalability techniques, how software talks to databases, caches and external APIs, and, as less common extras, disaster recovery and circuit breakers between services. Prepare these as designs you can draw and defend: components, failure modes, and what you gave up.

Communication is part of what candidates describe. Reports advise explaining the reasoning behind a design choice rather than only the mechanics, thinking aloud while coding, asking clarifying questions before solving, and admitting a gap in a tool instead of guessing. The one behavioral question reported asks for a significant technical bottleneck and how you resolved it, so have a measured, first-person story ready, and review your own past projects for the decisions and trade-offs you can discuss in detail.

The Croncrew Technologies interview process

Candidates describe two stages over roughly 2-4 weeks. The reports name the stages and what they cover, not a fixed format, so confirm details with your recruiter.

Initial Screening

Candidates report this opening stage as a check of your technical background against what the team currently needs. Reports do not say who runs it or what format it takes, so treat it as a conversation where you must describe your stack experience concretely. Name the variant you are strongest in (Spark, Hadoop or .NET), tie it to projects you owned, and be ready to state one design decision from each and the trade-off it carried. Ask which stack variant the later technical rounds will target so you can aim your preparation.

What to demonstrate

  • How your technical background lines up with the stack variants the role lists
  • How clearly you describe your own projects and decisions
  • Whether your experience matches what the team needs at the moment

How to prepare

  • Write a short summary of two projects, each with one decision, the alternative you rejected and the measured result
  • Prepare one-sentence answers on your primary stack: RDDs versus DataFrames, ways to tune a MapReduce job, or synchronous versus asynchronous code in .NET
  • List the agile, peer review and production-debugging experience you can point to, since the role description includes all three
  • Ask the recruiter which stack variant the technical rounds will use and whether coding is on a shared editor or a take-home

Deep-Dive Technical Rounds

Candidates report that the technical rounds cover coding, framework-specific knowledge and system architecture. In practice that means a coding task in your language, questions on how your framework behaves under load (performance tuning, concurrency, memory), and a design discussion about how a service works with databases, caches and external APIs. Reports advise explaining why you chose a design rather than only how it works, so state the trade-off for each decision. The reported framework and design questions, such as the MapReduce tuning question, the memory management question and the real-time streaming design question, are the right practice material for this stage.

What to demonstrate

  • Working code in your primary language, with your reasoning spoken as you go
  • Depth in framework internals: performance tuning, concurrency or parallelism, and memory management
  • System design covering data integrity, scalability and interaction with databases, caches and external APIs
  • How well you reason about trade-offs and respond to new information

How to prepare

  • Rehearse the RDD versus DataFrame comparison and the MapReduce tuning checklist out loud until you can give a reason for each item
  • For .NET, be able to explain what await does to the calling thread and how garbage collection treats short-lived and long-lived objects
  • Sketch the low-latency streaming design on paper: ingestion, partitioning, state, delivery guarantees, backpressure and failure handling
  • Prepare a debugging sequence for a distributed failure and a circuit breaker explanation, as circuit breakers and disaster recovery are listed as less common topics

Where candidates lose points

Answering the RDD versus DataFrame question with definitions only

Add the consequences. Say which one you would pick for which workload, what schema information lets the engine optimise a query, where typed or low-level control still matters, and what each costs in serialisation and memory. End with a case from your own work.

Listing MapReduce tuning tricks without first finding the bottleneck

Start from evidence: job counters, task times and shuffle volume. Then match the fix to the finding, such as a combiner or compression for heavy shuffle, a custom partitioner for skew, and fewer, larger input splits for many small files. A checklist given before diagnosis sounds memorised.

Saying async code in .NET is simply faster

Explain that await releases the calling thread while I/O is pending, which improves throughput under concurrent load, and does not speed up CPU-bound work. Mention the blocking-on-a-task deadlock risk and where Task.Run does and does not help.

Naming streaming tools in the low-latency design but never stating guarantees

Say what latency target you are designing for, how you partition, where state lives, whether delivery is at-least-once or exactly-once and how duplicates are handled, how backpressure works and what happens on replay. Cover data integrity and disaster recovery because the reported architecture topics include them.

Telling the bottleneck story without a measurement or a personal decision

Write the story as symptom, measurement, hypothesis, fix, trade-off, verified result. Keep one real number for each stage and one sentence on what you personally decided. If a teammate made a key call, say so and describe your part.

Questions & practice

What are the key differences between synchronous and asynchronous programming in.NET?

Medium · Technical and Domain Expertise

Approach

  1. A synchronous I/O call blocks its thread until the operation completes. With async/await, a method returns a Task and releases the thread at the await while the I/O is pending, and a continuation resumes it later. A single call gets no faster, but a server can handle many more concurrent requests on the same thread pool.
  2. The compiler turns an async method into a state machine. By default, await captures the current SynchronizationContext (UI apps, classic ASP.NET) and resumes on it. ASP.NET Core has none, and library code uses ConfigureAwait(false) so it never needs to return to a caller's context.
  3. Async is not parallelism. For I/O-bound work I use async APIs and run independent calls together with Task.WhenAll. CPU-bound work goes to Task.Run or Parallel.For/PLINQ, and wrapping I/O in Task.Run only burns a pool thread.
  4. The pitfalls I would name are these. Calling .Result or .Wait() on a Task (sync-over-async) can deadlock under a SynchronizationContext and starve the thread pool. async void is only for event handlers, because the caller cannot catch its exceptions. A Task stores its exceptions and rethrows them at await, and WhenAll surfaces the first one while the Task keeps all of them in an AggregateException.
  5. For hot paths I would consider ValueTask, which avoids allocating when the result is often ready synchronously, but it must be awaited only once. I also pass a CancellationToken down the whole call chain.

Follow-up

  • Why does .Result deadlock in a WinForms or classic ASP.NET app but usually not in an ASP.NET Core app?
  • What is the difference between awaiting three calls in a loop and awaiting Task.WhenAll on them, including how exceptions surface?
  • When would you return ValueTask instead of Task, and what rule does the caller have to follow?

Describe how you handle memory management in large-scale data applications.

Medium · Technical and Domain Expertise

Approach

  1. Rule one: never materialize the full dataset in one process. I stream with iterators or bounded batches (mapPartitions in Spark, a streaming reader or IAsyncEnumerable in .NET), and I avoid collect() or toPandas() on large results, which pull everything into the driver.
  2. In Spark I explain the executor layout. Unified memory is spark.memory.fraction (default 0.6) of the heap minus 300 MB reserved, shared by execution and storage, and cached blocks can be evicted for execution. Off-heap and Python memory come from spark.executor.memoryOverhead. When YARN kills a container for exceeding its limit, the fix is usually overhead, not heap.
  3. I control what stays resident. I cache only data that is reused, prefer MEMORY_AND_DISK_SER with Kryo for large caches, and unpersist when finished. I size partitions so each fits in a task's memory, and I treat one OOM task as a sign of skew: I salt the key or enable AQE skew-join handling.
  4. In .NET I describe the generational GC (gen 0/1/2). Objects of 85,000 bytes or more go on the Large Object Heap, which is not compacted by default and fragments. I rent buffers from ArrayPool<T>, slice with Span<T>/Memory<T>, avoid boxing in hot loops, and dispose unmanaged resources with using.
  5. I prove it with numbers: GC time and spill in the Spark UI, GC logs or heap dumps on the JVM, and dotnet-counters or dotnet-gcdump in .NET. I state the before and after figures of each change.

Follow-up

  • An executor dies with a container memory-limit error. How do you tell whether to raise the heap or the overhead?
  • When is caching a DataFrame a mistake rather than an optimization?
  • Why do large temporary arrays hurt a long-running .NET service, and how does ArrayPool change that?

Track a rolling failure rate per destination for circuit decisions

Easy · sliding window · ring buffer · circuit breaker

The egress service delivers about 1,500 webhooks per second across roughly 40,000 destinations, each call bounded by a 10 second timeout. Maintain, per destination, the failure rate over the trailing 60 seconds so a caller can ask before dispatch whether the circuit should open. Attempts arrive as (destination_id, finished_at_ms, outcome). Requirement: amortised O(1) per attempt, with total memory bounded by the destination count rather than by traffic. Give the structure, its exact memory, and the rule that stops a destination with three attempts from opening a circuit.

Approach

  1. Name the exact-deque version and then reject it as the default. Holding timestamps and advancing a tail pointer past anything older than now minus 60 seconds is a correct two-pointer window at amortised O(1) per attempt, but its memory tracks in-window traffic, so one destination in a retry storm holds hundreds of thousands of entries while thousands of quiet destinations hold none.
  2. Use a ring of 60 one-second buckets per destination, each bucket a pair of counters for attempts and failures. On an attempt, advance the ring by the elapsed whole seconds, zeroing at most min(elapsed, 60) buckets, then increment the head. That is amortised O(1) with a fixed footprint per destination.
  3. State the footprint: 60 buckets times two 4-byte counters is 480 bytes of payload per destination, so 40,000 destinations is roughly 20 to 25 MB with per-entry overhead, bounded by the catalogue rather than by the rate. The cost is granularity, since the oldest bucket ages out in whole seconds, which is far tighter than the decision needs.
  4. Require a minimum sample before the circuit may open. A destination with three attempts and three failures reads as 100 percent and is not evidence; a floor of roughly 20 attempts in the window makes the ratio meaningful, and below that floor use a run of consecutive failures as the trigger instead.

Follow-up

  • The fleet is 30 instances and each sees roughly a thirtieth of a destination's traffic. Where does the rate actually live, and what does a per-instance answer get wrong?
  • A destination answers in 9.5 seconds and succeeds. It is not failing but it is consuming your per-destination concurrency. What signal should open the circuit here?

Stop tag and share joins from fanning out a page

Medium · join fan-out · semi-joins · many-to-many

resource_tag is (resource_id, tag_id) with PK (resource_id, tag_id); resource_share is (resource_id, shared_with_user_id, permission). The tagged-and-shared listing inner-joins resource to both, filters tenant_id, tag_id = ANY($2) and shared_with_user_id = $3, orders by updated_at DESC and takes 50. Pages come back with fewer than 50 distinct resources and the total in the header is far too high. Explain the row multiplication, rewrite both the page query and the count query so each is correct, and name the index each one needs. PostgreSQL 16.

Approach

  1. Do the arithmetic against the predicates that are actually there. An inner join emits one row per matching child row, and both joins are filtered: tag_id = ANY($2) admits only the requested tags, shared_with_user_id = $3 admits one user's share rows. So a resource holding three of the requested tags and shared with $3 once yields three rows, not one — the multiplier is its count of matching tags times its share rows for that single user, and that second factor is 1 unless the table admits duplicate (resource_id, shared_with_user_id) pairs. LIMIT 50 then limits rows rather than resources, and COUNT(*) counts pairs — the header is the product, not the population.
  2. Reject DISTINCT as the fix. It deduplicates after the product has been built, so the planner must materialise and sort the fanned-out set before the LIMIT can apply, and it leaves any SUM or AVG in the same select list wrong.
  3. Rewrite both filters as semi-joins, keeping resource as the only row source: AND EXISTS (SELECT 1 FROM resource_tag rt WHERE rt.resource_id = r.resource_id AND rt.tag_id = ANY($2)) and the same shape against resource_share. A semi-join stops at the first match per resource and preserves the driving index order, so ORDER BY updated_at DESC, resource_id DESC LIMIT 50 still stops after 50 rows.
  4. Count with the same predicates and no join at all: SELECT count(*) FROM resource r WHERE r.tenant_id = $1 AND r.status = 'active' AND EXISTS (...) AND EXISTS (...). Nothing multiplies a resource, so the number is the population.

Follow-up

  • The filter changes from 'any of these tags' to 'all of these tags'. Rewrite it and state what it costs relative to the ANY form.
  • A resource can be shared with the same user twice under different permissions. Does your count change, and should it?

Keep soft-deleted accounts from blocking re-registration

Easy · soft delete · unique constraints · engine differences

app_user holds user_id, tenant_id, email CITEXT, password_hash (NULL for SSO principals), email_verified_at, auth_version, status ('invited','active','suspended','deactivated'), created_at, updated_at, deleted_at. Two live accounts for one address inside a tenant must be impossible, but an address freed by a soft delete must be reusable, and the same tenant may delete and re-register it repeatedly. Write the uniqueness DDL for PostgreSQL 16, then the equivalent for MySQL 8 where partial indexes do not exist, and say what each permits once three deleted rows already hold that address.

Approach

  1. Start from what is actually unique: not (tenant_id, email), but (tenant_id, email) among live rows. PostgreSQL says that directly — CREATE UNIQUE INDEX app_user_live_email ON app_user (tenant_id, email) WHERE deleted_at IS NULL. A full constraint over the same two columns burns the address permanently the first time someone deletes an account.
  2. Keep case-insensitivity in the type or the index, never in the application: CITEXT as given, or UNIQUE (tenant_id, lower(email)) as an expression index where the extension is unavailable. A case-sensitive unique column is exactly how two accounts for one human appear.
  3. For MySQL 8 the predicate has to move inside the key: add a discriminator column that is a constant 0 while the row is live and is set to user_id on delete, with UNIQUE (tenant_id, email, deleted_marker). Live rows share the constant and still collide; deleted rows differ from each other and stop colliding.
  4. State the NULL variant and its dependency: leaving the marker NULL for deleted rows also works, because a unique index treats NULLs as distinct — true in MySQL, and true in PostgreSQL only under the default NULLS DISTINCT, which PostgreSQL 15 lets you reverse. Check the polarity against the three existing deleted rows: constant-on-live is what preserves the collision you want, and reversing it silently admits duplicate live accounts.

Follow-up

  • A deleted account re-registers with the same address the next day. Do the old resource rows follow the new user_id, and how does the API keep the two principals apart?
  • How do you honour an erasure request while resource_revision.actor_user_id still references this table?

Explain the difference between RDDs and DataFrames in Apache Spark.

Medium · Technical and Domain Expertise

Approach

  1. An RDD is an immutable, partitioned collection of objects that I transform with functions (map, filter, reduceByKey). Spark only sees opaque lambdas, so it cannot reorder or prune anything inside them. A DataFrame carries a schema and builds a logical plan that the Catalyst optimizer rewrites, using predicate pushdown, column pruning and constant folding, before it runs.
  2. Execution differs too. DataFrames run on Tungsten: a compact binary row format, off-heap storage when enabled, and whole-stage code generation. RDDs hold ordinary JVM objects that need Java or Kryo serialization, which means more memory and more garbage-collection pressure.
  3. Both are lazy, partitioned and fault-tolerant through lineage, so the difference is optimization and typing, not reliability. In Scala and Java, a Dataset gives compile-time types plus Catalyst. A DataFrame is a Dataset[Row], so a misspelled column only fails when the plan is analyzed.
  4. In PySpark the gap is larger. RDD lambdas and plain Python UDFs ship rows to Python worker processes, while built-in DataFrame functions stay in the JVM. If a UDF is unavoidable, I reach for a vectorized pandas_udf (Arrow) first.
  5. I still pick RDDs for unstructured records, a custom Partitioner, or fine-grained mapPartitions logic. I convert back with createDataFrame and an explicit schema as early as possible, because df.rdd drops out of the optimizer.

Follow-up

  • Walk through what Catalyst does to a filter written after a join on two DataFrames.
  • Why is groupByKey on an RDD more expensive than reduceByKey, and what does groupBy().agg() on a DataFrame do instead?
  • A PySpark job with a Python UDF is slow: how do you confirm the UDF is the cost, and what do you replace it with?

How do you optimize a Hadoop MapReduce job for performance?

Medium · Technical and Domain Expertise

Approach

  1. I measure before tuning. In the JobHistory UI I compare job counters (map output bytes, spilled records, shuffle bytes, GC time) with per-task durations, to see whether the cost is in input, spill, shuffle or one slow reducer.
  2. Next I cut the shuffle. I filter and project in the mapper, and I add a combiner only when the reduce function is commutative and associative (sum, count, max). A combiner may run zero, one or many times, so an average has to emit sum and count, never a partial mean. I also compress map output with Snappy or LZ4 (mapreduce.map.output.compress).
  3. I fix the input layout. Many small files mean one mapper each, so I use CombineFileInputFormat or compact them into SequenceFiles, ORC or Parquet. Gzip files cannot be split; bzip2 and block-compressed container formats can.
  4. I tune spill and parallelism. mapreduce.task.io.sort.mb and the spill threshold reduce repeated disk spills. I set the reducer count to the data volume, and for a skewed key I write a custom Partitioner or salt the hot key and merge in a second pass.
  5. Memory and stragglers: I size mapreduce.map/reduce.memory.mb with the JVM heap (java.opts) safely below the container limit, reuse Writable objects in the mapper to cut allocation, and enable speculative execution only when tasks have no external side effects.

Follow-up

  • Why can't a combiner compute an average directly, and how do you restructure the job so it can help?
  • One reducer takes ten times longer than the rest. How do you confirm skew and fix it?
  • At what point would you rewrite this job in Spark, and what would you gain or lose?

How would you design a system to process real-time streaming data with low latency?

Medium · Problem-Solving and Logic

Approach

  1. I pin the requirements: event rate and peak, a p99 end-to-end latency target, whether per-key ordering matters, the delivery guarantee (at-least-once or exactly-once), how late events can arrive, and how much state the computation keeps.
  2. My pipeline would be producers, then a durable partitioned log such as Kafka keyed by entity for per-key ordering, then a stream processor, then a low-latency serving store, with a copy to cheap storage for replay and backfill. Spark Structured Streaming micro-batches trade some latency for throughput, while an event-at-a-time engine such as Flink gets lower latency. I say which my latency target needs.
  3. Latency levers: avoid shuffles in the hot path and keep state local (for example a RocksDB state store). I keep producer batching (linger.ms) and trigger intervals small enough for the target, knowing they cost throughput. Backpressure handles bursts so nothing is dropped silently.
  4. For correctness, I window on event time and use watermarks to bound lateness and state size. Exactly-once results need checkpointed offsets plus an idempotent or transactional sink. Otherwise at-least-once delivery with a dedupe key is the practical choice.
  5. Scaling and failure: partitions cap consumer parallelism within a group, so I size them for future load. A hot key gets salted and re-aggregated. Recovery replays from the last checkpoint, and consumer lag is the main health metric I alert on.

Follow-up

  • Events can arrive ten minutes late for a five-minute windowed count. What do you configure, and what does it cost in state and latency?
  • How do you get exactly-once results into a relational database that is not transactional with Kafka?
  • One key suddenly carries half the traffic. What breaks first, and how do you fix it?

How do you balance the trade-offs between system scalability and development speed?

Medium · Problem-Solving and Logic

Approach

  1. I frame it as a decision with numbers: current load, a realistic growth horizon, and the cost of being wrong in each direction. Building for scale too early slows delivery and adds operational load. Building too little forces a rewrite under pressure.
  2. I separate decisions that are hard to reverse from easy ones. Data model, partition or shard key, public API contracts and data formats are costly to change later, so I spend design time there. Internal implementation, caching and service boundaries can wait until a measured bottleneck appears.
  3. I start with the simplest design that handles a stated multiple of today's load. I keep the cheap seams that make later scaling horizontal rather than a rewrite: stateless services, idempotent operations, clear interfaces, and enough observability to see the next bottleneck coming.
  4. I make the debt explicit. I write down what was deferred, the metric threshold that triggers revisiting it (latency, queue lag, job runtime against its window), and a load test that shows the current headroom.
  5. I close with one concrete example of my own, such as a single-node batch job I later moved to Spark, or a cache I postponed. I explain what the deferral saved and what the migration cost.

Follow-up

  • Tell me about a time you got this wrong in either direction. What did it cost and what would you change?
  • What signal tells you it is time to pay down the scalability shortcut?
  • Which design choice would you refuse to defer even under deadline pressure, and why?

How do you approach debugging a distributed system failure?

Medium · Technical and Domain Expertise

Approach

  1. I scope it first: which services, hosts or partitions are affected, what share of requests or jobs fail, and exactly when it started. Then I line that start time up against recent deploys, config changes, traffic shifts and dependency incidents.
  2. I mitigate before I diagnose. If a rollback, traffic shift or feature-flag flip stops the impact, I do that, and I keep logs, heap dumps and the failing input for analysis.
  3. I follow one failing request or task across nodes, using a correlation or trace ID, centralized logs and distributed tracing. When I order events by timestamp, I allow for clock skew between machines.
  4. I test hypotheses specific to distributed systems: timeouts with retry storms, a slow dependency cascading through exhausted connection pools, partial network partitions, duplicate processing under at-least-once delivery, and in data jobs a skewed partition, or executor loss from memory limits that shows up as repeated stage retries.
  5. I confirm the cause by reproducing it or by a metric that moves when the fix ships. Then I add the guard that would have contained it (timeouts, backoff with jitter, a circuit breaker, an alert on the leading signal) and write a blameless postmortem.

Follow-up

  • A Spark stage keeps failing with lost executors. Where do you look first, and what does each finding rule out?
  • How do you tell whether retries caused the outage or only hid it?
  • Node clocks disagree by a few seconds. How do you still establish the order of events?

Suggested practice checklist

Seven days built on the two reported stages: day one fixes your stack and maps the loop, days two to four cover the Spark, Hadoop and .NET questions, and days five to seven cover debugging, streaming design, trade-offs and a full mock.

Day 1 — Choose your stack and map the loop

  • Write the two reported stages and next to each list the topics it covers: background and fit for the screening, then coding, framework knowledge and architecture for the deep-dive rounds
  • Pick Spark, Hadoop or .NET as your primary stack and list the internals you can explain without notes against those you cannot
  • Write a short summary of two projects, each with a decision, a rejected alternative and a measured result

Deliverable: A one-page loop map, a ranked list of weak internals in your primary stack, and two project summaries.

Day 2 — Spark: RDDs, DataFrames and tuning

  • Write a comparison of RDDs and DataFrames covering schema, optimisation, serialisation and when to drop to the low-level API
  • Add notes on shuffles, partitioning and skew: what triggers a shuffle, how you would spot one task running much longer than the rest, and two fixes
  • Answer the RDD versus DataFrame question aloud and record it, then fix the weakest sentence

Deliverable: A written RDD versus DataFrame comparison, a skew and shuffle note, and one recorded spoken answer.

Day 3 — Hadoop: tuning a MapReduce job

  • Write the stages of a MapReduce job (map, spill, shuffle and sort, reduce) and mark where each tuning lever acts
  • List levers with the symptom each fixes: combiner, compression of map output, custom partitioner for skew, input split size and the small-files problem, number of reducers
  • Answer the MapReduce optimisation question aloud, starting from how you would find the bottleneck

Deliverable: A stage diagram with levers and symptoms, plus a spoken answer that opens with diagnosis.

Day 4 — .NET async and memory in data-heavy applications

  • Write what await does to the calling thread, why blocking on a task can deadlock, and which work benefits from async (I/O-bound) and which does not (CPU-bound)
  • Write a memory note covering garbage collection generations, the large object heap, streaming data instead of loading it whole, and pooling buffers
  • Answer the memory management question aloud using one large-scale data example from your own work, whichever stack it was in

Deliverable: An async note, a memory management note and one spoken answer built on a real example.

Day 5 — Distributed debugging and streaming design

  • Write a debugging sequence for a distributed failure: establish what changed, check logs and metrics for the failing component, separate trigger from cause, bisect, confirm the fix
  • Sketch the real-time streaming design: ingestion, partitioning, state, delivery guarantees, backpressure and failure handling, then answer it aloud

Deliverable: A debugging checklist, a labelled streaming design diagram and a working solution to the rolling failure rate problem.

Day 6 — Trade-offs and behavioral stories

  • Write a scalability versus development speed answer: when you ship the simple version first, which signals trigger the rework, and what you instrument so you notice
  • Write the bottleneck story as symptom, measurement, fix, trade-off and verified result, then say it aloud

Deliverable: A written trade-off answer, a bottleneck story with real numbers and two practised behavioral answers.

Day 7 — Full mock and data integrity warm-up

  • Run a full mock in order: a short background walk-through, one framework question from your stack, the streaming design, then the bottleneck story, thinking aloud throughout
  • Write two questions for your interviewers, for example which stack variant the team uses and how code review works

Deliverable: A completed mock with a list of weak answers to revisit, both SQL problems solved and two questions to ask.

Behavioural preparation

The behavioral material in this loop is mostly about engineering judgement. The one behavioral question candidates report asks about a technical bottleneck, and the role description covers production debugging, peer review, estimates and working with product managers. Reports also mention explaining technical ideas to non-technical stakeholders. Prepare stories where you made the call, measured the result and can name the trade-off.

  1. How do you balance the trade-offs between system scalability and development speed?
  2. Tell me about a production issue you debugged end to end: how you found the cause, who you involved and what you changed afterwards.
  3. Describe a time you explained a technical decision to a product manager or other non-technical stakeholder and how you handled their push-back.
  4. Tell me about an estimate you got wrong: what you missed, how you communicated it and what you changed in your next estimate.

Describe a time you encountered a significant technical bottleneck and how you resolved it.

Medium · Problem-Solving and Logic

Approach

  1. Choose a bottleneck you diagnosed and fixed yourself, not one you watched from the side. Open with the symptom in one or two sentences using a real metric, for example job runtime, p99 latency or consumer lag, and say why it mattered to users or downstream jobs.
  2. Spend the middle on diagnosis. Say what you measured and with which tool (a profiler, the Spark UI stage and task view, MapReduce job counters, a .NET trace, a query plan), which hypotheses you ruled out and what the evidence was. A line like "I saw one task running far longer than its siblings, which pointed to key skew rather than memory" shows you measured before changing anything.
  3. Describe the fix and the cost you accepted. Examples: salting a skewed key adds a second aggregation step, caching spends memory and creates invalidation work, async I/O raises code complexity. Name one alternative you rejected and why, and mention what could have broken and how you limited that risk.
  4. Close with proof on the same metric: before and after, same data and load, plus what you left behind so it stays fixed (an alert, a benchmark, a runbook note). Use "I" for your own decisions and name teammates for theirs.
  5. The mistake that sinks this story is a fix without measurement, such as adding hardware or raising a timeout, or a bottleneck so small that the answer sounds like routine cleanup.

Follow-up

  • How did you know that was the bottleneck and not a symptom of something upstream?
  • What did your fix cost, and would you make the same trade-off again?
  • How did you confirm the improvement came from your change and not from a change in data volume or traffic?

Reverse your own decision and price the reversal

Medium · reversibility · measurement · migrations

Describe a technical decision you made and later reversed. Pick one that cost something: a service you split and merged back, a cache you added and removed, an index you created that pushed the planner onto a worse plan, a projection you rebuilt from scratch. State what you believed when you decided, the measurement that changed your mind, how long the wrong version ran in production, and what the reversal cost in migrations, dual writes, and a deprecation window for callers you did not own.

Approach

  1. State the original rationale without irony, in the version you would still defend given what was known then. If it is not defensible, the story is about carelessness rather than judgement, and a different example serves you better.
  2. Give the measurement that moved with a before and after: the p99 that did not improve, the cache hit rate that sat at 40%, the plan that flipped to a sequential scan once the table passed a size you can name.
  3. Cost the reversal in steps, not adjectives: expand-and-contract deploys, the dual-write window, the callers who had to be notified, the rows already written in the wrong shape that had to be backfilled or abandoned.
  4. Distinguish reversal from rewrite by naming what you kept. Most good reversals preserve the schema or the interface and undo one decision inside it, which is also why they were affordable.

Follow-up

  • What in that decision was irreversible, and did you know it was irreversible when you made it?
  • How did you tell the people who had already built on top of the original decision?

Turn a code review disagreement into a decision

Easy · code review · optimistic concurrency · communication

A colleague's change updates a row with UPDATE resource SET version = version + 1 WHERE resource_id = $1 AND version = $2 and treats an affected-row count of zero as a successful no-op. You read that as a silently lost update; they think returning 200 is friendlier to clients than returning a conflict. Describe how you have handled a review disagreement of this shape: what goes in the comment, when you leave the thread, and who decides. Then write the comment you would leave here, in under 80 words.

Approach

  1. Sort the disagreement before writing anything. A silently discarded write is a correctness claim about data; the choice between 409 and 412 is taste. Only the first justifies blocking a merge, and saying which one you are doing is most of the value of the comment.
  2. Make the claim reproducible in the comment itself with an interleaving rather than a principle: A reads version 7, B reads version 7, B commits version 8, A's predicate matches zero rows, A is told it succeeded and A's edit is gone.
  3. Offer the alternative with its cost attached: return 409 carrying the current version and the revision that won, so the client can re-read and re-apply. Note that automatic retry is not the fix, because a retry re-reads the winner's state and reapplies an intent formed against data that no longer exists.
  4. Apply an escalation rule you can state: two round trips on the thread, then a call, and the service's owner decides rather than the reviewer. A reviewer who cannot be overruled is a bottleneck with extra steps.

Follow-up

  • Where would you put the test that fails if someone reintroduces the swallowed zero rowcount?
  • The author says clients cannot handle a 409. How do you check whether that is true?