cswork / libraryOpen practice workspace

Articles

Apache Druid Interview Questions: Rollup, Time Granularity, and Reingestion

Practice Apache Druid interviews with a four-event rollup trace, query versus segment granularity, best-effort streaming, and safe interval reingestion.

An Apache Druid table shows three rows for four incoming events. A dashboard calls COUNT(*) and reports three visits. Someone changes the timestamp granularity, reingests yesterday, and the total grows again. Which count was ever supposed to mean “visits”?

The answer starts with the stored grain. Druid can aggregate events during ingestion; a stored row may summarize more than one input event. The official Druid rollup guide explains the mechanism and the loss of individual-event detail. The event ledger and correction below are original interview exercises, not candidate reports or a claim that any named employer asks these questions. For adjacent metric-grain practice, see Model Data for Analytics, Reporting, and Applications.

Question 1: what does rollup combine?

Druid's native ingestion granularitySpec controls rollup, queryGranularity, and segmentGranularity. With rollup enabled, input records with the same timestamp after ingestion-time truncation and matching selected dimensions can be combined; aggregators produce metric values. That is an official product behavior. It is not automatic deduplication of business events. Two identical source deliveries can contribute twice to a count unless the source or preparation path removes the retry. Likewise, two rows that differ in any retained dimension may remain separate even in the same minute.

Consider four invented click events on September 1, all UTC. The ingestion design retains channel and region as dimensions, truncates timestamps to the minute, and stores two metrics: events as a count aggregator and revenue as a sum. Assume a batch method that guarantees perfect rollup for this tiny example.

Raw eventTime UTCChannelRegionRevenue
A12:00:05webUS5
B12:00:40webUS7
C12:00:50appUS3
D12:01:03webUS11

Events A and B collapse to one row at 12:00 with events=2 and revenue=12. C stays separate because channel=app; D stays separate because it belongs to 12:01. This yields three stored rows, four input events, and revenue 26. The equation is easy; the meaning of each field is the interview signal.

Stored minuteDimensionseventsrevenue
12:00web, US212
12:00app, US13
12:01web, US111

Now COUNT(*) over those stored rows is 3, SUM(events) is 4, and SUM(revenue) is 26. The first answers “how many physical rollup rows?”; the second answers “how many counted input events under this ingestion contract?” The Druid rollup documentation gives the same general warning: use a stored count metric and sum it to recover an input-event count. If your metric is a distinct user sketch or another aggregate state, query it with the compatible finalization function instead of assuming every metric is additive in the same way.

Question 2: which granularity are we talking about?

There are three separate decisions, and a fourth related one: the batch interval you overwrite. queryGranularity in an ingestion spec is the timestamp resolution stored within a segment; minute granularity floors these timestamps to a minute. segmentGranularity groups storage into time chunks such as a day; it does not itself floor every event to a day. A query's own requested granularity groups results for that query. Finally, a REPLACE time interval selects which time chunks are rewritten. The official ingestion spec says ingestion queryGranularity must be as fine as or finer than segment granularity.

ControlIn this exerciseWhat changesWhat it does not do
Ingestion queryGranularityminuteStored time and rollup matching windowChoose the physical day chunk
segmentGranularitydayPlacement in a daily time partitionForce daily metric aggregation
Query-time granularityhourResponse bucket for a particular queryRestore lost seconds from ingestion
REPLACE intervalSeptember 1 dayWhich time chunk is overwrittenIdentify a single event by primary key

If you stored only minute timestamps, an hourly query can combine minute metrics; a second-level query cannot recover the original five- and forty-second positions. If you set ingestion queryGranularity to none, Druid's reference says timestamps remain at millisecond resolution, but rollup can still combine rows with exactly equal timestamps and dimensions. none is therefore not synonymous with rollup=false. The query granularity reference also cautions that none in a native timeseries query has different implications for dense empty buckets. Keep ingestion and query usage distinct.

Question 3: why might streaming show more than three rows?

The three-row table is a perfect rollup result for our stated batch assumption. Druid's current rollup guide says Kafka and Kinesis streaming ingestion are best effort: records with the same truncated timestamp and dimensions may land in different segments as ingestion publishes incrementally. Native batch may be perfect or best effort according to configuration, and Druid's SQL-based batch ingestion has a perfect-rollup path. Do not promise three physical rows merely because rollup=true appears in a streaming spec.

If A and B land in different segments, two web/US 12:00 rows can survive. COUNT(*) might now be four instead of three, while SUM(events) can still be four and SUM(revenue) 26 if all four input events were each applied once. That qualification matters: best-effort rollup does not remove replay duplicates. A metric discrepancy could come from duplicate source records, different dimensions, missing events, or reading the wrong metric, not just imperfect physical compaction.

The practical diagnostic is to query a small bounded interval grouped by stored __time and dimensions, compare physical row counts with SUM(events), inspect segment/task history, and reconcile source identities where raw data exist. Druid's rollup page suggests a rollup ratio based on input count divided by physical row count. Compaction or reindexing can combine compatible rows later, but cannot manufacture event IDs after they were dropped. If exact source-event accounting matters, retain a raw source of truth outside the rolled-up datasource or a separate no-rollup datasource.

Question 4: how would you correct yesterday's revenue?

Suppose event B should have revenue 9, not 7. The correct four-event revenue becomes 28, while the event count remains four. After rollup, the stored web/US 12:00 row contains total 12, not an independently addressable B amount of 7. You cannot identify and update B in place by its event ID if that ID was not retained. Druid's data updates guide says it does not support single-record updates by primary key; it rewrites time ranges instead.

The safe repair is to obtain an authoritative corrected input for the relevant September 1 interval, compute the expected bucket totals before ingestion, and replace the affected time chunk. SQL REPLACE INTO ... OVERWRITE WHERE ... or native batch overwrite are documented routes; choose the method and interval for your configured datasource. A blind INSERT of corrected B into the existing rolled-up datasource would add to the prior aggregate, yielding 35 for the invented total rather than 28. A replay of all four events with INSERT could double the whole day. Neither is a correction.

There is a subtle reindexing limit. Druid can reindex a datasource from its existing segments to change schema or repartition, but our original B value is already mixed with A in the aggregate 12. Reindexing only those rolled-up segments cannot replace B's 7 with 9 unless an external correction can be applied at the aggregate grain with proven completeness. The simplest defensible answer uses the retained raw events as the correction source. This is an inference from the documented irreversible loss of individual events after rollup, not a claim that Druid cannot perform any aggregate-level adjustment.

For SQL REPLACE, the official multi-stage query concepts require the overwrite range to align with PARTITIONED BY granularity. With daily partitioning, choose an aligned full-day range, such as 2026-09-01 00:00:00 inclusive through 2026-09-02 00:00:00 exclusive, in the agreed time zone. Preview the corrected input and count its four IDs once. Then run the replacement and validate SUM(events)=4, SUM(revenue)=28, and the dimensions/time-bucket table. Preserve task ID, input version, and a before/after comparison for an audit trail.

Question 5: what can readers see during reingestion?

Druid's update documentation says read-only queries can continue to see the prior version while an overwrite for a time range runs, and that its atomic update mechanism flips old to new time chunk by time chunk. Ingestion targeting that same range is queued while the overwrite holds the relevant lock; other ranges can proceed. This is a stronger and more specific statement than “the entire datasource updates atomically.” If your job rewrites multiple chunks, do not promise one whole-table instant flip. Ask whether a dashboard spans several chunks and whether a publish barrier is needed for a cross-day report.

Before starting, inspect the interval and other ingest jobs. A REPLACE with OVERWRITE ALL is much broader than a one-day correction. A misaligned range may be rejected or require a different partitioning plan. After completion, query the corrected chunk, adjacent chunks, and a query crossing the boundary. Confirm September 2 did not change. If live data continued arriving for September 1 during the repair, reconcile the queued ingestion and runout before declaring the final value. Task success plus a changed chart is not sufficient evidence of correct replay semantics.

A concise interview answer and follow-ups

Try this answer aloud: “The four raw events produce three perfect-rollup rows because two share minute and dimensions. COUNT(*) counts stored rows; SUM(events) counts the four source events under the stated clean-input assumption. The minute controls stored timestamp grain; day controls segment placement. A streaming job may have more physical rows from best-effort rollup, so I would still aggregate the metrics. To change B from 7 to 9, I would rebuild the aligned day from corrected raw input, replace that time chunk, and verify four events and revenue 28.”

Then answer three harder follow-ups. If the interviewer asks for exact per-event deletion, explain that rollup discarded the event identity and propose a raw archive or separate no-rollup datasource. If they ask why the rollup ratio collapsed, inspect high-cardinality dimensions and timestamp resolution before changing infrastructure size. If they ask whether compaction fixes duplicated source deliveries, say no: compaction can merge physical aggregates, but a duplicate counted twice remains in the sum. Each answer ties an operational action to the data grain it can actually change.

Sources and further reading