Articles
HBase Interview Questions: Row Keys, Hot Regions, and Read-Write Trade-Offs
Practice HBase interviews with a four-bucket row-key layout matrix, one-hot-device counterexample, Get/Scan trade-offs, and region metrics.
An HBase row-key interview is a workload-design exercise. A timestamp-first key makes an all-device time scan tempting, but a stream of increasing timestamps can push new writes into one narrow row-key range. A prefix that distributes those writes can protect throughput, yet it changes which reads remain contiguous. The useful answer names the exact Get and Scan requests, draws the resulting key ranges, and measures a hot region instead of calling any key “balanced” by intuition.
The behavior below follows the Apache HBase row-key design guide, client/region architecture, and read-consistency reference. The four-bucket telemetry fixture and workload decisions are original interview exercises, not candidate reports or evidence that a company asks these exact prompts. For adjacent practice, use Design a Threshold-Based Database Partition Rebalancer.
Why does row-key order dominate the design?
HBase stores rows in lexicographic row-key order. Its reference guide emphasizes that related rows can be close together for scans, and that this same ordering creates hotspot risk when many operations land in one range. Regions partition contiguous key ranges and are served by RegionServers. The client locates the relevant range and contacts its server, as described in the client architecture. Therefore the leading bytes of a key influence both placement and read locality.
Start an interview by asking for the access contract. Is the common operation a point lookup by device and exact event ID? The latest 100 readings for one device? A range of timestamps across all devices? A scan of all events for one tenant? No single ordering makes every one of these a cheap contiguous scan. A predicate on a column that is not encoded into the useful key prefix cannot be assumed to prune to one small row range. Filtering after a broad scan is not the same as fetching a narrow range.
Also state uniqueness. If device_id|timestamp is the entire key and two readings have the same timestamp for one device, a write may address the same row rather than create two independent events. Add a sequence or stable event ID where the business grain requires it. This is separate from hotspotting: a perfectly distributed key can still overwrite logical events if it is not unique enough.
Why can a timestamp-first key create a hot region?
Imagine a table of telemetry with keys timestamp|device_id|event_id, where timestamps grow as events arrive. At any moment, new writes have similar leading bytes. They cluster toward the right edge of the sorted keyspace; the region covering that edge can become much busier than its neighbors. HBase's hotspotting discussion defines a hotspot as traffic concentrated on one or a few cluster nodes and recommends distributing writes among regions. This is a workload tendency, not a proof that every timestamp table is hot: ingest rate, pre-splits, client distribution, and key encoding all matter.
Averages conceal it. Suppose four RegionServers each host several regions. Their total disk utilization looks similar, but one newest-time region receives 8,000 writes per second while the next highest receives 900. The relevant evidence is region-level request rate, queueing, latency, compaction pressure, and split movement. A cluster-wide average does not reveal the hottest range. If writes arrive in time order and all target the same high-end region, simply adding nodes may leave the single range as the bottleneck until key layout or split strategy changes.
Pre-splitting is not a magic cure. HBase's performance guidance warns that pre-split boundaries must fit the actual keyspace; data can still land in one region when the planned distribution is wrong. With a monotonically increasing leading timestamp, a set of historic split boundaries may do little for the new right-edge writes. Show expected key samples against region start/end keys before declaring the split plan effective.
Compare three key layouts on one workload
Use this original four-bucket fixture. There are 16 devices with roughly even traffic in the baseline. The application needs (1) an exact event Get, (2) recent readings for one device, and (3) a global time-window scan. Compare these layouts before selecting one:
| Layout | Write placement | One-device recent scan | Global time scan |
|---|---|---|---|
| `time | device | event` | New timestamps tend to one high-end range |
| `hash(device)%4 | device | reverse_time | event` |
| `hash(event)%4 | device | time | event` |
The key strings are explanatory sketches, not a production encoding specification. Make widths, delimiters, byte order, and timestamp reversal explicit in a real implementation. For the second layout, a device's prefix is deterministic, so the client can construct a point key and scan its recent readings within a contiguous device subrange. But one device that accounts for 80% of writes can still hot-spot its assigned bucket. Hashing device ID spreads devices, not an individual hot device's events. The third layout spreads the individual device by event ID at the cost of four-way fan-out for its history.
This illustrates the central trade-off: distributing writes changes how much work readers do. HBase's row-key guide explicitly describes random salting's read cost and deterministic hashing's ability to reconstruct a key for Get. A random prefix may spread writes but makes an exact lookup harder unless the prefix is stored in another index or derivable from known fields. An exact Get is cheap only when the caller knows the complete row key. “We can always query by ID” is false if the ID alone cannot reconstruct its prefix.
Should the prefix be random, hashed, or reversed?
Random salting at the front of the key disperses adjacent natural keys. The HBase reference says this can reduce hot writes but makes ordered retrieval more work because related records no longer occupy one natural run. A deterministic hash of a known stable field allows the client to regenerate the prefix and perform an exact Get; it may also keep one entity's rows together. Choose the hash input to match the skew pattern. Hashing device_id helps with many similarly active devices; hashing event_id may distribute one overwhelming device but sacrifices a single-range device scan.
Reversing a timestamp or fixed-width numeric key can put changing low-order bytes earlier in the key and alter hotspot behavior; HBase documents reversing as one technique, with ordering trade-offs. Do not claim that a reversed timestamp automatically spreads all concurrent writes across the cluster. A reverse-time suffix after an identical device prefix still sends that device's traffic into its device range. The purpose there may instead be efficient newest-first retrieval within a device. Every prefix and suffix should have a stated reader or writer benefit.
Choose a bucket count with measured load. Four is convenient for the interview matrix, not a recommended production constant. Too few buckets may fail to spread traffic; too many increase scan fan-out and operational overhead. A bucket prefix only distributes across regions if the region split boundaries and actual keys place those prefixes on multiple servers. Inspect distribution after ingestion, especially if the source's ID distribution is skewed or the hash function changed.
How do point reads and scans change under those layouts?
For a Get, the client needs the full key or a way to derive it. In hash(device)%4|device|reverse_time|event, a caller with device, timestamp, and event ID can compute the bucket and construct the exact key. A caller with only event ID cannot, unless another mapping exists. That mapping has write and consistency costs of its own. For a one-device recent scan, the deterministic device bucket plus device prefix lets the client bound the scan; reverse-time encoding can put recent rows earlier in that subrange.
For a global time interval across all devices, the device-leading layout is unfavorable because time is not a common leading prefix. The application may issue bounded scans for each bucket and device, use a second purpose-built table/index, or choose a different primary access pattern. Parallel scans increase client complexity and merge work. A one-off wide scan may be acceptable for an offline job; a latency-sensitive API doing it on every request may not be. Interviewers value this admission more than a promise of universal constant-time reads.
Does HBase mean eventual consistency?
Avoid importing another database's rule. Apache HBase's timeline-consistent-read documentation states that Consistency.STRONG is the default for reads through the primary region. It also describes an optional Consistency.TIMELINE path using region replicas, where a secondary can serve a stale result. So “HBase is eventually consistent” is an incorrect blanket answer; “every HBase read under every option is strongly current” is also too broad. Name the API consistency choice and replica path.
In the telemetry fixture, an exact read immediately after a confirmed write should be evaluated under the chosen consistency and failure model. If a dashboard opts into timeline-consistent secondary reads for availability, it should tolerate staleness and communicate the freshness contract. If an account-balance workflow demands current state, it should not silently switch that read path because a region is busy. Key design and consistency are separate decisions, though a hot region may motivate someone to consider replica reads.
Do not claim that a salted key fixes read consistency. It distributes requests; it does not change the semantics of primary versus secondary reads. Similarly, region splitting and rebalancing are placement mechanisms, not a substitute for a caller knowing the correct row key. This distinction is often what a system-design interviewer is looking for.
What evidence would confirm a hotspot?
Capture a time window of per-region write and read requests, latency, handler queues, region-server CPU, flush/compaction activity, and start/end keys. Compare the busiest region with the rest, then sample actual row keys. If the hot region owns the newest timestamp range, the key hypothesis has support. If a single device dominates but keys are device-hashed, the skew hypothesis has support. If reads, not writes, dominate, randomizing writes may not address the incident.
Build a small load test with the real key encoder and a representative key distribution. Insert a synthetic stream, verify the bucket count per prefix, and compare region-level latency. Then run the two actual read paths: one-device recent scan and global time interval. Include results returned, rows scanned, bytes read, RPCs, and p95/p99 latency. A design that improves write p99 while multiplying a critical read path by four needs an explicit product decision, not a victory claim.
Check correctness too. Assert unique event keys across timestamp ties, stable key reconstruction, sorted recent results after any multi-bucket merge, and no missing rows at range boundaries. Test a deliberately skewed device that generates most traffic. The baseline with 16 even devices is only a first exercise; the 80% device case tests whether the chosen hash input actually addresses the hardest workload.
What is a concise interview answer?
“I would list the required Get and Scan requests before choosing the row key. Timestamp-first order helps a global time scan but can concentrate increasing writes on the newest region. A deterministic device hash prefix spreads many active devices and lets a caller reconstruct a device key, while making global time scans fan out; it does not cure one dominant device. Hashing per event can spread that device but makes its history a multi-bucket query. I would inspect region start/end keys and per-region request rates to prove the hotspot, then benchmark both writes and reads with the real skew. I would also state that primary-region reads are strongly consistent by default, while optional timeline replica reads can be stale.”
If the dominant query changes from recent-by-device to global time-range, revisit the selected layout.