Databricks Liquid Clustering Interview Questions: Changed Keys, Old Files, and OPTIMIZE FULL

You change a Delta table's clustering key from region to customer_id, then rerun a customer query. It is still slow, even though the metadata looks correct. The metadata shows the new key, so why did the physical read pattern barely change? A strong interview answer separates the clustering policy from the layout of files already written.

For Databricks Liquid Clustering interview questions, trace the old files, new writes, and maintenance command separately. Then explain what evidence would show that the layout helps the actual workload.

This guide uses official Databricks AWS documentation checked October 6, 2026 and original illustrative file models. The examples concern an existing unpartitioned Delta table with manual clustering keys. They are not measured Databricks runs or reported employer questions.

What changes when you change clustering keys?

Official facts: Databricks's clustering guide says ALTER TABLE ... CLUSTER BY changes the clustering approach for subsequent writes and optimization; it does not rewrite existing data. Ordinary OPTIMIZE works incrementally. OPTIMIZE ... FULL reclusters existing data as necessary when enabling clustering or changing keys. After a previous full operation with unchanged keys, the guide describes subsequent FULL behavior as incremental, like ordinary optimization.

The interview distinction is therefore policy versus physical work. The policy states how clustering should organize eligible data. The file layout represents how existing data has actually been written. Reading a metadata field verifies the policy; it does not prove that old files reflect it, so inspect completed maintenance and read behavior separately.

Consider this original command sequence for the stated exercise:

ALTER TABLE customer_events CLUSTER BY (customer_id);
-- A metadata change alone does not recluster all old data.

OPTIMIZE customer_events;
-- Incremental maintenance is not proof of complete migration.

OPTIMIZE customer_events FULL;
-- Deliberately address existing data under the current keys.

These statements illustrate different intentions, not instructions to run all three in sequence on production. If the requirement is complete physical migration of old data, justify that requirement before choosing the expensive operation. If the table receives new data and old access patterns remain acceptable, incremental maintenance may be the relevant first action.

An answer that says “the key changed successfully” has verified only one step. Ask what changed in the query workload, whether old data dominates its reads, and what maintenance has completed since the change. Those questions determine whether slow performance is surprising.

Trace old and new files through a key change

Use an illustrative file ledger rather than inventing the engine's exact filenames or output file count. Assume F1 and F2 are already clustered under region, and a later append introduces F3 that needs maintenance. The new policy is customer_id.

Event	F1 and F2: previously clustered data	F3: newly appended data
Before key change	Existing region-oriented layout	Newly appended data
Change metadata to customer ID	Existing physical layout remains	Current policy changes for subsequent eligible work
Ordinary incremental optimization	Do not assume old-key files are fully migrated	Eligible new data can be organized under current keys
Full reclustering after the key change	Existing data is addressed as necessary for current keys	Included in the current-key layout work as necessary

The ledger's file identities are conceptual. Actual operations can replace, combine, or split files differently. Do not promise that F1 becomes exactly one F4 or that all trays contain equal bytes. The useful invariant is the separation between unchanged old layout after the metadata edit and deliberate physical maintenance later.

Now change the exercise: the workload still filters primarily by region and only rarely by customer. A key change based on one slow query might improve a small slice while hurting the common read pattern or increasing maintenance cost. Ask for representative filters, selectivity, and workload frequency before recommending a new policy.

Change another condition: only recent customer events are queried, and almost all of those arrived after the key change. The old-file migration may have less immediate value than it did when historical data dominated. This is an original decision inference from the workload, not a guarantee that recent files are already well clustered.

Finally, ask what happens if the policy changes again tomorrow. Repeated full maintenance without a stable workload hypothesis can consume resources while chasing transient behavior. Record why the keys were selected and what comparison will tell you whether to keep them.

State the runtime and documentation boundary

Official documentation discrepancy: the clustering guide names Databricks Runtime 16.4 LTS and above for forcing reclustering, while the current OPTIMIZE syntax reference labels FULL as available from 16.0 and describes rewriting files. These current official pages use different availability and rewrite wording; the discrepancy is not resolved by this documentary exercise. Use 16.4 LTS or later as this exercise's conservative baseline, and verify the actual runtime's supported behavior before a real change.

This is a better answer than silently choosing one version number or asserting that two pages say the same thing. The specific clustering guidance also says a repeated FULL with unchanged keys can be incremental. Avoid turning the keyword into an unconditional claim that every execution rewrites every byte.

Official syntax boundary: the reference supports partial reclustering with FULL WHERE from Runtime 18.1. It allows simple range predicates on one clustering column. A file is included when its range overlaps the predicate; the predicate is not a promise that only individually matching rows incur maintenance work.

For an original boundary example, suppose a file's date range is days 10 through 20 and the requested maintenance range is after day 15. The file overlaps the range. Treat it as eligible file-level work rather than predicting that the system can rewrite only the rows for days 16 through 20 without touching the rest of that file.

When an interviewer removes the Runtime 18.1 assumption, remove the partial-maintenance recommendation too. Do not suggest unsupported syntax and hope a newer feature happens to exist. State what you would inspect: runtime version, table type, current clustering configuration, and the applicable command reference.

This article deliberately does not generalize the Delta exercise to every Iceberg, automatic-key, hierarchical-clustering, or partition-conversion configuration. Those features have their own prerequisites and behavior. Identify the table and feature scope before borrowing a command from a neighboring example.

Explain data skipping without promising a speedup

Official mechanism: Databricks's data-skipping documentation describes file statistics used to avoid unnecessary reads. Statistics coverage matters; changing statistics configuration does not automatically recompute statistics for existing data. The clustering guide connects filtering on clustering keys to improved data skipping.

Here is an original min/max model for an equality filter on customer 42. All ranges are synthetic, with usable statistics assumed:

Before: candidate file ranges for customer_id
F1: [1,100]   -> cannot exclude 42
F2: [20,80]   -> cannot exclude 42
F3: [40,45]   -> cannot exclude 42
Candidate files: 3

After an illustrative improved layout
G1: [1,20]    -> exclude for 42
G2: [21,45]   -> cannot exclude 42
G3: [46,100]  -> exclude for 42
Candidate files: 1

“Cannot exclude” does not mean the file definitely contains customer 42. A range can include a value that has no matching record. Statistics narrow the candidate work; the query still evaluates records in retained files. This distinction prevents an incorrect claim that clustering creates a direct index lookup for every customer.

The model supports a three-to-one reduction in candidate files under its assumptions. It does not support a measured threefold runtime improvement. Files may differ in size, the cache may differ, other operators may dominate, and actual statistics may be less useful. State those limits next to the numeric example rather than attaching them as an unrelated disclaimer.

What if the query does not filter by customer? The changed layout may have little pruning value for that query. What if it asks for most customers? The candidate set can remain large despite a good layout. What if customer statistics are absent? The assumed exclusions cannot be justified from these ranges. Each follow-up changes the mechanism, not just the amount of tuning effort.

Use the model to explain the hypothesis you would test: a customer-selective workload should read less relevant file data after effective customer-oriented clustering, with the same results. Then measure that hypothesis on the actual system rather than treating the model as a benchmark.

Prove correctness and maintenance value separately

Before a physical change, preserve a comparable input snapshot or a defined observation window that accounts for concurrent writes. Record row count, key-level aggregates, and representative query results. Afterward, compare the same logical data. A layout change should not be credited for a faster answer that accidentally reads less required data.

For an original correctness check, use three customers with totals 40, 25, and 10. Their total is 75. A different file arrangement must preserve those values. Testing only the global total would miss an incorrect swap between customer amounts, so include per-customer results. If concurrent writes are allowed, control or account for them before attributing any difference to optimization.

Next, compare physical evidence: relevant files or bytes read, execution time under comparable cache conditions, and resources spent on maintenance. Add a representative query that still filters on the old dimension. A successful change should be evaluated against the agreed workload, not only the one query that motivated the change.

If the customer query remains slow, diagnose in order. Was the query actually filtering the selected key? Are the predicate and statistics useful? Did the intended old-data maintenance complete? Is the scan the expensive part, or does a join or aggregation dominate afterward? This avoids repeating FULL simply because a query took longer than expected.

Consider cost over a useful period. An expensive maintenance operation may be justified if a frequent workload saves enough resources afterward. It may be wasteful for an occasional query over a small slice. This is an original decision framework; the article provides no measured cost threshold or production speedup.

Liquid Clustering interview questions to rehearse

Question	Follow-up to rehearse
Optimize a SQL query plan tree	Separate a logical plan transformation from a physical file layout.
Optimize User-and-Region Queries on a Million-Row Table	Connect a selective predicate to the work a query actually reads.
Implement Column Pruning and Filter Pushdown on a Query Plan Tree	Explain pruning and pushdown without claiming every file is rewritten.
Explain Database Index Benefits and Write Costs	Compare a database index with clustering and include write costs.
Optimize least-k revenue queries for read/write load	Defend a workload-specific read/write optimization trade-off.

Rehearse the key-change scenario on paper before considering commands in a real workspace. State the table scope and runtime, identify which files predate the policy, explain why ordinary optimization does not prove a completed migration, and give the correctness and performance evidence you would collect.

Then change one requirement: only recent data matters, the original key remains the dominant filter, or partial reclustering is unavailable. Your recommendation should change with the evidence. Use Column Pruning and Filter Pushdown practice to explain why a logical plan change and a physical file-layout change solve different parts of the work. A strong answer predicts the work a command can change and the result it must preserve.

Sources and Further Reading

• Databricks: Use liquid clustering for tables
• Databricks: OPTIMIZE syntax and runtime parameters
• Databricks: Data skipping and statistics