cswork / libraryOpen practice workspace

Articles

Data Engineer vs Analytics Engineer: Pipelines, Models, and Interview Skills

Compare data engineer and analytics engineer responsibilities through CDC, model grain, and metric tests, then choose practical interview preparation.

Data engineers and analytics engineers both turn operational data into something other people can trust. The useful distinction is which contract they usually own. A data engineer often focuses on getting data into usable systems reliably. An analytics engineer often focuses on turning that data into tested, documented models with consistent business meaning.

That boundary moves between organizations. If you are choosing a role or deciding what to practice, inspect the work behind the title: recovery from a failed ingestion job demands different evidence from explaining why two revenue dashboards disagree.

Explain both how records arrive and what one output row represents. Where your explanation becomes uncertain is a useful preparation signal.

What each role usually owns

Official perspective: dbt describes analytics engineering as transforming, testing, deploying, and documenting analytical data, using practices such as version control and continuous integration. This is a vendor’s description of the discipline, not a universal job specification. dbt’s explanation of analytics engineering.

IBM’s comparison places data engineers closer to data infrastructure and pipelines, and analytics engineers closer to preparing trustworthy data for analysis. Those descriptions help orient a search, but a small team may assign both sets of responsibilities to one person. IBM’s role comparison.

A practical distinction is the failure you are expected to investigate. The data engineering question might be: “Can we recover yesterday’s missing records without corrupting today’s load?” The analytics engineering question might be: “Does this model count an order once, and do Finance and Product agree on the definition?”

Both require programming, debugging, and communication. SQL is central to many analytics engineering tasks, but it is also important in data engineering. Python experience alone does not settle the choice. Neither does familiarity with one orchestration or transformation tool.

Follow one order from a source change to a metric

Consider this original practice scenario, not an employer interview report. An online shop records order changes in an operational database. A change data capture pipeline, or CDC pipeline, sends those changes to a warehouse. A modeled dataset then feeds a paid-order dashboard.

For this exercise, each order has a monotonically increasing source version. An update replaces the previous full order state. The practice contract excludes partial updates and deletions; a real implementation would need explicit rules for both.

Order A appears at version 1 with status pending, then version 2 with status paid and an amount of $100. Order B appears at version 1 with status paid and an amount of $60. The pipeline receives A’s version 2 twice after a retry. Later, A’s older version 1 arrives out of order.

The expected current state is two paid orders totaling $160. The duplicate must not add another $100. The late older version must not turn A back into a pending order. That gives you observable correctness conditions before you choose a technology.

Official technical fact: Debezium documents that PostgreSQL connector failure recovery can produce duplicate events. This supports preparing for replay; it does not establish that every CDC product behaves identically or that the simplified version field in this exercise is a real connector offset. Debezium PostgreSQL connector documentation.

The data engineering preparation task is to preserve source identity, define ordering, retain recoverable input, and make reprocessing safe. Explain how you identify the entity, distinguish a repeated event from a newer state, and detect missing data. A successful scheduler run is evidence of execution, not proof of completeness.

The analytics engineering preparation task starts by defining the output grain: one row per current order. Then define the metric: sum the amounts of orders whose latest status is paid. State whether refunds, taxes, currency conversion, and later cancellations belong in this particular measure. Calling the result “revenue” without those decisions would overstate what the model represents.

A correct pipeline can still produce an incorrect dashboard

Now join the two current orders to their line items. A has two line items; B has one. If you sum the order-level amount after that join, the result is $260: A’s $100 is repeated twice, and B contributes $60 once.

The records arrived correctly. The query changed the grain. This is a modeling error rather than proof that ingestion failed.

A good answer explains why SUM(DISTINCT amount) is not a general repair. Two different orders may have the same amount, so deduplicating monetary values can discard legitimate business activity. Instead, preserve order grain for an order-level measure, or aggregate the line items to the intended grain before joining.

The exercise has three separate checks. Current-state selection should retain the highest valid version per order. The order model should have a unique order identifier. The metric should equal $160 under the stated definition, including after replaying the same input.

Then deliberately change the requirements. Suppose Finance wants recognized revenue while Product wants paid checkout value. Those measures may legitimately differ. The engineer should document the definitions and lineage rather than force matching numbers through an unexplained filter.

Two job task cards that change your preparation

The following cards are synthetic examples designed to help you read job descriptions. They are not anonymized evidence from actual employers, and their interview emphasis is our preparation recommendation.

Card A: ingestion and recovery ownership. The team needs someone to ingest changing source tables, investigate delayed loads, manage backfills, and reduce infrastructure costs. It expects operational ownership when a source schema changes or a scheduled run fails halfway through.

For this card, prepare a pipeline design that includes checkpoints, safe reruns, rejected records, and recovery validation. Bring an example of a failure you introduced intentionally. Show the incorrect result, the diagnostic evidence, and the corrected result after recovery. Be ready to discuss what happens when a source deletes a record or modifies data outside your usual incremental window.

Card B: analytical model ownership. The team needs someone to maintain shared business models, resolve inconsistent metric definitions, review transformation changes, and help analysts use stable datasets. It expects the engineer to explain dependencies and communicate breaking changes to consumers.

For this card, prepare SQL that makes grain explicit, tests key assumptions, and separates reusable logic from a dashboard-specific calculation. Bring a small model contract with accepted inputs, output columns, ownership, and known limitations. Show a change request that requires a definition decision before it requires another query.

The title on either card could vary. A company may call Card B “data engineer,” or combine both cards into an analytics engineering position. Apply the comparison to the responsibilities you will be assessed on, rather than rejecting a relevant opening because its title feels imperfect.

What to practice for the interviews

For an ingestion-heavy role, practice coding transformations with explicit failure behavior. Given duplicate files or repeated events, can your program produce the same final result? Given a malformed record, can it preserve enough context for investigation without silently dropping an entire partition?

For a modeling-heavy role, practice joins, window functions, aggregation, and reconciliation. Explain why a filter belongs before or after an aggregation. Show how a slowly changing business attribute affects historical reporting. When requirements are ambiguous, ask what the consumer wants to know before selecting a SQL pattern.

For both, practice reading an execution plan and explaining a performance trade-off. A faster query that changes the answer is not an optimization. Likewise, adding partitions or incremental logic creates new correctness assumptions that you should be able to name.

Communication is also technical evidence. Describe a disagreement using the competing definitions, the affected consumers, and the decision record. “I worked with stakeholders” tells an interviewer much less than “we separated booking value from recognized revenue and migrated the dashboard to the agreed measure.” Only use that wording as your own experience if it is true.

Use these question pages as targeted practice prompts. They are relevant exercises, not a claim that a particular employer will ask them.

QuestionWhat to demonstrate
Model Data for Analytics, Reporting, and ApplicationsExplain grain, consumers, and the difference between operational and analytical needs.
Design Incremental Load Process for Large Relational TableDefine the change boundary and how reruns avoid corruption.
Design a scalable dimensional modelDefend fact-table grain and dimension relationships.
Diagnose data quality and pipeline performance issuesSeparate missing inputs, incorrect transformations, and slow execution.
Define and validate product metricsState the business definition and checks that could falsify it.

Build a project that exposes the boundary

You do not need a large cloud deployment to make this comparison tangible. Use a small dataset, a local database, and version-controlled transformations. Label it as a local exercise. Do not describe a simulated outage as production incident experience.

Create a raw input layer containing the five deliveries from the order example: A version 1, A version 2, B version 1, a duplicate A version 2, and another A version 1. Keep an ingestion timestamp separately from the source version. That distinction lets you explain why “last received” and “latest source state” are different concepts.

Build a current-orders table and a paid-order summary. Add the line-item join deliberately, observe the inflated total, and repair it. Save the expected outputs alongside the code so a reviewer can understand the failure without running a complex environment.

For data engineering evidence, emphasize the recovery path. Stop the process after some deliveries, restart it, and compare the final table with a clean run. Explain the limits of your test: a local fixture does not establish distributed exactly-once processing or production throughput.

For analytics engineering evidence, emphasize the model change. Add a requirement to exclude cancelled orders, then explain whether historical reports should restate. Write down the decision, the affected downstream output, and the regression check. The business assumption should be visible in the project rather than buried in a WHERE clause.

A compact repository with these artifacts can support a more useful discussion than an elaborate architecture diagram whose correctness you cannot demonstrate. Include setup instructions, the expected $160 result, the deliberately broken $260 result, and the reason the fix works.

Decide which work you want to own

Choose a preparation direction by noticing which questions you want to pursue. If you want to investigate checkpoints, source behavior, storage choices, and recovery, prioritize data engineering roles with that ownership. If you want to resolve grain, definitions, lineage, and trusted reusable models, prioritize analytics engineering roles with those responsibilities.

Neither preference is a permanent boundary. A data engineer can develop stronger modeling judgment; an analytics engineer can develop deeper orchestration and platform skills. Start from your existing evidence and add the missing responsibilities named in the jobs you actually want.

Before an interview, ask who owns ingestion incidents, who approves metric changes, and who supports downstream consumers. Ask whether the role includes on-call work and how that responsibility is shared. These answers are more useful than assuming a job title guarantees a particular schedule or technology stack.

Return to the incremental-load practice question and the modeling question above. Explain both in plain language, identify the assumption you cannot yet defend, and build one small test around it. That is a concrete next step toward choosing a role and becoming ready for its interviews.

Sources and Further Reading