cswork / libraryOpen practice workspace

Articles

Python itertools Interview Questions: Lazy Iteration, groupby, and Shared Iterators

Practice Python itertools interview questions with lazy pipelines, groupby consumption, shared iterators, tee buffering, and explicit memory boundaries.

A Python pipeline returns an iterator, but consumes every input before its first result. Another groups identical keys, but emits A twice. A third duplicates a source, yet keeps a growing backlog in memory. These are different consequences of iterator contracts, not contradictions in the word “lazy.”

For Python itertools interview questions, track input consumption, retained state, and output production separately. That distinction explains groupby's adjacent runs, tee's consumer lag, and product's input pools more precisely than claiming every iterator uses constant memory.

Evidence boundary: Official CPython 3.14 documentation was reviewed October 1, 2026. Fourteen original iterator checks ran on Python 3.12.14. Newer batched(strict=True) behavior is documented, not claimed as executed on that runtime. Examples are original practice, not an employer question set.

Begin with Implement a Lazy Tee Iterator in Python to practice the shared-source and buffering boundary directly.

What does lazy iteration actually guarantee?

An iterator provides successive values through next. Laziness describes when work happens; it does not specify every operation's memory use, whether input is reusable, or how much upstream work one output needs. The official itertools documentation source documents each function's behavior separately.

For an interview, ask three questions about the exact pipeline. When does it first read the source? Which objects remain reachable between outputs? What happens if a consumer stops or advances ahead of another? A function name alone cannot answer all three.

Also distinguish an iterable from a particular iterator. Calling iter on a list can create another traversal. Sharing one generator object shares its position. Two variables referencing that object do not create two independent passes.

A source that performs I/O adds ownership questions. Iterator composition does not automatically provide deterministic file closure, retries, or cancellation. Keep this article's finite in-memory probes separate from a production file or network reader's lifecycle contract.

Why does groupby produce three groups for AABA?

itertools.groupby groups consecutive equal keys. For A, A, B, A, it produces an A run of length two, a B run of length one, and another A run of length one. It does not collect all A values across the sequence like a global SQL grouping operation. CPython groupby reference

from itertools import groupby

runs = [(key, list(group)) for key, group in groupby("AABA")]
# Native probe: [('A', ['A', 'A']), ('B', ['B']), ('A', ['A'])]

This behavior is useful when the task is run-length encoding, detecting consecutive states, or aggregating a stream already ordered by the grouping key. Sorting first is not always a repair: it can destroy the original adjacency that the task wanted to measure.

If the requirement is a global total per key, consider dictionary accumulation. If whole groups must be produced in key order, sorting by the same key and then grouping may fit, but introduces materialization and comparison-sort cost. State the requirement before choosing the technique.

A dictionary comprehension creates another trap. {key: list(group) for key, group in groupby("AABA")} overwrites the first A entry with the later A run. A plausible dictionary result can therefore hide a mismatch between adjacent grouping and global grouping.

Why can an earlier group become empty?

The outer groupby iterator and its yielded group iterators share a source. Advancing the outer iterator moves past the previous run. Saving a group object does not save its items.

outer = groupby("AABA")
key_a, first_a = next(outer)
key_b, first_b = next(outer)
print(list(first_a))
# Native probe: []

The output is not evidence that the original A run had zero elements. It means the source has advanced beyond the position from which that group could read. This is a lifetime problem for a view onto shared iteration state.

Materialize a group while it is current if you need its contents later. If only a count is needed, consume and count it at that point rather than retaining a list of every value. Both choices acknowledge that consuming the outer iterator changes the availability of earlier groups.

list(groupby(data)) is particularly misleading. It stores pairs containing group iterators, not independent lists. In the native finite fixture, all saved group iterators were empty after the outer iterator had been exhausted. To preserve data, materialize each inner group during the traversal.

Does islice skip without consuming the source?

No. Skipped positions in a one-pass iterator must still be traversed. For a fresh source over zero through nine, fully consuming islice(source, 2, 6, 2) returns 2 and 4, while the next source item is 6.

from itertools import islice

source = iter(range(10))
selected = list(islice(source, 2, 6, 2))
following = next(source)
# Native probe: selected = [2, 4], following = 6

The result has two values, but obtaining it advanced across six source positions. If reading a source item costs a database call or decompression step, output length is not a reliable work estimate. The API's islice description explicitly distinguishes returned positions from input advancement.

Creating an islice does not generally perform the traversal itself. Consuming it triggers the reads. If consumption stops after the first selected item, the source position differs from the fully exhausted case above. Be precise about what the test actually consumed.

Negative slicing is another boundary: islice is not a list slice supporting negative indices or reverse steps. A one-pass source cannot jump backward just because the requested syntax resembles sequence slicing.

What happens to the first item that fails takewhile?

Takewhile reads values while the predicate succeeds. To discover that the next item fails, it must consume that item. It does not leave the failing boundary value available for a later next call.

The original probe used [1,2,0,3] with bool as the predicate. Takewhile yielded [1,2]; the next source value was 3, not 0. That matters for parsers where zero, a delimiter, or a sentinel must be handled by another stage.

If the boundary item belongs to the next logical section, use a design that explicitly preserves it: a lookahead buffer, a parser that returns the delimiter alongside the prefix, or a suitable helper whose contract you verify. Do not choose a library function merely because its name sounds like the English requirement.

Distinguish this from dropwhile, which suppresses an initial matching prefix and then yields the remaining input. Similar-looking predicate APIs have different ownership of the transition item. A four-value trace is a faster clarification than a long abstract explanation.

Does tee provide two copies with constant memory?

Tee provides consumers with independent iteration positions over one underlying source. Values needed by a slower consumer must remain available after the faster consumer has passed them. Auxiliary storage can grow with the gap between consumer positions. The documentation also warns that tee iterators are not thread-safe. CPython tee reference

In the native probe, the fast consumer requested 0, 1, and 2, causing three source reads. The slow consumer then requested 0 from retained state; no fourth source read occurred. This proves sharing and replay of retained items for that trace, not a measured byte count.

If one consumer exhausts a million large records while another has not started, the retained backlog can be substantial. Laziness of output does not make that gap disappear. If both consumers will eventually traverse a finite dataset sequentially, an explicit list may communicate the storage cost more honestly.

For continuous pipelines, define whether consumers must stay within a bounded lag, whether a slow consumer can be dropped, or whether replay comes from external storage. Tee alone does not impose backpressure or a fixed memory ceiling.

Avoid continuing to consume the original iterator separately after handing ownership to tee branches. That would bypass their shared consumption mechanism and make a supposedly identical traversal skip items. Independent branch positions are not independent underlying I/O operations.

Why can product read everything before yielding one tuple?

Product generates Cartesian-product outputs incrementally, but first pools its input values. The native fixture logged five input reads before the first product tuple was requested. Its lazy result iterator is therefore not an input-streaming guarantee. CPython product reference

For two pools of lengths three and four, there are twelve output tuples. Product need not materialize those twelve results at once, but it retains the input pools. With five parameters each having ten choices, output cardinality is 100,000 even though the pools contain only fifty input values.

Passing an infinite source is not a clever way to avoid storage: product cannot finish constructing that pool. Bound the inputs before constructing it when a finite search space is intended. Combinations similarly needs a pool from which to select; do not classify every combinator as fully streaming.

Cycle has a related retention issue for a different reason. It saves source items to replay them. With an unbounded source, it keeps seeing new items without completing its first cycle. Explain the source bound and retention purpose rather than promising constant memory for “an infinite iterator.”

How do you batch without hiding a malformed final chunk?

Python 3.12 introduced batched. In the native runtime, batching five values into groups of three produced (0,1,2) and (3,4). A short final batch is allowed by that default contract.

Python 3.13 added strict mode, which rejects an incomplete final batch when enabled. That newer behavior was reviewed in the official documentation, not executed in the Python 3.12 fixture. Do not submit a strict keyword on an older assessment runtime and expect it to work.

Decide whether a short batch is valid input, an error, or something the consumer can process differently. Padding is another policy and may introduce artificial values; it is not a neutral way to make shapes match.

Use tests with zero, exactly three, and five inputs. The ordinary full batch cannot distinguish acceptance from rejection of a short tail. Version checks and boundary fixtures work together: neither a memorized signature nor one passing output is enough.

Sources and Further Reading