← Blog
Automation & Scripting

Number Python iterations without indexing the data

Use enumerate when a loop needs a counter, then separate display positions from durable identity and make retries, mutation, and streaming behavior explicit.

By Emmanuel Corels

When a Python loop needs both each value and a running number, use enumerate(iterable). It yields (count, value) pairs lazily, starts at zero by default, and accepts an explicit start. The useful production question is not only how to obtain the counter, but what that number is allowed to mean.

Use the number your interface actually promises

items = [8, 23, 45]

for position, value in enumerate(items, start=1):
    print(f"item #{position} = {value}")

start=1 is appropriate for human-facing positions. Keep the zero-based default when the count intentionally corresponds to a zero-based offset. Naming the variable position, line_number, or attempt is usually more honest than calling every count index.

The counter does not come from the iterable and is not necessarily a valid subscription key. It works with generators, file objects, database cursors, and custom iterators that cannot be indexed:

with open("jobs.ndjson", encoding="utf-8") as stream:
    for line_number, line in enumerate(stream, start=1):
        process(line_number, line)

This streaming property avoids loading the entire input merely to manufacture positions. Memory remains bounded by the iterator and processing logic.

Do not replace iteration with range(len(...)) by reflex

# Usually unnecessary:
for i in range(len(items)):
    process(i, items[i])

# Expresses the actual need:
for i, item in enumerate(items):
    process(i, item)

The first form couples the loop to sequence length and integer subscription. It fails for general iterables and encourages repeated lookups. It is still legitimate when the algorithm truly needs neighboring elements, writes back by position, or traverses several arrays under a verified alignment invariant. In that case, document the invariant and test mismatched lengths.

A position is not durable identity

Automation frequently writes the enumerate count to logs or checkpoints. That count is only the position in this particular traversal. Filtering, sorting, retrying, pagination, concurrent insertion, or a changed upstream query can assign a different number to the same business object.

for display_position, job in enumerate(ready_jobs, start=1):
    logger.info(
        "processing job",
        extra={"position": display_position, "job_id": job.id},
    )

Use the job’s durable ID for idempotency, deduplication, audit correlation, and resume checkpoints. Use the position for progress presentation. If an upstream API supplies a stable cursor or offset, store that separately rather than assuming the local count can recreate it.

Understand consumption and restart behavior

An enumerate object is itself an iterator. Once consumed, it does not restart:

numbered = enumerate(load_jobs(), start=1)
first = next(numbered)
for position, job in numbered:
    process(job)

The loop begins with the second item because the first pair was already consumed. This is useful for peeking only when deliberate. To repeat traversal, create a new iterator from a repeatable source; generators and network streams may not be repeatable at all.

On retry, enumerate(..., start=saved_position) changes labels but does not skip input. Resume requires the source itself to seek, filter, or continue from a durable cursor:

cursor = checkpoint.load()
for position, job in enumerate(fetch_after(cursor), start=1):
    handle_idempotently(job)
    checkpoint.save(job.cursor)

Write the checkpoint only after the side effect is safely complete, or use a transaction/outbox design. The local position can still appear in progress logs, but it is not the recovery boundary.

Avoid mutation surprises

Changing a list’s length while enumerating it can skip work, repeat work, or produce hard-to-reason-about results. Build a new collection or iterate over an intentional snapshot:

for position, item in enumerate(list(pending), start=1):
    if obsolete(item):
        pending.remove(item)

A snapshot costs memory and may be stale under concurrency, so it is not a universal cure. For queues and databases, use their claiming and locking semantics instead. Modifying fields on the current object can be safe, but structural changes to the iterable deserve an explicit design.

Parallel work breaks the “current position” illusion

You may assign positions before submitting tasks to a pool, but completion order can differ. Log both submitted position and durable identity, and do not report the latest completed position as a contiguous checkpoint unless all earlier tasks are confirmed complete.

submitted = list(enumerate(items, start=1))
for position, result in executor.map(run_numbered, submitted):
    record(position, result)

Materializing the list is acceptable for bounded inputs; for large streams, use bounded submission and backpressure. If ordered output matters, select an API with ordered-result semantics or buffer explicitly, understanding the memory and latency cost.

Related tools solve different alignment problems

  • zip: pairs values from several iterables. In current Python, strict=True can detect unequal lengths rather than silently stopping at the shortest.
  • itertools.count: supplies a configurable arithmetic counter when enumeration is not tied to one iterable.
  • dictionary iteration: yields keys (or explicit key/value pairs via .items()); the enumeration counter is not a dictionary key.
  • sequence indices: are appropriate when random access and mutation by offset are genuinely part of the algorithm.

Test the boundary cases that matter

Verify empty input, one item, a non-zero or negative start where allowed, generator exceptions, early break, invalid records, and retries. Assert that a filtered pipeline numbers the intended stage:

# Numbers only records that will be processed.
valid = (row for row in rows if validate(row))
for position, row in enumerate(valid, start=1):
    process(row)

If operators need source line numbers, enumerate before filtering and carry the line number forward. That small placement decision changes diagnostics.

Failure recovery and operational trade-offs

If a deployment used local positions as IDs, stop new writes before replaying. Determine whether records were overwritten, duplicated, or associated with the wrong item; reconcile using source identifiers and timestamps. Roll back to the previous worker only if it will not repeat non-idempotent effects. Otherwise deploy a repair that recognizes completed durable IDs first.

enumerate is cheap, readable, and streaming-friendly. The trade-off is semantic: its counter is transient. Production-grade automation becomes safer when progress numbering, source location, and business identity are three separately named fields.

Attribution and verification

This guide was prompted by Joan Venge’s Stack Overflow question and Mike Hordecki’s accepted answer, used under CC BY-SA. The accepted recommendation remains current. Iterator behavior and the start parameter were independently verified against Python’s primary enumerate documentation; multi-iterable strictness and counting alternatives were checked in the official zip documentation and itertools.count documentation.

Primary source: Review the official reference ↗