Stop Python state leaking between function calls

A queue helper passes every unit test in isolation, yet a worker’s second job inherits tasks from the first. Restarting the process makes the symptom disappear. The leak is not a thread, cache, or database problem: the function’s default list is one object created when Python executes the def statement, and calls that omit the argument keep receiving that same object.

Reproduce the failure before changing it

def build_batch(task, tasks=[]):
    tasks.append(task)
    return tasks

print(build_batch("check disk"))
print(build_batch("restart service"))

The second result contains both tasks. That observation matters because a superficially similar leak can come from a module global, class attribute, closure, object pool, or caller-owned list. Confirm the default itself is shared:

print(build_batch.__defaults__)
first = build_batch("send report")
second = build_batch("rotate logs")
print(first is second)  # True

Python stores positional defaults in the function object’s __defaults__ tuple. This is useful diagnostic evidence, but application code should not mutate that tuple as a repair.

Why Python behaves this way

Default expressions are evaluated once, at function definition time. That rule applies to every expression, not just lists:

from datetime import datetime

def record(started_at=datetime.now()):
    return started_at

Every omitted call returns the timestamp captured when the function was defined. An immutable default such as None, 0, or a tuple cannot accumulate in-place changes, so the rule is less visible. A mutable list, dictionary, set, or application object exposes it.

The accepted Stack Overflow answer explains defaults as state attached to a first-class function object. Current Python documentation confirms the operational rule and explicitly recommends a sentinel when the caller should receive a fresh object.

Make “omitted” different from “supplied”

def build_batch(task, tasks=None):
    if tasks is None:
        tasks = []
    tasks.append(task)
    return tasks

Now each omitted call creates a list during that call. A caller that deliberately passes a list still gets that list updated:

release = ["backup"]
result = build_batch("deploy", release)
assert result is release
assert release == ["backup", "deploy"]

Use is None, not if not tasks. An empty caller-owned list is a valid supplied object; truthiness would replace it and silently break the API contract.

When None is valid business data

If callers may intentionally pass None, use a private sentinel so omitted and explicit null remain distinguishable:

_MISSING = object()

def build_batch(task, tasks=_MISSING):
    if tasks is _MISSING:
        tasks = []
    elif tasks is None:
        return None
    tasks.append(task)
    return tasks

Document that behavior. A sentinel fixes ambiguity; it does not make a confusing contract self-explanatory.

Choose whether to mutate caller input

The previous fix preserves the original function’s in-place behavior. That may itself be unsafe. A pure alternative copies the input:

def build_batch(task, tasks=None):
    batch = [] if tasks is None else list(tasks)
    batch.append(task)
    return batch

This prevents the function from changing a list owned by its caller, but it is a behavior change. It also performs only a shallow copy: nested mutable objects remain shared. State the ownership policy explicitly rather than treating “copy” as universally safer.

Audit the real call graph

1. Search definitions for mutable defaults: lists, dictionaries, sets, constructors, and helper calls.
2. Find every caller and separate omitted, positional, and keyword arguments.
3. Check whether any caller relies on accumulation or identity.
4. Inspect decorators and wrappers; the public signature may not be where the default lives.
5. Review async tasks and long-lived workers, where shared state survives far longer than a request.

python -m ruff check --select B006 .
python -m pytest -q

Static analysis can locate common cases, but it cannot decide whether persistence was intentional.

Prove isolation and ownership with tests

def test_omitted_batches_are_isolated():
    first = build_batch("a")
    second = build_batch("b")
    assert first == ["a"]
    assert second == ["b"]
    assert first is not second

def test_supplied_list_contract():
    supplied = []
    result = build_batch("a", supplied)
    assert result is supplied
    assert supplied == ["a"]

Add a regression that runs calls in the same process. A test suite that creates a fresh interpreter for every case can conceal process-lifetime leaks. For a web or queue worker, exercise two sequential requests or jobs against one worker.

Failure modes after the obvious fix

• Nested sharing: copying the outer list does not copy dictionaries inside it.
• Concurrency: removing the default does not make a caller-supplied shared list thread-safe.
• Serialization: changing None semantics may alter API payloads.
• Introspection: frameworks may inspect defaults to generate schemas; a private sentinel may need an adapter.
• Performance: copying a very large collection on every call trades isolation for memory and CPU.

Roll out and recover safely

Deploy the regression test first. If the function is public, release the ownership change separately from the mutable-default repair. Monitor batch sizes, duplicate work, queue depth, memory, and error rates. Do not “clean” already persisted records automatically: determine whether they are duplicate side effects, legitimate accumulated state, or both.

Rollback is a code rollback only if the old behavior produced no durable effects. If it sent notifications, created records, or executed jobs, preserve evidence, identify affected process lifetimes and idempotency keys, and reconcile those effects before retrying. Reintroducing the shared default may restore compatibility but also restores the defect, so prefer a feature flag or compatibility wrapper while callers migrate.

Use intentional persistent state openly

Sometimes persistence is the requirement. Put it in an object with a name and lifecycle:

class BatchBuilder:
    def __init__(self):
        self.tasks = []

    def add(self, task):
        self.tasks.append(task)
        return list(self.tasks)

Now tests can create a fresh builder, production code can scope it to a request or worker, and operators can reason about when the state is discarded.

The safe fix is not merely replacing [] with None. It is deciding who owns the collection, how long it should live, and how that contract is verified.

Sources: the Stack Overflow question by Stefano Borini and accepted answer by rob (CC BY-SA), and the current Python documentation for default argument values and function definitions.