Choose containers or virtual machines by the isolation boundary

“Containers are lightweight VMs” is a useful first analogy and a dangerous production model. A virtual machine normally boots a guest operating system with its own kernel behind virtual hardware. A Linux container is an isolated process tree that uses the host kernel. That distinction explains the speed and density advantage—and why the two are not interchangeable security or compatibility boundaries.

Correct the historical explanation

The accepted Stack Overflow answer correctly emphasized shared host resources, process isolation, image layers, and lower overhead. Some implementation detail is now obsolete: Docker is not defined by LXC or AUFS. Current Docker documentation describes containers as isolated processes, Docker’s Linux isolation as using kernel features such as namespaces, and images as immutable packages composed of filesystem-change layers.

Boundary	Container	Virtual machine
Kernel	Shares the host kernel	Guest OS normally has its own kernel
Packaged unit	Application files, libraries, configuration and image metadata	Guest OS plus application stack
Startup/density	Usually fast and high-density	Usually more overhead and slower to boot
Isolation failure	Host-kernel boundary is central	Hypervisor/virtual-hardware boundary is central
Typical use	Portable application delivery and scaling	Kernel/OS separation, legacy systems, stronger tenancy boundary

These are architectural tendencies, not performance guarantees. Desktop container products may themselves run Linux containers inside a VM. Cloud platforms commonly run a container runtime inside VMs. “Container versus VM” is often a question of layering them, not selecting exactly one.

Diagnose the workload before choosing

Start with constraints that cannot be negotiated:

• Does the workload require a different kernel, kernel module, hardware driver, or operating-system family?
• Is the trust boundary between mutually hostile tenants, between internal services, or merely between application dependencies?
• What compliance evidence is required for isolation, patching, image provenance, and administrator access?
• Does the workload need stable device passthrough, nested virtualization, a full init system, or legacy boot behavior?
• What are the measured CPU, memory, storage, startup, and network requirements?
• How will persistent data, secrets, backups, logs, health checks, and upgrades work?

If an application requires its own kernel or a Windows guest on a Linux host, a conventional container cannot supply that boundary. If the requirement is repeatable packaging of a stateless service on compatible hosts, a container is often the smaller operational unit. If untrusted workloads share infrastructure, evaluate a stronger sandbox or VM boundary instead of assuming a default container profile is equivalent to a VM.

Build a representative container safely

Pin a small trusted base by immutable digest, use a multi-stage build where useful, run as a non-root user, and keep secrets out of image layers and build arguments. Treat images as immutable releases; rebuild rather than patching a running container.

FROM example/runtime@sha256:APPROVED_DIGEST
WORKDIR /app
COPY --chown=10001:10001 ./dist/ ./
USER 10001:10001
ENTRYPOINT ["/app/service"]

The names are illustrative; select a real maintained base through your organization’s provenance and vulnerability policy. A digest gives reproducibility, not permanent safety. Continuously rebuild and rescan when the base or dependencies receive fixes.

At runtime, grant only necessary capabilities, use read-only filesystems where compatible, constrain memory and CPU, configure seccomp/AppArmor/SELinux policy as the platform supports, avoid the host network and privileged mode, and mount only required paths. Never expose the container-runtime socket to an ordinary application: control of that socket is commonly equivalent to control of the host.

Make state and failure explicit

Docker’s current documentation notes that changes in a container’s writable layer disappear when the container is removed unless placed in persistent storage. Keep durable data in managed volumes or external services with independent backup and restore procedures. Do not turn a running container into the source of truth by committing its changed filesystem.

Define:

• startup, readiness, liveness, graceful shutdown, and maximum termination time;
• resource requests/limits based on load tests, including behavior at memory exhaustion;
• immutable release identifiers and a signed/provenanced promotion path;
• secret injection and rotation without baking secrets into layers;
• log and metric export that survives container replacement;
• volume backup consistency, restore verification, and recovery objectives.

Run a fair comparison

Benchmark the same application and workload on representative hardware. Warm and cold startup, steady-state throughput, tail latency, memory working set, storage I/O, network overhead, noisy-neighbor behavior, patch time, and recovery time all matter. Include the control plane and host OS in operational cost; counting only application containers hides real overhead.

Exercise failure, not only success: terminate the process, exhaust its memory limit, fill writable storage, revoke a secret, remove a node, interrupt the registry, restore a volume, and roll back an image. Repeat equivalent tests for the VM design, including guest boot, snapshot restore, hypervisor maintenance, and guest-agent failure.

Failure modes that invalidate the design

• Kernel incompatibility: the image has user-space files but cannot bring a different host kernel. Choose a compatible host or VM.
• Privileged escape surface: privileged mode, broad capabilities, host PID/network namespaces, device access, or runtime socket mounts erase intended isolation. Remove the grant or strengthen the boundary.
• Mutable snowflake: operators install fixes inside a running container. Rebuild, test, and redeploy an immutable image.
• Lost state: data lived in the writable layer. Restore from the authoritative volume/service backup and correct storage architecture.
• Unbounded resources: one workload harms its neighbors. Add measured limits and admission/capacity controls; limits are not a substitute for capacity.
• Tag drift: a mutable tag deploys different bytes. Promote and record digests, then verify the running image identity.
• False portability: CPU architecture, kernel features, devices, filesystem behavior, or external services differ. Test the actual target platform.

Rollback and recovery are different operations

Application rollback means redeploying the last approved image digest and compatible configuration. It does not automatically reverse a database migration. Use expand/contract migrations, forward-compatible readers, and separate database recovery procedures. Preserve the previous deployment specification until acceptance checks pass.

If the container runtime or host kernel is implicated, drain workloads, preserve evidence, replace or patch the node through the infrastructure process, and redeploy clean images. Do not “repair” a suspected compromised container in place. For a VM, revert only when snapshot consistency and downstream state are understood; restoring an old disk while external databases continue forward can create a worse incident.

A defensible decision

Choose containers for compatible, image-packaged workloads when fast replacement and density help and the shared-kernel trust model is acceptable. Choose VMs when the workload needs another kernel/OS, legacy machine semantics, or a stronger infrastructure boundary. Combine them when VMs provide tenant or node isolation while containers provide application packaging and scheduling.

Record the trust model, kernel ownership, persistent-state design, resource envelope, hardening controls, benchmark evidence, patch responsibilities, and tested recovery path. The winning platform is the one the team can secure, observe, update, and restore—not the one with the shortest startup demo.

Attribution and current verification

This guide was prompted by zslayton’s Stack Overflow question and Ken Cochrane’s accepted answer, used under CC BY-SA. The core shared-kernel and density explanation remains useful, while the historical LXC/AUFS specifics were corrected. Current behavior was independently verified against Docker’s official guides to containers and virtual machines, image layers, and the current Docker architecture overview.