Docker 101 : An overview of Docker Storage

Docker file structure on the host:

After we install docker, it creates the below folder structure in the host directory "/var/lib/docker".

For example, the files related to containers are stored in the containers folder and the volumes created by docker are under the volumes folder.

Docker images layered architecture:

Docker build images in layers, each layer corresponds to a line in the dockerfile.

Example of a Dockerfile:

Each layer only stores the changes from the previous layer, similar elements are not stored.

If a layer already exist on the host, docker retrieves it from the cache so it is not built from scratch.

The layers are created when we run the "docker build" command, these layers form the final image docker uses as a base for its containers on.
Container are just running images.

The layers are read-only. When we execute the "run" command to run a container (based on an image), docker adds a new
writable layer on top of the read-only ones. (created with "docker build" command)

The writable layer lives as long as the container is running.

The read-only layers can be shared between multiple containers. When we modify a file for example within our container, docker creates a copy of this file in the writable layer of the image, so we can change it.

All modification to the file will be done in the writable layer.

This process is called a C.O.W system or Copy On Write.

When the container crashes or exits, the writable layer as well as our modified file get removed.

Creating persistent storage:

We could add a persistent storage to the container, using:

It creates a volume_1 folder under the "/var/lib/docker/volumes" directory and mounts it on the docker writable layer at "/var/lib/python".

If we create a file in "/var/lib/python" it will also be created in the folder "/var/lib/docker/volumes/volume_1" hierarchy on the host.

We call this kind of persistent storage in the default location "/var/lib/docker/volumes" a volume mount.

Bind mounts (storing data on other locations):

If we don't want to store data in the default folder "/var/lib/docker/volumes", we need to provide the full path of the new location.

This is called a "bind mount" which is the more "explicit" way of mounting volumes:

We use the "--mount" option instead of the "-v" as we can see below:

We could also use the "type=volume".

Remark:

The component that handles all this layered architecture is called the storage driver.

Below are some examples of a storage driver:

AUFS
overlay2
BTRFS
Device Mapper
....

The below command displays the storage driver, the containers (Stopped, Running and Paused), the images and other information about docker and the system we are running it on.