Containers and CernVM-FS¶
CernVM-FS can also be used to distribute container images, providing many of the same benefits that come with any CernVM-FS installation. Especially the on-demand download of accessed files means that containers start nearly instantly, and are more efficient for large images when only a fraction of the files are read, which is typically the case.
Any CernVM-FS repository can be used to distribute container images (although often, dedicated repositories are used, like /cvmfs/unpacked.cern.ch
).
In order to provide de-duplication and on-demand download, images must be stored unpacked. This requires some dedicated tools, provided by CernVM-FS itself - see the section "Ingesting container images" below.
Accessing containers hosted in CernVM-FS via Apptainer¶
Apptainer is the recommended way to run containers from CernVM-FS, as it can start a container directly from an unpacked root file system, which is ideal for CernVM-FS.
Docker can be used as well but the setup is more complicated, requiring the CernVM-FS graphdriver plugin.
For example, to run the tensorflow/tensorflow:2.15.0-jupyter
image from Docker Hub that has been unpacked in /cvmfs/unpacked.cern.ch
, use the following commands:
container="registry.hub.docker.com/tensorflow/tensorflow:2.15.0-jupyter"
python_code="import tensorflow as tf; print(tf.__version__)"
apptainer run /cvmfs/unpacked.cern.ch/${container} python -c "${python_code}"
This directory just contains the root file system of the image:
ls /cvmfs/unpacked.cern.ch/registry.hub.docker.com/tensorflow/tensorflow:2.15.0-jupyter
Ingesting container images¶
CernVM-FS provides a suite of container unpacking tools called cvmfs_ducc
(provided by the cvmfs-ducc
package). This can be used to unpack and ingest container images by simply running
cvmfs_ducc convert recipe.yaml
recipe.yaml
is a 'wishlist' of container images available in external registries that should be made available:
version: 1
user: cvmfsunpacker
cvmfs_repo: 'unpacked.repo.tld'
input:
- 'https://registry.hub.docker.com/tensorflow/tensorflow:2.15.0-jupyter'
...
For more information, see the CernVM-FS documentation.
Using /cvmfs
inside containers¶
The easiest way to access CernVM-FS repositories from a container is to set it up on the host and bind-mount it inside the container:
docker run -it --volume /cvmfs:/cvmfs:shared ubuntu ls -lna /cvmfs/atlas.cern.ch
For Apptainer, the same can be done by setting the $SINGULARITY_BIND
(or $APPTAINER_BIND
) environment variable:
export SINGULARITY_BIND="/cvmfs"
(next: Creating a CernVM-FS repository)