Environment management

This document explains in detail how environments are managed in an installation of OrangeQS Juice. An environment is the Python installation together with all installed packages. OrangeQS Juice uses different tools to manage environments in different contexts. There are two main types of places where a Python environment is required:

• On the host system: Host environment.

  • Used by the Juice Orchestrator.

• Inside a container: Containerized environments.

  • Used by OrangeQS Juice services.

  • Used by JupyterHub user containers.

A graphical overview of the environment management is shown below. This example uses the system package and a container based on a uv environment. Note that while this example shows a single environment image, there can be multiple images each containing their own environment.

Host environment

The host environment runs directly on the host system of the OrangeQS Juice installation. The main goal of this environment is to provide access to the Juice CLI. One is free to manage this environment manually, as long as the juice command is present in the $PATH. This environment should contain the orangeqs-juice-core package, along with its dependencies.

System package

The OrangeQS Juice system package provides a managed environment for the Juice CLI. The environment is an isolated virtual environment built using virtualenv-tools3 This environment uses the Python binary from the system package manager. For more information see Package contents.

Containerized environments

OrangeQS Juice services and JupyterHub user containers run in isolated containers with a Python environment built into the image. These images are built directly on the host machine. In principle the image can use any environment manager, e.g. micromamba, virtualenv, uv. As of now, Juice implements only environments based on uv.

An installation can contain an arbitrary number of environments. Every container that is started within OrangeQS Juice is able to use a different environment. By default there are two configured environments:

• default: Contains orangeqs-juice-core and an editable install of the lab repository. This environment is used by default for OrangeQS Juice services.

• singleuser: Contains everyting from the default environment as well as jupyterhub and jupyterlab. This environment is used by default for JupyterHub user containers.

It’s possible to define a new environment in OrchestrationSettings.environments and configure a service to use this environment in ServiceSettings.environment.

Building the environment

The uv environment build process consists of multiple steps. These steps are specific for a uv environment. They can be adapted for any other environment manager. Note that these steps are meant to illustrate what the code does, they are not meant to be followed manually.

1. Read the environment configuration from the Juice configuration.

2. Determine an installation source for the orangeqs-juice-core python package. There are two possibilities:

   • It’s bundled in the system package as orangeqs_juice_core.whl.

   • Juice is installed as editable install, so we determine the source directory. We only recommend using an editable Juice Install on the host system if you are actively developing Juice and you know what you are doing, as this could lead to undefined behaviour.

3. Create a folder on the host system specific for this environment: /var/juice/envs/<name>. In this folder:

   • Generate a /var/juice/envs/<name>/pyproject.toml with the required dependencies.

   • Generate a /var/juice/envs/<name>/Containerfile that defines the image.

4. Write a podman systemd build unit for the build process. Several directories need to be mounted:

   • Mount any editable packages in the container under /editable.

   • Mount any wheels in the container under /dist.

5. Reload the system daemon such that the service is loaded.

6. Start the oneshot systemd service juice-<name>-build.service, which builds the image. It has RemainAfterExit=yes, which means that it is only rebuilt if you restart this unit. The command inside the container does the following:

   • Install the required python version under /env/.python.

   • Install a uv environment under /env/.venv.

7. The build service is specified as a dependency for all OrangeQS Juice service containers that use this environment. This means two things:

   • If juice-<name>-build is not active, it will build the container before starting the service container.

   • If juice-<name>-build is restarted (rebuilt), it will restart all service containers using this environment.

Limitations

• We do not yet use uv.lock to lock the environment. This means environments are not 100% reproducible.

• Rebuilding the environment automatically restarts all services that use that environment.

• We do not use the uv cache. We should mount a cache directory for uv to speed up the build process, especially with multiple environments.