Skip to content

Usage with Cylc workflow manager and Slurm

The command line interface can be used with cylc in environments which use a scheduler like slurm.


This example requires:

  • slurm, e.g. on a high performance computing cluster.
  • familiarity with and a working installation of cylc (e.g. by going through the tutorial)
  • virtualenv
  • python (so you can run virtualenv venv -p python)

A new environment will be created during the setup phase of the cylc workflow run.

Cylc requires a site-specific setup when using a scheduler like slurm. See the cylc documentation for a guide on setting up cylc on your platform. For Oscar at Brown University, we can use the following configuration in ./global.cylc

        hosts = localhost
        install target = localhost
        job runner = slurm
        retrieve job logs = True
        global init-script = """
            module load python/3.9.0


To initialize the workflow, we define a file in thelib/python directory (a cylc convention) with the code for the experiment: lib/python/, including all the required functions.

import pandas as pd
from sklearn.linear_model import LinearRegression

from autora.experimentalist.grid import grid_pool
from autora.state import StandardState, estimator_on_state, on_state
from autora.variable import Variable, VariableCollection

def initial_state(_):
    state = StandardState(
            independent_variables=[Variable(name="x", allowed_values=range(100))],
        experiment_data=pd.DataFrame({"x": [], "y": []}),
    return state

experimentalist = on_state(grid_pool, output=["conditions"])

experiment_runner = on_state(
    lambda conditions: conditions.assign(y=2 * conditions["x"] + 0.5),

theorist = estimator_on_state(LinearRegression(fit_intercept=True))

These functions will be called in turn by the autora.workflow script.

The flow.cylc file defines the workflow.

    cycling mode = integer
    initial cycle point = 0
    final cycle point = 5
        R1/0 = """
        setup_python => initial_state
        R1/1 = """
            initial_state[^] => experimentalist => experiment_runner => theorist
        2/P1 = """
            theorist[-P1] => experimentalist => experiment_runner => theorist

        script = """
            virtualenv "$CYLC_WORKFLOW_SHARE_DIR/env" -p python
            source "$CYLC_WORKFLOW_SHARE_DIR/env/bin/activate"
            pip install --upgrade pip
            pip install -r "$CYLC_WORKFLOW_RUN_DIR/requirements.txt"
        platform = oscar
        execution time limit = PT20M

    script = """
            $CYLC_WORKFLOW_SHARE_DIR/env/bin/python \
                -m autora.workflow \
                runner.initial_state \
                --out-path "$CYLC_WORKFLOW_SHARE_DIR/$CYLC_TASK_CYCLE_POINT/result"
        platform = oscar

        script = """
            $CYLC_WORKFLOW_SHARE_DIR/env/bin/python -m autora.workflow \
                runner.experimentalist \
                --in-path "$CYLC_WORKFLOW_SHARE_DIR/$((CYLC_TASK_CYCLE_POINT - 1))/result" \
                --out-path "$CYLC_WORKFLOW_SHARE_DIR/$CYLC_TASK_CYCLE_POINT/conditions"
        platform = oscar

        script = """
            $CYLC_WORKFLOW_SHARE_DIR/env/bin/python -m autora.workflow \
                runner.experiment_runner \
                --in-path "$CYLC_WORKFLOW_SHARE_DIR/$CYLC_TASK_CYCLE_POINT/conditions" \
                --out-path "$CYLC_WORKFLOW_SHARE_DIR/$CYLC_TASK_CYCLE_POINT/data"
        platform = oscar

        script = """
            $CYLC_WORKFLOW_SHARE_DIR/env/bin/python -m autora.workflow \
                runner.theorist \
                --in-path "$CYLC_WORKFLOW_SHARE_DIR/$CYLC_TASK_CYCLE_POINT/data" \
                --out-path "$CYLC_WORKFLOW_SHARE_DIR/$CYLC_TASK_CYCLE_POINT/result"
        platform = oscar
        execution time limit = PT1H
            --partition = gpu
            --gres = gpu:1


We can call the cylc command line interface as follows, in a shell session:

Validate, install and play the flow: First, we validate the flow.cylc file:

cylc vip .

We can view the workflow running in the graphical user interface (GUI):

cylc gui


We can load and interrogate the resulting object in Python as follows:

from autora.serializer import load_state

state = load_state("~/cylc-run/cylc-slurm-pip/runN/share/result")