Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Getting started

This section takes a new contributor from a fresh checkout to a running development system.

This guide assumes GNU/Linux on either an x86_64 or an ARM64 (aarch64) machine. The automatic system dependency installer supports Debian and Arch Linux. Other distributions may work, but their system dependencies must be installed manually.

Read the chapters in order on a first setup:

Basic setup

Clone the repository, enter its root, and initialize the development environment:

source ./init_environment.sh

The initialization script adds scripts/bin to PATH, checks the native Rust build dependencies, and installs the pinned frontend and RedisJSON artifacts. Review each installation prompt before accepting it.

Building the system

After sourcing init_environment.sh, build the runtime modules from the repository root:

build_all_debug.sh  # builds all runtime modules
build_core.sh       # builds the gui, control, and peripheral-manager modules
build_gui_debug.sh  # builds just the gui, this includes building the gui's frontend assets
build_peripheral_debug.sh # builds just peripheral-manager, including dummy peripherals
build_control_debug.sh    # builds just the control module

Release and packaging builds are covered in Deployment.

Running the system

The complete debug system can be built and started from the repository root. To avoid a rebuild, supply any of these commands with --no-build:

run_system_debug.sh # builds all modules, then runs the system module
run_core.sh         # builds and runs the control, peripheral-manager, and GUI modules
run_gui_debug.sh
run_peripheral_debug.sh
run_control_debug.sh

Each run script starts Valkey with the Bio-C and RedisJSON modules and waits for initialization. run_system_debug.sh then launches the runtime modules selected in the System configuration. The other scripts launch the module or fixed group named by the script.

Development workflow

Use the watch scripts for short edit-build-run cycles. These scripts require watchexec, which is not installed by init_environment.sh. They use it to watch for source file changes in the bio-c directory:

watch_system.sh
watch_core.sh
watch_gui.sh
watch_control.sh
watch_peripheral.sh

Run the workspace checks before submitting a change:

run_merge_check.sh

The merge check runs the workspace tests, all Clippy lints, and cargo fmt --check. It uses an existing development Valkey instance when one is available. Otherwise, it starts one for the tests and stops it when the checks finish.

Core concepts

Bio-C brings together the monitoring and control of a bioreactor. It reads measurements from connected devices and lets operators and automated processes adjust equipment such as pumps and gas flow controllers from one system.

What Bio-C is trying to solve

A reactor setup should be able to grow with the experiment. Some existing biocontrollers restrict you to a fixed set of devices, such as a single pH probe. Bio-C is designed to support an arbitrary number of devices: adding a second probe or another pump should not mean rethinking the whole controller.

There can also be several sources asking those devices to do different things. An operator might set a pump speed at the reactor while a control loop or the BCS sends its own setpoint. Bio-C needs to sort out which request takes effect without compromising the safety of the person standing in front of the reactor. That means giving the local operator a way to take control that remote requests cannot override, while still allowing safety processes to intervene. The authority structure handles these competing requests.

That flexibility creates some technical challenges. A control loop needs to refer to the probe doing a particular job, even if someone replaces the physical probe. This is why Bio-C uses aliases for peripherals. It also needs a common way to describe what each device can measure and control, so the control logic and GUI can work with different devices without needing a separate implementation for every model. The following chapters explain how these ideas fit into the system architecture.

Crate overview

The crate map groups executable modules, supporting utilities, and drivers. Arrows show which groups shared code supports and where drivers are used.

Bio-C crate overview Shared crates: alias, config, database, json-utils, notifications, and common. These support modules, utilities, and drivers. Peripheral Manager uses the driver layer. supports Modules system control gui scada peripheral-manager Utilities sds-autoconfig redis-bioc test-structure uses Drivers peripheral stream-util Device drivers alicat · arc · bronkhorst engel · mettlertoledo · overdigit pumpower · sartorius · thermo peripheralderive Shared crates alias config database json-utils notifications common

Valkey, RedisJSON, and the data model

Valkey is the shared state and communication layer for the Bio-C modules. RedisJSON stores structured values, and the custom redis-bioc module provides operations specific to Bio-C.

Top-level structures

Most application state is stored as RedisJSON documents. Ownership identifies the module primarily responsible for initializing or maintaining a structure; other modules may still read it or update specific fields.

KeyPrimary ownerContents
bioc_idSystemThe unique identity and version information for the Bio-C.
reactorSystem and GUIReactor definitions. System creates the initial structure, while the GUI applies operator changes.
aliasSystem and GUIPer-reactor mappings from alias IDs to peripherals.
controlControl and GUIPer-reactor control loops and safety processes.
peripheralPeripheral ManagerDiscovered peripherals, their actions, measurements, and setpoints.
guiGUIPer-reactor dashboard and tile configuration.
active_statusSystem and redis-biocCurrent system and module statuses.
eventsSystem and notification producersThe notification stream consumed by the GUI.

The System module creates the shared base structures. The GUI and Peripheral Manager create their own structures when they start if those structures do not already exist.

Typed database access

The database crate wraps the Valkey connection and maps Rust types to RedisJSON operations. A JsonPath<T> combines a top-level JsonKey with a typed path into its JSON document. Types that implement ToJsonPath or ToJsonPathWithKey expose accessors for constructing these paths.

For example, ReactorMap::path() addresses the reactor document, and path_to(&reactor_id) narrows that path to one reactor. Passing the result to DatabaseClient::json_get returns the Rust type represented by that path. Writes are normally assembled as DbCommand values so related RedisJSON and redis-bioc operations can be executed together.

Prefer typed paths over hand-written RedisJSON path strings. A path constructed for the wrong Rust type then fails at compile time instead of becoming a runtime serialization error.

Initialization and reset

During normal startup, the System module connects to Valkey and initializes the Bio-C identity, notifications, reactors, aliases, control structures, and active statuses when needed. If the System configuration specifies a database bootstrap file, it loads reactor, alias, control, and GUI state from that file. The Peripheral Manager initializes peripheral, and the GUI initializes gui when it is absent.

The development run_valkey.sh script starts Valkey with RedisJSON and redis-bioc. When the database is empty, it creates the base structures before the runtime modules are launched.

Use reset_valkey.sh to run FLUSHALL and recreate the development structures. This permanently removes all data in the running development instance.

Use dump_valkey.sh to inspect its core JSON structures. The command requires jq, which is not installed by init_environment.sh and must therefore be installed separately.

System architecture

Bio-C consists of several runtime modules in a shared Cargo workspace. Each module is a separate executable with a focused responsibility. The modules share application state through Valkey instead of communicating through the System process or some other form of IPC.

System starts and supervises the module processes. Each process connects to Valkey, while SCADA and Peripheral Manager provide the external interfaces.

Bio-C system overview System starts four module processes. All five processes connect to Valkey. SCADA exposes HTTP to BCS; Peripheral Manager uses drivers to communicate with peripherals. System spawns SCADA Control GUI Peripheral Manager shared state Valkey HTTP drivers BCS Peripherals

Before introducing the various modules, this chapter first covers the shared configuration.

Configuration

Bio-C is configured through a shared TOML file. Every module reads its respective configuration parameters from this file. The only exceptions are the following three parameters, which are passed as environment variables:

  • BIO_C_REDIS_ADDR: the address used to communicate with Valkey. It defaults to redis://127.0.0.1:6379.
  • BIO_C_CONFIG: the path to the bioc-config.toml file. It defaults to ./bioc-config.toml.
  • BIO_C_BINARY_DIR: the directory in which the System module looks for the other module binaries. More on this in the next section.

System module

The System module simply manages the uptime of the other modules. When the System module is started, it will:

  • Read the System configuration, connect to Valkey, and initialize the shared database structures. If a database bootstrap file is configured, this bootstrap file is inserted into Valkey instead of the structure that is empty by default.

  • Check its PID file and Unix domain socket for an existing System process. Only one System module should be active at a time. This enforces that.

  • Dry-run every configured module. Startup stops if any module cannot initialize successfully. What exactly happens during a dry run is decided by the modules themselves.

  • Write its PID file, create its Unix domain socket, and start each configured module from BIO_C_BINARY_DIR (or the current directory when it is unset).

  • Check the child processes once per second and restart any module that has exited.

  • On SIGINT or SIGTERM, stop all managed modules and remove its PID file and Unix domain socket.

Responsibilities of this kind are usually handed off to something like systemd. Bio-C handles them itself primarily for the added dry-run functionality. This can help catch critical bugs during initial startup rather than at runtime.

If this explanation went over your head a little, that’s fine. The primary thing you need to remember is that when starting via the System module, all other modules are started by the System module. You will never need to run both the System module and another module separately.

Peripheral Manager

The Peripheral Manager handles all peripheral communication. It represents the largest chunk of complexity in Bio-C, and later chapters go into more detail on how to contribute to it.

First, some terminology:

  • A peripheral is a device connected to the Peripheral Manager that implements a common interface. The source code for the supported peripherals can be found under the drivers directory. The Peripheral Manager supports both TCP connections and USB devices. Drivers range from Modbus RTU devices to HTTP-based devices.

  • A peripheral action represents a measurement, output, or setting of a peripheral. Examples of actions include:

    • The pH measurement of a pH probe.
    • The PWM setting of a PWM-controlled digital output device.
    • The output speed of a Pump Tower pump.
    • The rotation direction of the same pump.
    • The serial number of a DO probe.
  • The peripheral structure is the JSON structure in Valkey that the Peripheral Manager reads for instructions and writes to with process values.

  • A process value is a timestamped measurement. In the context of Bio-C, this is not limited to decimal numbers; it can also be a string or boolean value.

  • A setpoint is a value set by the user and wrapped in an Authority structure. More on this later.

The Peripheral Manager spawns logical threads for reading from and writing to Valkey and for each device to which it is connected. The Peripheral Manager polls Valkey at fixed intervals to gather changed setpoints. For each connected device, it steps through a priority queue that decides which commands to send to the physical device. It also sends commands corresponding to the setpoint changes detected during Valkey polling. Whenever a device responds, the new measurement is converted into a batch of database commands and written to Valkey. Whenever the Peripheral Manager has time to spare, it scans for new devices.

Aliases

The Peripheral Manager exposes a list of peripherals keyed by their peripheral ID. This identifier is designed to match the exact physical device. Two pH probes should not share the same peripheral ID. In practice, users want to refer to the idea of a measurement rather than the measurement of a specific probe.

That is where aliases come in. The alias structure is a lookup table in Valkey that links alias IDs (UUIDs) to peripheral IDs. When other modules refer to a peripheral, they store its alias ID instead of its peripheral ID. The peripheral backing an alias can later be replaced without disrupting a control loop.

Control module

The Control module handles PID control of peripherals. The GUI can define control loops in the control structure in Valkey. These control loops contain the following information:

  • A target action. This is the measurement being controlled.
  • A setpoint for the target action. This is the value the control loop will strive for.
  • A list of actuator actions to send setpoints to when the target is below the setpoint.
  • A list of actuator actions to send setpoints to when the target is above the setpoint.
  • PID settings.
  • Whether the control loop is active.

The Control module does not communicate with peripherals directly. It uses the peripheral structure in Valkey to set setpoints and read process values.

Safety processes

Aside from control loops, the Control module has one more important responsibility: safety processes.

A safety process is similar to a control loop in that it reads a measurement and actuates a list of devices based on that measurement. It differs in its purpose. A safety process is meant to be the last line of defence against pressure buildup in the reactor or other dangerous reactor states. A safety process defines a safe range of values for a target measurement. If the measurement is outside this safe range, a list of user-defined actuators is set to a user-defined setpoint. Usually, this turns off pumps and opens release valves.

GUI

The GUI is a web application designed to run in kiosk mode on the touchscreen of a Bio-C. Tiles are the grid items on the main page of the GUI. They allow the user to read process values and write setpoints.

The GUI reads information from the Control and Peripheral Manager modules through Valkey. This means it sits at the top of the dependency tree.

When running the GUI locally, it helps to emulate the Bio-C as closely as possible. Run it in Firefox with Responsive Design Mode and touch emulation enabled. Set the resolution to 1920 x 1080.

BCS module

The BCS module, previously known as SCADA, serves as an API for the BCS. It provides a way to control Bio-C and its peripherals without directly accessing Valkey. The API is a REST API over HTTP. Its specification is in bio-c/scada/openapi.yaml.

common crates

The codebase contains several library crates:

  • alias contains the alias structure.
  • config contains the TOML configuration structure.
  • database contains Valkey-related code.
  • json-utils contains code related to JSON in Valkey.
  • notifications contains the structure used to store overall system status and user-facing log messages.
  • common contains everything else that could not easily be extracted into its own crate.

Authority

An authority value allows several sources to propose a value while exposing one effective value according to a fixed priority order. It is how we ensure that a value set by a user in front of the reactor has priority over a value set by a remote user through the BCS.

The structure roughly looks like this;

#![allow(unused)]
fn main() {
struct Authority<T> {
    safety: Option<Timestamped<T>>,
    user: Option<Timestamped<T>>,
    remote: Option<Timestamped<T>>,
    scada: Option<Timestamped<T>>,
    control: Option<Timestamped<T>>,
    default: Option<Timestamped<T>>,
}
}

It stores an optional value for each ‘source of control’ of the Bio-C.

Authority levels

The various AuthorityLevels represent these sources:

  • Safety is reserved for the safety processes of the control module. When set, it supersedes all other authority levels.
  • User represents a setpoint that a GUI user chooses to lock. Unlocked setpoints use the default field instead.
  • Remote is currently a placeholder for a user accessing the GUI remotely.
  • Scada is set whenever the BCS sends a setpoint.
  • Control is set whenever the control module issues a setpoint to one of its actuators.
  • Default is set whenever a GUI user sets a setpoint without locking it.

Reducing an authority structure

Applying the priority rules to an authority structure to get the current value of the setpoint is referred to as ‘reducing’ the authority structure. The process is best explained with some code.

#![allow(unused)]
fn main() {
// This is simplified, but it illustrates the priority rules
fn reduce(authority: Authority<f32>) -> Option<f32> {
    // The safety and user levels are simple
    // the safety level has priority over everything
    if auth.safety is set, return it's value

    // the user level has priority over everything except safety
    if auth.user is set, return it's value

    // From here on the priority is decided by the timestamp of the value
    // This means these sources can 'fight' for control. This is intended.
    return the most recent value of scada, control, or default. If any.
}
}

How locking and unlocking a setpoint in the GUI works

As previously discussed, the GUI uses two sources for it’s setpoints. The user level, and the default level. This procedure modifies the authority structure in a unique way, making it an interesting point of discussion for this section.

When locking a setpoint:

  • The GUI copies the value currently set for the default level, to the user level

When unlocking a setpoint:

  • The GUI copies the value currently set at the user level to the default level.
  • The GUI sets the user level to None/null

This has one important implication, if the following events happen in order:

  • GUI user sets value to 5.00 (unlocked)

    {
      safety: null,
      user: null,
      remote: null,
      scada: null,
      control: null,
      default: { value: 5.00, timestamp: now },
    }
    effective setpoint: 5.00 (from default level)
    
  • GUI user locks the value

    {
      safety: null,
      user: { value: 5.00, timestamp: now },
      remote: null,
      scada: null,
      control: null,
      default: { value: 5.00, timestamp: 1s ago },
    }
    effective setpoint: 5.00 (from user level)
    
  • BCS sets a value of 10.00 through the scada module

    {
      safety: null,
      user: { value: 5.00, timestamp: 1s ago },
      remote: null,
      scada: { value: 10.00, timestamp: now },
      control: null,
      default: { value: 5.00, timestamp: 2s ago },
    }
    effective setpoint: 5.00 (from user level)
    
  • GUI user unlocks the setpoint again

    {
      safety: null,
      user: null,
      remote: null,
      scada: { value: 10.00, timestamp: 1s ago },
      control: null,
      default: { value: 5.00, timestamp: now },
    }
    effective setpoint: 5.00 (from default level)
    

… then the BCS’s setpoint is effectively ignored. This is more predictable than the alternative, but it can be a confusing quirk for developers.

Overview

Drivers

This chapter will explain how to write a driver for the Bio-C. First, we discuss what a driver’s role is and how we handle multiple peripherals. This includes the peripheral and peripheral-manager crates. In short, drivers are crates that communicate with peripherals – external devices – that are used either as measurements or actuators. The PeripheralManager is a structure that represents an outgoing connection and controls one or more peripherals found on that connection using their respective drivers.

Hardware Overview

SDS means our I/O is relatively slow (avg 30ms per send_and_receive)

Modbus

The modbus protocol is a communication protocol used for serial buses where multiple devices are connected. The server (PeripheralManager in our case) is the only producer of messages, while clients (peripherals) only respond. Clients only ever send messages as a response to the server. Each client has an id, its modbus id, and each message the server sends starts with a modbus id. This means a single peripheral is addressed at once, and only the addressed peripheral will respond.

For the Bio-C, the stream-util crate contains a ModbusStream utility to more easily send and receive messages over modbus. For example on usage, check out the alicat, arc, bronkhorst and overdigit crates.

Naming scheme

The crates for the drivers in the Bio-C are per manufacturer. This is because peripherals from the same manufacturer often share the same communication protocol. As a little cheat sheet:

  • alicat: Backpressure Controller (BPC)
  • arc: pH, DO/pO2 (oxygen), Conductivity and Redox/ORP probes.
  • bronkhorst: Mass Flow Controller (MFC)
  • engel: Stirrer
  • mettler-toledo: Heavy weight scale
  • overdigit: Digital I/O (DIO) and Analog In (AI) modules
  • sartorius: Scale
  • thermo: Thermostat
  • unlock: Duo/Quattro Pump towers

Terminology

This section is a little cheat sheet on the terminology that will be used in this chapter. The terms will be discussed in more detail in later sections.

  • Peripheral: external device such as a sensor, scale or motor
  • Driver: library that contains software to communicate with a peripheral
  • Peripheral manager: structure that controls one or more drivers.
  • Action
  • Command
  • ‘Thing’: something the peripheral can measure and/or contorl.
  • Process Value (pv): the current value of a given ‘Thing’.
  • Setpoint (sp): The desired value for a given ‘Thing’.
    • The setpoint has a pv as well.

Driver API

As mentioned previously, a driver is a crate that can communicate with peripherals in order to read sensor data and/or control them. Very simply put, drivers send commands and peripherals respond to those commands. For example, the pH probe driver sends a “read ph” command, and the peripheral responds with the current pH. The drivers are then responsible for parsing this response and returning those to the peripheral manager.

Each measurement must have a pv, as we cannot effectively do anything with a measurement if we cannot read back a value. Some measurements also have a setpoint, which consists of two parts: the setpoint’s pv and the desired value from the PeripheralManager’s point of view. Let’s see a few examples.

The Hamilton Arc pH probe can have its pH measured, but not controlled. In this case, we have a pv, but no setpoint. The Engel stirrer can have its current velocity measured, but also controlled. The stirrer internally uses a control loop for its velocity. This means that if we want the stirrer at 500 RPM, it will take a little time before it actually reaches that. The pv represents the current actual velocity, the setpoint.pv is the desired value (500 rpm for example) and the setpoint.value represents the value the PeripheralManager wants to set the stirrer to. To put this on a timeline:

  1. Peripheral Manager wants stirrer at 500 rpm (pv = 0, setpoint.pv = 0, setpoint.value = 500)
  2. Stirrer driver sends command to set setpoint at 500 rpm (pv = 0, setpoint.pv = 500, setpoint.value = 500)
  3. Wait a little for the stirrer to accelerate (pv = 500, setpoint.pv = 500, setpoint.value = 500)

If the setpoint does not use an internal control loop, then the pv and the setpoint.pv represent the same value. For example, the stirrer has a setting for the acceleration speed, which is immediately set.

In the database, a peripheral is represented with a PeripheralStructure. This contains a lot of information of a peripheral and its actions.

Actions & Commands

As should be explained in a previous section, we use Valkey-JSON as our way to communicate between modules. If the GUI wants to read out a specific sensor, or set a setpoint, how do we handle that? For this we use Actions. An Action is a ‘thing’ the driver can measure and possibly control. For example, the Engel stirrer driver has a Velocity action that holds information on the current value (pv), current desired value (setpoint.pv) and it can set the current desired value (setpoint.value).

An Action is linked to one or more Commands, which represent the actual commands that are required to communicate with the peripheral. An Action serves as the public API for what a driver can do, so Actions should only be made for things that would be interesting for users to see. To use the example above, the Velocity action is linked to three commands: ReadVelocity (pv), ReadVelocitySetpoint (setpoint.pv) and SetVelocitySetpoint (setpoint.value). Each driver has a single Action enum that is linked to a single Command enum. In the stirrer’s case, the example above would be EngelStirrerAction::Velocity that is linked to EngelStirrerCommand::ReadVelocity, EngelStirrerCommand::ReadVelocitySetpoint and EngelStirrerCommand::SetVelocitySetpoint.

Each Action also has some metadata, including an id, unit, setpoint range, default value, gui display name etc. The most important of these is the id, which must be unique per driver (no two actions of one driver should have share the same id). The id is used to store the action in the database. As you may have noticed, this metadata is stored in an #[action(..)] attribute that is part of the PeripheralAction derive macro, which will be discussed later.

A Command represents a single command the driver can send to the peripheral. In most cases these correspond to the commands that can be found in the manual of a peripheral. Executing a Command involves sending a message to the peripheral, reading back the response and parsing it. To link back the parsed response to an action, the corresponding action’s id is used. To make this simpler,Commands automatically implement PeripheralCommand (thanks to PeripheralAction), which provides the to_measurement() method. This method takes in some measured value and returns a Measurement with the id of the corresponding action. For example, EngelStirrerCommand::ReadVelocity.to_measurement(2.0) returns a measurement with pv = 2.0 with EngelStirrerAction::Velocity’s id.

PeripheralDriver

In order to control multiple devices, we need all drivers to speak a common language. In our case, this is the PeripheralDriver trait. A very inspired name as you can probably tell. Every driver is required to implement the trait, and the trait serves as the API for the peripheral manager. Each driver holds a DriverData, which contains information on the serial id and device type. Furthermore, drivers should all be able to generate a unique id, the serial id is preferred for this.

As a rule of thumb, methods that take a &mut Connection argument cause the driver to communicate with the peripheral. A Connection represents, as the name probably implies, a serial or network connection. In most cases, this is either a direct serial connection or a connection to the SDS.

Drivers are identified in the PeripheralDriver::identify() method. This method takes a connection and optionally a modbus id and returns an instance of the driver if it can succesfully identify the peripheral. Identification should be done by reading the device type and compare it to a known string. If the peripheral does not support that, then one or more Commands should be tried. Identification should be relatively cheap and preferably only require 1 or 2 commands. The serial id and/or device type must be read if possible and assigned to DriverData. Finally, identify() should generate a unique id using PeripheralDriver::generate_unique_id(). See existing drivers for examples.

Initialisation happens in PeripheralDriver::init(), which is called by the PeripheralManager shortly after PeripheralDriver::identify() and returns a PeripheralStructure. This method has a default implementation that does a few important steps. First, it initialises the priority queue using PeripheralDriver::default_priority_queue(). Second, it creates a PeripheralStructure using PeripheralDriver::peripheral_structure(), which simply creats a new PeripheralStructure with all currently available ids. Lastly, the PeripheralStructure’s actions are populated (populate_actionmap()). This reads out all the pv’s and setpoint.pv’s for all Actions of the driver and writes them to the PeripheralStructure. If any other initialisation steps are required, PeripheralDriver::perform_init_actions() can be implemented. The method is called in init() right before populating the actionmap. For an example, see the engel crate.

For the Bio-C, we want measurements to be read out about once per second. Drivers are responsible for keeping track of what measurement should be read next. A PriorityQueue is used for this, which is a queue of Commands that the peripheral should read next. After a command is read, it is put back into the queue, which effectively means the driver can keep reading indefinitely.

For this we use PeripheralDriver::read(), which is periodically called by the PeripheralManager and makes the driver perform the next Command on its priority queue. The name is a little confusing, as performing the next Command is not necessarily a read operation; Setting a setpoint is also done via a Command. This method is automatically implemented for each driver.

PeripheralDriver::read() returns a Vec<Measurement>, which is a list of measurements. Each Measurement has a value and an action id, which lets the peripheral manager know where to store the measurement in the database. Measurement are for the pv and/or setpoint.pv based on its MeasurementKind. Similar to the action id, this is also automatically set by PeripheralCommand::to_measurement().

PeripheralDriver::read_status() is very similar to read(), except it always reads the current status of the peripheral if that is possible. It returns a ReadStatus (again, very inspired name), which is very similar to a Measurement. The biggest difference is that it holds a StatusMessage instead of a TimestampedValue. read_status() is periodically called by the PeripheralManager, drivers don’t have to worry about storing the status information, unless it impacts other Commands.

Setting setpoints is done with PeripheralDriver::process_setpoints(). This method takes in a map of (action_id, Setpoint) pairs and converts those into Commands, which are then pushed on the driver’s PriorityQueue with a very high priority. This ensures that the next call to PeripheralDriver::read() will execute the command for the setpoint. process_setpoints() is periodically called by the PeripheralManager with setpoints obtained from the database. The PeripheralManager filters out setpoints that have the same value as the current pv, so drivers don’t have to worry about that. Similar to read(), process_setpoints() has a default implementation.

Child peripherals

You have probably noticed that some methods in PeripheralDriver have a child suffix. This relates to our notion of child peripherals. Some peripherals contain multiple sub-units that we want to treat as separate devices. Crucially, these sub-units can not be communicated with separately. For example, the UNLOCK Quattro pump tower has four pumps that we want to treat as separate peripherals. However, we can only communicate with the pump tower, not the four pumps separately.

For this, we have devised a system of child peripherals. A child peripheral is a peripheral with a PeripheralStructure in the database, but without its own dedicated driver. Instead, the child peripheral is controlled via its parent. For example, the Quattro pump tower produces 5 PeripheralStructures, one for the tower, and four for each of the pumps (its children).

Measurement contains an optional peripheral_id field, which is used exactly for this. When the pump tower measures the pump speed, it does so for all pumps at once. It then sets the peripheral_id of each Measurement to the corresponding pump’s id. The PeripheralManager then interprets this as a measurement for that individual pump.

Setpoints for children are processed separately from the parent in PeripheralDriver::process_setpoints_children(). This takes a map of setpoints per child and the driver is then responsible for batching those setpoints into one Command. For example, the pump tower could receive setpoints for the Speed of each of its four pumps and it creates a single SetPumpSpeed 1 command that sets the speed for all four pumps at once.

When implementing a driver for a peripheral with child peripherals, be sure to override PeripheralDriver::has_children(), PeripheralDriver::children_ids() and PeripheralDriver::process_setpoints_children().

This system has a few downsides. Child peripherals require their own Action enum and a “fake” Command enum. These Commands are required for the PeripheralAction derive to work nicely, but are not actually ever executed. Furthermore, we chose to treat having children as a special case, as to not complicate drivers that do not have children. This does result in a lot of logic along the lines of do_thing_for_driver() followed by do_thing_for_driver_children(). Any improvements to this system are more than welcome.

Unique id

Drivers are stored in the database by their unique id, which is generated upon initialisation. Preferably the serial id is used, so that each peripheral can truly be uniquely identified.

For peripherals that do not support this, the address of the connection is used instead as a best effort.

PriorityQueue details

Each command has a different priority, we use frequencies to denote this. Normally we push back with commands period, but special cases exist:

  1. measurement fails: push back as if its the highest priority frequency
  2. setpoint is set: push back setpoint read asap

PeripheralAction

derive macro


  1. In reality the details differ, but the idea is the same

GUI development guide

Deployment