Installation¶

Install BenchCI on the machines that need to run the CLI:

your development machine
a hardware-connected lab machine
a CI runner
a self-hosted runner

For local tests, the machine running BenchCI must be connected to the hardware.

For Cloud Mode, CI only needs the CLI. The hardware-connected Agent executes the test against the connected device.

Install BenchCI¶

BenchCI requires Python 3.11 or newer.

pip install benchci

Verify the CLI:

benchci --help

Upgrade:

pip install --upgrade benchci

Log in¶

BenchCI access is tied to a BenchCI account and workspace.

benchci login

Check your active account and workspace:

benchci whoami

Log out:

benchci logout

Create or access your workspace from:

https://app.benchci.dev

New accounts start a 14-day free trial after email verification. For commercial usage after the trial, contact tech@benchci.dev.

Verify your environment¶

Run diagnostics:

benchci doctor

benchci doctor automatically inspects available serial ports, USB devices, GPIO chips, Python modules, and common flashing tools:

benchci doctor
benchci doctor --json
benchci doctor --export doctor-report.zip

Check a specific bench file:

benchci doctor --bench bench.yaml

Check an Agent:

benchci doctor --agent http://192.168.1.50:8080

If the Agent requires authentication:

benchci doctor --agent http://192.168.1.50:8080 --token "$BENCHCI_AGENT_TOKEN"

Install on a hardware-connected machine¶

This is the machine physically connected to the DUT, debugger, UART/CAN/Modbus adapters, GPIO lines, relays, or power control hardware.

pip install benchci

For direct Agent mode:

benchci agent serve

To protect Agent endpoints:

export BENCHCI_AGENT_TOKEN=secure-token
benchci agent serve

For Cloud Mode:

benchci agent cloud \
  --token YOUR_AGENT_TOKEN \
  --bench bench.yaml \
  --bench-id my-bench

Install on CI runner machines¶

A CI runner usually does not need direct hardware access when using Cloud Mode.

pip install benchci

Cloud Mode run:

benchci login

benchci run --cloud \
  --bench-id my-cloud-bench \
  --suite suite.yaml \
  --artifact build/fw.elf

Direct Agent run:

benchci run \
  --agent "$BENCHCI_AGENT_URL" \
  --bench bench.yaml \
  --suite suite.yaml \
  --artifact build/fw.elf \
  --token "$BENCHCI_AGENT_TOKEN"

Registered-bench direct Agent run:

benchci run \
  --agent "$BENCHCI_AGENT_URL" \
  --bench-id my-bench \
  --suite suite.yaml \
  --artifact build/fw.elf \
  --token "$BENCHCI_AGENT_TOKEN"

External tools¶

BenchCI is installed through Python, but your bench may require external system tools depending on what you automate. Python backend libraries are installed with the BenchCI package; this section covers tools, OS access, and device permissions outside the package.

Common tools include:

OpenOCD for many STM32 and ARM workflows
STM32CubeProgrammer for STM32 workflows
SEGGER J-Link tools for J-Link workflows
esptool for ESP32 workflows
Linux GPIO access through /dev/gpiochipX
CAN tools and SocketCAN configuration for CAN workflows
serial device permissions for UART workflows

Example Ubuntu packages for a basic STM32/OpenOCD workflow:

sudo apt-get update
sudo apt-get install -y openocd gcc-arm-none-eabi make

For GPIO, make sure the user running BenchCI can access /dev/gpiochipX.

For serial, make sure the user can access /dev/ttyUSBX or /dev/ttyACMX.

Next step¶

Continue with:

Quickstart

After installation: verify BenchCI readiness¶

After installing or upgrading BenchCI, use the readiness workflow on the hardware-connected machine:

benchci doctor
benchci validate --bench bench.yaml --suite suite.yaml
benchci benches self-test --bench bench.yaml --open-hardware --log-dir bench-health
benchci run --bench bench.yaml --suite suite.yaml --artifact build/fw.elf --dry-run-plan

This confirms that the machine, YAML files, and physical bench are ready before a real run.