BenchCI Documentation

Run real hardware tests from CI

BenchCI lets embedded teams flash firmware, run tests on real devices, and return logs/results to CI automatically.

benchci run --cloud --bench-id my-bench --suite suite.yaml --artifact build/fw.elf

No simulation. No manual serial-terminal testing. Real hardware in the loop.

The problem BenchCI solves

Most embedded validation still depends on manual bench work:

  • flash firmware by hand

  • open a terminal

  • send commands manually

  • watch logs

  • copy results into a ticket or release note

That is fine for early bring-up.

It is not enough for repeatable CI, shared hardware, pull-request validation, release gates, or remote teams.

What BenchCI gives you

BenchCI turns hardware validation into a repeatable CI workflow:

CI builds firmware
        ↓
BenchCI schedules a hardware run
        ↓
Agent flashes and tests a real device
        ↓
Results, logs, and artifacts return to CI/dashboard

You can start locally, then move the same test model to a remote Agent or BenchCI Cloud.

Fastest path

If you already have a board connected to your machine:

pip install benchci
benchci login
benchci run -b bench.yaml -s suite.yaml -a build/fw.elf

BenchCI will load your hardware configuration, flash the firmware, run the suite, and write results under benchci-results/.

Core mental model

bench.yaml  -> describes the hardware
suite.yaml  -> describes the test logic
benchci run -> executes the suite on real hardware

A bench is your physical setup: DUT, debugger, UART/CAN/Modbus adapters, GPIO, relays, and related resources.

A suite is what should happen: flash, reset, send commands, wait for logs, read registers, check GPIO, validate CAN frames, or cycle power.

An Agent is the hardware-connected process that lets CI or remote users run tests without sitting next to the device.

BenchCI Cloud Mode adds workspace-aware scheduling, shared/private benches, dashboard visibility, and CI-friendly execution.

What BenchCI supports today

Communication:

  • UART

  • Modbus RTU

  • Modbus TCP

  • CAN

Flashing:

  • OpenOCD

  • STM32CubeProgrammer

  • SEGGER J-Link

  • esptool

Control:

  • Linux GPIO

  • remote GPIO through Agent

  • mock GPIO

  • relay-backed power workflows

Execution:

  • local runs

  • direct Agent runs

  • backend-controlled Cloud Mode runs

Documentation