Test Harness Guide

The AddVantage test harness provides automated hardware-in-loop (HIL) testing for firmware validation. It flashes firmware, captures boot output, and validates correct operation.

Overview

The test harness performs these checks:

Flash firmware via J-Link
Capture UART output during boot
Parse version string and verify against expected
Parse boot telemetry and validate fields
Boot loop detection - ensure device doesn’t restart
Password prompt check - verify boot completed
CAN validation (optional) - compare CAN data to ECU telemetry

Quick Start

# Basic test
python tools/test_harness.py --hex build/addvantage3.hex

# With version verification
python tools/test_harness.py --hex build/addvantage3.hex --expected-version 3.2.7

# With CAN validation
python tools/test_harness.py --hex build/addvantage3.hex --can-validate

Command Line Options

Option	Description	Default
`--hex`	Path to firmware .hex file	Required
`--port`	Serial port	Auto-detect
`--baud`	UART baud rate	57600
`--expected-version`	Version to verify	None
`--skip-flash`	Skip flashing, just test	False
`--can-validate`	Run CAN validation	False
`--can-tolerance`	CAN comparison tolerance %	10
`--can-channel`	PCAN channel name	PCAN_USBBUS1

Test Sequence

1. Flash Firmware

Uses J-Link to program the firmware:

[TEST] Flashing firmware...
  [PASS] Flash firmware

If flashing fails, the test aborts - there’s nothing to test.

2. Wait for Boot

After flashing, waits 2 seconds for the device to reset and boot.

3. Capture UART Output

Captures serial output, waiting for 10 telemetry lines to ensure stability:

[TEST] Capturing UART output (waiting for 10 telemetry lines)...
[UART] addvantage PPG V3.2.7 250kbit
[UART] d0g0e99p0r0t0c0v0k0b250f327w1x0y100z0V3.2.7
[UART] Password required:
[UART] d0g0e0p35r0t0c60v0k0...
  [PASS] UART capture (12 lines)

4. Parse Version

Extracts and validates the firmware version:

[TEST] Parsing version...
  [PASS] Parse version (found: 3.2.7)

[TEST] Verifying version matches expected (3.2.7)...
  [PASS] Version match

5. Parse Boot Telemetry

Validates the boot telemetry string (contains e99):

[TEST] Parsing boot telemetry...
  [PASS] Parse telemetry
    Firmware version field: 327
    Version string: 3.2.7
    Fuel map: 1
    Baud rate: 250 kbit

6. Boot Loop Detection

Checks for repeated reboots (indicates watchdog issues):

[TEST] Checking for boot loop (need 10 telemetry lines, <=3 version strings)...
  [PASS] Telemetry stability (10 'd' lines received)
  [PASS] No boot loop detected (1 version string(s))

Detection Logic:

Must see at least 10 telemetry lines (proves device is running)
Must see ≤3 version strings (more indicates repeated reboots)

7. Password Prompt

Verifies the boot sequence completed:

[TEST] Checking for password prompt...
  [PASS] Password prompt present

8. CAN Validation (Optional)

If --can-validate is specified, compares CAN bus data to ECU telemetry:

[TEST] Running CAN vs Telemetry validation (tolerance: 10%)...
Serial port: /dev/cu.usbmodem55014603
CAN channel: PCAN_USBBUS1
Capture duration: 10.0s

Capturing CAN and UART data...
CAN samples: RPM=500, Torque=500, Coolant=10
Telemetry lines: 10

==============================================================================
Statistics Comparison
==============================================================================
Parameter            Source       Mean     StdDev        Min        Max      N
------------------------------------------------------------------------------
Engine RPM           CAN       1250.3        5.2     1242.0     1258.0    500
                     ECU       1251.0        4.8     1243.0     1259.0     10

Engine Torque %      CAN         55.2        2.1       51.0       59.0    500
                     ECU         55.5        1.9       52.0       58.0     10

Coolant Temp °C      CAN         85.0        0.5       84.0       86.0     10
                     ECU         85.0        0.0       85.0       85.0     10

==============================================================================
Validation Results
==============================================================================
Parameter             CAN Mean     ECU Mean       Diff     Diff %     Result
------------------------------------------------------------------------------
Engine RPM              1250.3       1251.0        0.7       0.1%       PASS
Engine Torque %           55.2         55.5        0.3       0.5%       PASS
Coolant Temp °C           85.0         85.0        0.0       0.0%       PASS

==============================================================================
VALIDATION PASSED: All parameters within 10.0% tolerance
  [PASS] CAN data matches telemetry

Test Results Summary

==================================================
Results: 8/8 tests passed

Exit code: 0 = all passed, 1 = failures

Troubleshooting

No Serial Output

[FAIL] UART capture: No output received

Causes:

Wrong serial port - use --port /dev/cu.usbmodemXXXX
Baud rate mismatch - verify 57600
TX/RX swapped

Version Mismatch

[FAIL] Version match: Expected 3.2.7, got 3.2.6

Cause: Old firmware still in flash, or wrong hex file

Boot Loop Detected

[FAIL] Boot loop check: Too many version strings (5 > 3) - device is boot looping!

Causes:

Watchdog not disabled properly (check startup_wdog.S)
Hard fault during initialization
Invalid vector table

No Password Prompt

[FAIL] Password prompt: No password prompt found - boot may not have completed

Causes:

Boot hangs before reaching main loop
Serial output corrupted
Baud rate wrong

CAN Validation Failures

[FAIL] CAN validation: Mismatch: Engine RPM (15.2%), Engine Torque % (12.1%)

Causes:

CAN bus not connected
Wrong baud rate (should be 250kbit)
ECU not transmitting (check ignition)
Tolerance too tight - try --can-tolerance 15

Automated CI/CD

The test harness can run in CI pipelines. Example GitHub Actions:

- name: Run HIL Tests
  run: |
    python tools/test_harness.py \
      --hex build/addvantage3.hex \
      --expected-version ${{ github.ref_name }}

DFU Test GUI

The DFU Test GUI (tools/dfu_test_gui.py) provides comprehensive testing for the bootloader and firmware update system.

Launch

python tools/dfu_test_gui.py

Features

Mode Detection:

Automatically detects APPLICATION vs BOOTLOADER mode on connect
Shows appropriate panels for each mode

Application Mode:

Live telemetry display (RPM, torque, diesel, gas, pressure, temp, etc.)
Configuration read/write via M2M protocol
“Trigger DFU Mode” button to enter bootloader without manual reset

Bootloader Mode:

Version info and header dump
Firmware upload with XMODEM-CRC
Erase and reset commands
Progress indication during upload

Automated Test Scenarios:

T1: Mode auto-detection
T2: Full upload cycle
T3: DFU trigger from application
T4: Corrupt header recovery
T5: More scenarios available

Workflow: Update Firmware from Application Mode

Connect to device (auto-detects APPLICATION mode)
Click “Trigger DFU Mode”
Device resets, GUI detects BOOTLOADER mode
Browse for .bin file (must be patched with patch_header.py)
Click “Upload Firmware”
Wait for upload and CRC verification
Click “[R] Reset to App” to boot new firmware

Real-time CAN Viewer

For debugging CAN issues, use the GUI viewer:

python tools/can_viewer.py

Features:

Live RPM, Torque, Temperature gauges
Time series graphs
CAN vs ECU comparison with % difference

Adding Custom Tests

Extend test_harness.py:

class TestResult:
    def add_pass(self, name: str): ...
    def add_fail(self, name: str, reason: str): ...

# Add after existing tests:
print("\n[TEST] Custom validation...")
if my_condition:
    results.add_pass("Custom test")
else:
    results.add_fail("Custom test", "Reason for failure")

Hardware Setup

Required Connections

Equipment	Connection
J-Link	SWD to target (SWDIO, SWCLK, GND, VTref)
USB-Serial	TX/RX to target UART pins
PCAN-USB	CAN_H, CAN_L to target CAN bus

J-Link Serial

The flash script uses a specific J-Link by serial number to avoid the GUI picker:

# In flash.py
JLINK_SERIAL = "50114603"

Change this if using a different J-Link.

Auto-Detect Serial Port

The test harness auto-detects serial ports, preferring:

J-Link CDC port
FTDI adapters
CP210x/CH340 adapters
Any available port

Use --port to override:

python tools/test_harness.py --hex build/addvantage3.hex --port /dev/ttyUSB0