# Test Harness Guide

The AddVantage test harness provides automated hardware-in-loop (HIL) testing for firmware validation. It flashes firmware, captures boot output, and validates correct operation.

## Overview

The test harness performs these checks:

1. **Flash firmware** via J-Link
2. **Capture UART output** during boot
3. **Parse version string** and verify against expected
4. **Parse boot telemetry** and validate fields
5. **Boot loop detection** - ensure device doesn't restart
6. **Password prompt check** - verify boot completed
7. **CAN validation** (optional) - compare CAN data to ECU telemetry

## Quick Start

```bash
# Basic test
python tools/test_harness.py --hex build/addvantage3.hex

# With version verification
python tools/test_harness.py --hex build/addvantage3.hex --expected-version 3.2.7

# With CAN validation
python tools/test_harness.py --hex build/addvantage3.hex --can-validate
```

## Command Line Options

| Option | Description | Default |
|--------|-------------|---------|
| `--hex` | Path to firmware .hex file | Required |
| `--port` | Serial port | Auto-detect |
| `--baud` | UART baud rate | 57600 |
| `--expected-version` | Version to verify | None |
| `--skip-flash` | Skip flashing, just test | False |
| `--can-validate` | Run CAN validation | False |
| `--can-tolerance` | CAN comparison tolerance % | 10 |
| `--can-channel` | PCAN channel name | PCAN_USBBUS1 |

## Test Sequence

### 1. Flash Firmware

Uses J-Link to program the firmware:

```
[TEST] Flashing firmware...
  [PASS] Flash firmware
```

If flashing fails, the test aborts - there's nothing to test.

### 2. Wait for Boot

After flashing, waits 2 seconds for the device to reset and boot.

### 3. Capture UART Output

Captures serial output, waiting for 10 telemetry lines to ensure stability:

```
[TEST] Capturing UART output (waiting for 10 telemetry lines)...
[UART] addvantage PPG V3.2.7 250kbit
[UART] d0g0e99p0r0t0c0v0k0b250f327w1x0y100z0V3.2.7
[UART] Password required:
[UART] d0g0e0p35r0t0c60v0k0...
  [PASS] UART capture (12 lines)
```

### 4. Parse Version

Extracts and validates the firmware version:

```
[TEST] Parsing version...
  [PASS] Parse version (found: 3.2.7)

[TEST] Verifying version matches expected (3.2.7)...
  [PASS] Version match
```

### 5. Parse Boot Telemetry

Validates the boot telemetry string (contains `e99`):

```
[TEST] Parsing boot telemetry...
  [PASS] Parse telemetry
    Firmware version field: 327
    Version string: 3.2.7
    Fuel map: 1
    Baud rate: 250 kbit
```

### 6. Boot Loop Detection

Checks for repeated reboots (indicates watchdog issues):

```
[TEST] Checking for boot loop (need 10 telemetry lines, <=3 version strings)...
  [PASS] Telemetry stability (10 'd' lines received)
  [PASS] No boot loop detected (1 version string(s))
```

**Detection Logic:**
- Must see at least 10 telemetry lines (proves device is running)
- Must see ≤3 version strings (more indicates repeated reboots)

### 7. Password Prompt

Verifies the boot sequence completed:

```
[TEST] Checking for password prompt...
  [PASS] Password prompt present
```

### 8. CAN Validation (Optional)

If `--can-validate` is specified, compares CAN bus data to ECU telemetry:

```
[TEST] Running CAN vs Telemetry validation (tolerance: 10%)...
Serial port: /dev/cu.usbmodem55014603
CAN channel: PCAN_USBBUS1
Capture duration: 10.0s

Capturing CAN and UART data...
CAN samples: RPM=500, Torque=500, Coolant=10
Telemetry lines: 10

==============================================================================
Statistics Comparison
==============================================================================
Parameter            Source       Mean     StdDev        Min        Max      N
------------------------------------------------------------------------------
Engine RPM           CAN       1250.3        5.2     1242.0     1258.0    500
                     ECU       1251.0        4.8     1243.0     1259.0     10

Engine Torque %      CAN         55.2        2.1       51.0       59.0    500
                     ECU         55.5        1.9       52.0       58.0     10

Coolant Temp °C      CAN         85.0        0.5       84.0       86.0     10
                     ECU         85.0        0.0       85.0       85.0     10

==============================================================================
Validation Results
==============================================================================
Parameter             CAN Mean     ECU Mean       Diff     Diff %     Result
------------------------------------------------------------------------------
Engine RPM              1250.3       1251.0        0.7       0.1%       PASS
Engine Torque %           55.2         55.5        0.3       0.5%       PASS
Coolant Temp °C           85.0         85.0        0.0       0.0%       PASS

==============================================================================
VALIDATION PASSED: All parameters within 10.0% tolerance
  [PASS] CAN data matches telemetry
```

## Test Results Summary

```
==================================================
Results: 8/8 tests passed
```

Exit code: 0 = all passed, 1 = failures

## Troubleshooting

### No Serial Output

```
[FAIL] UART capture: No output received
```

**Causes:**
- Wrong serial port - use `--port /dev/cu.usbmodemXXXX`
- Baud rate mismatch - verify 57600
- TX/RX swapped

### Version Mismatch

```
[FAIL] Version match: Expected 3.2.7, got 3.2.6
```

**Cause:** Old firmware still in flash, or wrong hex file

### Boot Loop Detected

```
[FAIL] Boot loop check: Too many version strings (5 > 3) - device is boot looping!
```

**Causes:**
- Watchdog not disabled properly (check `startup_wdog.S`)
- Hard fault during initialization
- Invalid vector table

### No Password Prompt

```
[FAIL] Password prompt: No password prompt found - boot may not have completed
```

**Causes:**
- Boot hangs before reaching main loop
- Serial output corrupted
- Baud rate wrong

### CAN Validation Failures

```
[FAIL] CAN validation: Mismatch: Engine RPM (15.2%), Engine Torque % (12.1%)
```

**Causes:**
- CAN bus not connected
- Wrong baud rate (should be 250kbit)
- ECU not transmitting (check ignition)
- Tolerance too tight - try `--can-tolerance 15`

## Automated CI/CD

The test harness can run in CI pipelines. Example GitHub Actions:

```yaml
- name: Run HIL Tests
  run: |
    python tools/test_harness.py \
      --hex build/addvantage3.hex \
      --expected-version ${{ github.ref_name }}
```

## DFU Test GUI

The DFU Test GUI (`tools/dfu_test_gui.py`) provides comprehensive testing for the bootloader and firmware update system.

### Launch
```bash
python tools/dfu_test_gui.py
```

### Features

**Mode Detection:**
- Automatically detects APPLICATION vs BOOTLOADER mode on connect
- Shows appropriate panels for each mode

**Application Mode:**
- Live telemetry display (RPM, torque, diesel, gas, pressure, temp, etc.)
- Configuration read/write via M2M protocol
- "Trigger DFU Mode" button to enter bootloader without manual reset

**Bootloader Mode:**
- Version info and header dump
- Firmware upload with XMODEM-CRC
- Erase and reset commands
- Progress indication during upload

**Automated Test Scenarios:**
- T1: Mode auto-detection
- T2: Full upload cycle
- T3: DFU trigger from application
- T4: Corrupt header recovery
- T5: More scenarios available

### Workflow: Update Firmware from Application Mode

1. Connect to device (auto-detects APPLICATION mode)
2. Click "Trigger DFU Mode"
3. Device resets, GUI detects BOOTLOADER mode
4. Browse for `.bin` file (must be patched with `patch_header.py`)
5. Click "Upload Firmware"
6. Wait for upload and CRC verification
7. Click "[R] Reset to App" to boot new firmware

## Real-time CAN Viewer

For debugging CAN issues, use the GUI viewer:

```bash
python tools/can_viewer.py
```

Features:
- Live RPM, Torque, Temperature gauges
- Time series graphs
- CAN vs ECU comparison with % difference

## Adding Custom Tests

Extend `test_harness.py`:

```python
class TestResult:
    def add_pass(self, name: str): ...
    def add_fail(self, name: str, reason: str): ...

# Add after existing tests:
print("\n[TEST] Custom validation...")
if my_condition:
    results.add_pass("Custom test")
else:
    results.add_fail("Custom test", "Reason for failure")
```

## Hardware Setup

### Required Connections

| Equipment | Connection |
|-----------|------------|
| J-Link | SWD to target (SWDIO, SWCLK, GND, VTref) |
| USB-Serial | TX/RX to target UART pins |
| PCAN-USB | CAN_H, CAN_L to target CAN bus |

### J-Link Serial

The flash script uses a specific J-Link by serial number to avoid the GUI picker:

```python
# In flash.py
JLINK_SERIAL = "50114603"
```

Change this if using a different J-Link.

### Auto-Detect Serial Port

The test harness auto-detects serial ports, preferring:
1. J-Link CDC port
2. FTDI adapters
3. CP210x/CH340 adapters
4. Any available port

Use `--port` to override:

```bash
python tools/test_harness.py --hex build/addvantage3.hex --port /dev/ttyUSB0
```