← Back to the learning library
Before you begin: TFA-101 and basic requirements, assertions and target-board operation.
Reading time and practice time are estimates. Examples use fictional products and illustrative engineering values, not approved product limits.
Learning objectives
By the end of this module, you should be able to:
- Allocate objectives across unit, integration and system-level tests.
- Design seams, fakes, simulators and instrumentation for testability.
- Use hardware-in-the-loop where electrical or physical behaviour matters.
- Assess coverage as evidence, not as a substitute for test quality.
Lesson 1
1. Allocate tests to the cheapest credible level
Unit tests give fast, precise feedback for algorithms, state transitions and boundary behaviour. Integration tests verify interfaces, data representation and collaboration between components. Target tests exercise compiler, processor and real drivers. Hardware-in-the-loop tests close the loop through electrical interfaces and realistic timing.
Do not repeat every test at every level. Allocate each objective where the relevant failure can be detected with adequate fidelity and diagnosis.
Lesson 2
2. Design firmware for controllability and observability
Inject time, peripheral interfaces and configuration rather than reading global hardware directly. Use fakes for deterministic host tests, simulators for protocol peers and instrumentation for target timing. Test interfaces should not create production backdoors; compile-time controls and access restrictions must be deliberate.
Lesson 3
3. HIL verifies the boundary the code cannot model alone
HIL is valuable for ADC ranges, signal timing, communication errors, actuator feedback and power/reset behaviour. The rig should have a controlled configuration, calibrated measurements where accuracy matters and scripts that retain raw observations.
Include fault insertion: disconnected sensors, stuck values, noisy signals, delayed responses, brownout and interrupted communications.
Lesson 4
4. Coverage answers a limited question
Requirements coverage shows that specified behaviour has tests. Structural coverage shows which code was executed and may reveal missing tests or unintended code. Neither proves that assertions are strong, inputs are representative or requirements are correct. Review uncovered code and justify exclusions rather than pursuing a percentage without meaning.
Lesson 5
5. Design a test oracle before the stimulus
The oracle is the rule used to decide whether observed behaviour is correct. A test that copies the implementation formula can repeat its defect. Derive expected values from an independent calculation, reference model, physical standard or approved requirement, and document tolerances.
Test boundaries and combinations that distinguish correct behaviour from plausible mistakes. For a threshold, include just below, exactly at and just above the limit, plus invalid and stale measurements. For a state machine, check prohibited transitions and unchanged state after rejection, not only that expected transitions occur.
Use mutation thinking: if the comparison changed from greater-than to greater-than-or-equal, would the test fail? If a timeout were removed, would the suite notice? This challenges assertion strength without assuming that executed code is adequately verified.
Lesson 6
6. Control HIL fidelity and uncertainty
A HIL rig is another engineered system. Its simulator, wiring, power supply, timebase, measurement bandwidth, scripts and calibration affect results. State which physical properties are represented and which are not. A voltage source emulating a sensor may not reproduce the sensor’s impedance or transient response.
For a 10 ms response limit, a rig sampled every 10 ms cannot establish a small compliance margin. Combine timestamp quantisation, channel alignment, transport delay and measurement uncertainty. Select instrumentation with sufficient resolution and use an acceptance rule consistent with the required confidence.
Keep fault injection electrically safe. A software command to simulate an open circuit is not evidence of an actual disconnected pin unless the injection boundary represents the intended failure. Record the location and mechanism of each inserted fault.
Lesson 7
7. Manage regression and trustworthy automation
Classify tests by requirement, risk, component, target variant and environment. A change-impact analysis can then choose focused regression and the system checks needed to preserve confidence. Test selection itself becomes controlled evidence rather than a developer’s memory of what seems relevant.
| Evidence item | What must be retained |
|---|---|
| Test identity | Versioned procedure and requirement link. |
| Configuration | Firmware hash, board, rig and scripts. |
| Observation | Raw data or logs with timing and units. |
| Verdict | Acceptance rule, result and reviewer where needed. |
| Anomaly | Unexpected behaviour and documented disposition. |
Retries are diagnostic information. Report an initial failure even if a rerun passes, then determine whether the cause is the device, fixture or automation. Silently retrying until green removes evidence of intermittent behaviour.
Decision guide
Where should a verification objective run?
| Objective | Preferred level | Reason |
|---|---|---|
| State-machine guard and transition | Host unit test | Deterministic, fast and exhaustive across combinations. |
| Driver / service contract | Target integration test | Uses compiled target interface and representative peripheral behaviour. |
| Sensor fault causes safe actuator response | HIL / system test | Requires electrical stimulus, timing and physical output observation. |
Worked example
Worked example — pump controller test pyramid
A controller regulates flow from a pressure sensor and stops on occlusion.
Host tests cover filtering, control calculations, state transitions and limit boundaries. Target integration tests verify ADC conversion, task timing and the driver-service interface. HIL supplies calibrated pressure profiles, disconnection and noise while measuring the stop output. One traceability record allocates requirements and risk controls across levels.
Most defects are found quickly, while the high-value end-to-end tests demonstrate real electrical and timing behaviour.
Engineering evidence
What should the work produce?
Verification strategy
Allocates methods and levels to objectives.
Test specifications
Defines setup, stimulus, expected result and acceptance.
Rig configuration record
Identifies hardware, software, calibration and scripts.
Results and anomaly records
Retain observations, verdicts and departures.
Common failure patterns
What goes wrong?
- Testing pure logic only on hardware, making feedback slow and diagnosis poor.
- Mocks that reproduce the implementation rather than the interface contract.
- HIL scripts that record only pass/fail and discard measured traces.
- Claiming adequacy from code coverage percentage alone.
Applied exercise · approximately 25 minutes
Write a fault-response protocol
An illustrative requirement says a heater-enable output shall be inactive within 20 ms after a sensor circuit is disconnected during Heating, and shall remain inactive until an authorised reset.
- Define preconditions, the injection boundary, observation channels and timing origin.
- Specify tests at different sample phases and a case where reset is requested before the fault clears.
- State the configuration and raw evidence to retain.
Review a suggested approach
Begin in a stable Heating state with the released configuration. Physically open the sensor path using a controlled fixture and measure both the fault insertion and enable output on a common timebase. Repeat near different acquisition phases to expose detection latency. The pass rule includes the 20 ms response and continued inhibition.
Test a reset request while the circuit is still open: the required re-entry policy must prevent renewed heating until safety conditions are satisfied. Retain traces, firmware and rig hashes, board identity and uncertainty. If the requirement leaves reset authority or fault-clear criteria undefined, record the gap before execution rather than improvising the expected behaviour.
Completion criteria
- The test observes a physical outcome and its timing.
- Persistent inhibition and recovery are checked separately.
- Evidence identifies the actual device, rig and acceptance rule.
This is a self-directed engineering exercise, not an accredited examination. Compare the rationale as well as the final answer.
Practical check
Questions for your next design review
- Is every requirement and risk control allocated to a credible test level?
- Can time and hardware dependencies be controlled in lower-level tests?
- Does HIL include abnormal electrical and timing conditions?
- Are rig versions and calibration status recorded?
- Are unexpected results handled as controlled anomalies?
A strong test strategy combines fast isolation of logic with credible verification at the physical boundary.
References and further reading
Educational guidance only. Apply the authorised standards editions, regulations and approved procedures relevant to your product and market. Linked publications support further study; the fictional exercises do not demonstrate compliance.