Assessing system readiness from component technology readiness levels
Technology readiness levels form an ordinal scale, and a system is only as ready as its least mature component. Two systems with the same average TRL can differ by years in time to readiness. A Monte Carlo simulation of component maturation quantifies the difference.
"The system is at TRL 7.8." I have seen this sentence in more than one status report, and it always deserves a pause. Technology readiness levels run from 1 (basic principles observed) to 9 (proven in operation). The scale has discrete levels, and 7.8 is not one of them.
TRLs are among the most widely adopted management tools from the space sector: a shared vocabulary for maturity that engineers, managers and funding agencies understand. Widely adopted vocabularies tend to be stretched. This article examines what an average TRL conceals and how the readiness of a whole system should be assessed.
The scale was conceived at NASA in the 1970s; the nine-level version most people use was described by John Mankins in a 1995 white paper. Mankins (1998) added a companion measure, the R&D degree of difficulty, because two technologies at the same level can differ greatly in how difficult they are to advance. ISO 16290:2013 standardised the definitions for space systems, and the European Space Agency adopted it through the ECSS. The European Commission's Horizon 2020 programme used a nine-level scale too, which carried TRLs far outside aerospace.
The TRL scale
| TRL | Meaning (short form) |
|---|---|
| 1 | Basic principles observed |
| 2 | Technology concept formulated |
| 3 | Experimental proof of concept |
| 4 | Validated in the laboratory |
| 5 | Validated in a relevant environment |
| 6 | Demonstrated in a relevant environment |
| 7 | Prototype demonstrated in the operational environment |
| 8 | System complete and qualified |
| 9 | Proven in operation |
The numbers label stages, and the stages are not equally long, equally expensive or equally risky. Getting from 3 to 4 may take months in a lab; getting from 6 to 7 may need a flight. The scale is ordinal: it orders states but says nothing about the distance between them. Averaging ordinal values is therefore not meaningful.
System readiness and the least mature component
A system of components developed in parallel is ready when its slowest component is. Its readiness is governed by the minimum, not the mean. A system of four flight-proven components and one laboratory prototype is therefore not close to readiness: the prototype determines the schedule, and the maturity of the other components does not transfer to it.
A house renovation works the same way. When the walls, roof and windows are finished and the electrical installation has not been started, the house is not 80% habitable. The date on which the family can move in is set by the electrician, and finishing the paintwork earlier does not bring it forward.
The figure below compares two systems. The claim: two systems with the same average TRL can need very different times to be ready, because the least mature component drives the schedule.
The model is illustrative, not calibrated: each step takes a random time scaled by the R&D difficulty, and from TRL 4 onwards a failed test can send a component back one level.
- System A. Four components at TRL 9 and a new sensor at TRL 3. Average 7.8. At medium difficulty, the median time to a ready system is about 11 years, and the chance of being ready within five years is essentially zero.
- System B. Components at 8, 8, 8, 7 and 8. Also average 7.8. Median time about 5 years, with roughly even odds of being ready within five. With the same average, it becomes ready in less than half the time.
- Effect of difficulty. Set the difficulty to 1 and to 5 for each system. The gap between A and B stays, but the absolute times move by a factor of four. This was Mankins' argument in 1998: a TRL is incomplete without a difficulty estimate.
- Targeted improvement. Click the components of system A to raise one TRL at a time. Raising a TRL 9 component has no effect. Raising the sensor from 3 to 4 cuts the median by a year or more. Investment is most effective at the least mature component.
The role of the minimum. If component becomes ready at the random time and the system needs all of them, the system is ready at . Then for independent components, which is never larger than the probability for the slowest component alone. Improving a component that is already almost surely ready by time multiplies the product by a factor close to 1 and changes nothing.
Recommendations:
- Report a system's readiness as its lowest component TRL, with the component named, not as an average.
- Pair every TRL with a difficulty estimate and a plan for the next step, since the scale says nothing about distance.
- Put development money where the minimum is. Further work on mature components does not shorten the schedule.
References
- J. C. Mankins, Technology Readiness Levels: A White Paper, NASA Office of Space Access and Technology, Advanced Concepts Office, 1995.
- J. C. Mankins, Research & Development Degree of Difficulty (R&D3): A White Paper, NASA Office of Space Flight, Advanced Projects Office, 1998.
- ISO 16290:2013, Space systems - Definition of the Technology Readiness Levels (TRLs) and their criteria of assessment.
- European Commission, Horizon 2020 Work Programme 2014-2015, General Annexes, G. Technology readiness levels (TRL), 2014.
- O. L. de Weck, Technology Roadmapping and Development: A Quantitative Approach to the Management of Technology, Springer, 2022.