Methodology for evaluating usability in HMI-design for cockpit in combat aircraft

Authors:

  • Diana Saleh
  • Per-Anders Oskarsson
  • Patrik Lif

Publish date: 2021-10-12

Report number: FOI-R--5182--SE

Pages: 36

Written in: Swedish

Keywords:

  • Validation
  • evaluation
  • user test
  • interview
  • observation
  • mental workload
  • situational awareness
  • performance measures
  • usability
  • HMI

Abstract

The large number of methods that are used in evaluations of usability and humansystem interaction could make it feel far from trivial to know which method is appropriate to use in a specific evaluation. This report has been written to support the selection work for evaluation methods at Saab when validating the Human Machine Interface in the cockpit of the Gripen. The report presents four concepts that are central to testing usability: qualitative methods, situation awareness, mental workload and performance. The description of each concept begins with a theoretical background. This is followed by descriptions of accepted methods for data collection, measurement and analysis of various aspects of the concepts. In total, the report presents 18 evaluation methods and performance measures. Nine of these methods are recommended in for validations at Saab. Descriptions of how these methods are used are described in a method guide in the report's appendix. The method guide's two main purposes are to (1) facilitate the understanding of when each method can or should be used and (2) act as a practical guide when evaluations are carried out with each method. The work has been carried out on behalf of the Swedish Defense Materiel Administration (FMV) within the HMI-support Gripen project, where the overall purpose is to contribute to an improved evaluation process.