Validation of CyberTest Future Soldiers

Authors:

  • Patrik Lif
  • Ove Jansson
  • Christian Valassi
  • Daniel Eidenskog

Publish date: 2020-06-29

Report number: FOI-R--4977--SE

Pages: 38

Written in: Swedish

Keywords:

  • Cyber test
  • validation
  • reliability
  • test fairness
  • CTFS 2019
  • CTFS
  • cyber
  • cyber defence

Abstract

To strengthen the Swedish cyber defence against adversaries through the cyber domain, a new education is set to open in 2020 with selected academic courses in computer science and operative training related to cyber defence. In the last quarter of 2019, 70 individuals were selected through an overall judgement where a conscription test called Inskrivningsprovet was one important part. In this extended selection process, another test targeted towards certain knowledge and abilities was used. The test used was Cyber Test Future Soldiers 2019 (CTFS 2019) that was developed by the Swedish Defense Research Agency in 2019. The validation of CTFS 2019 will take place in four phases: selection, education, internship, and deployment. This report presents the results of the validation during the selection phase and is based on data collected from the 62 individuals who fulfilled the extended selection process in early 2020. The validation consists of extensive statistical analysis that aim to measure validity, reliability, and test justice. Additional analysis shows that the different parts of CTFS 2020 are able to capture the extent of each individual's knowledge in specific domains as well as in several different cognitive abilities. The validation process shows that CTFS 2019 is working as intended in the selection phase. In future work, the Swedish Defence Research Agency aims to investigate how well CTFS 2019 is capable of predicting the performance during education.