How to Test Usability and Security

While security testing for software is becoming an important topic among software testers, little attention has been paid to a critical aspect of security: its usability. Security testing is focused on finding vulnerabilities of the software, regardless of whether additional security problems occur from bad usability designs. This article proposes a framework for testing the usability of security sensitive systems. Based on the systemic approach to usability, it defines general usability attributes and calls for their experimental evaluation. The authors apply this framework to test the usability of alternative realizations of a widespread PKI-based system. These alternatives differ for the personal security devices used, that is, traditional cryptographic smart cards or USB tokens. The tests clearly show that usability issues lead to security problems. The interpretation and explanation of the test results bring up recommendations for software practitioners to properly address usability in security software based on security devices.