Combining multivariate statistics and the think-aloud protocol to assess Human-Computer Interaction barriers in symptom checkers

• Published in: Journal of Biomedical Informatics
• Main Authors: Marco-Ruiz, Luis, Bones, Erlend, de la Asuncion, Estela, Gabarron, Elia, Aviles-Solis, Juan Carlos, Lee, Eunji, Traver Salcedo, Vicente, Sato, Keiichi, Bellika, Johan G.
• Other Authors: Universitat Politècnica de València. Departamento de Ingeniería Electrónica - Departament d'Enginyeria Electrònica, Helse Nord RHF, Research Council of Norway, UiT The Arctic University of Norway
• Format: Article in Journal/Newspaper
• Language: English
• Published: Elsevier 2017
• Subjects: Human computer interaction; Usability testing; Clinical decision support systems; Symptom checkers; TECNOLOGIA ELECTRONICA; Arctic; Norway; Romero; Arctic University of Norway; UiT The Arctic University of Norway
• Online Access: http://hdl.handle.net/10251/152267; https://doi.org/10.1016/j.jbi.2017.09.002

[EN] Symptom checkers are software tools that allow users to submit a set of symptoms and receive advice related to them in the form of a diagnosis list, health information or triage. The heterogeneity of their potential users and the number of different components in their user interfaces can make testing with end-users unaffordable. We designed and executed a two-phase method to test the respiratory diseases module of the symptom checker Erdusyk. Phase I consisted of an online test with a large sample of users (n = 53). In Phase I, users evaluated the system remotely and completed a questionnaire based on the Technology Acceptance Model. Principal Component Analysis was used to correlate each section of the interface with the questionnaire responses, thus identifying which areas of the user interface presented significant contributions to the technology acceptance. In the second phase, the think-aloud procedure was executed with a small number of samples (n = 15), focusing on the areas with significant contributions to analyze the reasons for such contributions. Our method was used effectively to optimize the testing of symptom checker user interfaces. The method allowed kept the cost of testing at reasonable levels by restricting the use of the think-aloud procedure while still assuring a high amount of coverage. The main barriers detected in Erdusyk were related to problems understanding time repetition patterns, the selection of levels in scales to record intensities, navigation, the quantification of some symptom attributes, and the characteristics of the symptoms. (C) 2017 Elsevier Inc. All rights reserved. This work was supported by Helse Nord [grant HST1121-13], the Faculty of Health Sciences from UIT The Arctic University of Norway [researcher code 1108], and The Research Council of Norway [grant 248150/O70]. We thank Professor Emeritus Rafael Romero-Villafranca for reviewing the statistical analysis of this paper. Marco-Ruiz, L.; Bones, E.; De La Asuncion, E.; Gabarron, E.; Aviles-Solis, JC.; Lee, ...