Is inter-rater reliability of Global Trigger Tool results altered when members of the review team are replaced?

• Published in: International Journal for Quality in Health Care
• Main Authors: Mevik, Kjersti, Griffin, Frances A., Hansen, Tonje Elisabeth, Deilkås, Ellen C Tveter, Vonen, Barthold
• Format: Article in Journal/Newspaper
• Language: English
• Published: Oxford University Press 2016
• Subjects: VDP::Medical disciplines: 700::Health sciences: 800::Health service and health administration research: 806; VDP::Medisinske Fag: 700::Helsefag: 800::Helsetjeneste- og helseadministrasjonsforskning: 806; inter-rater reliability; Global Trigger Tool; adverse events; quality measurement; incident reporting and analysis; medical errors; drug errors; Mevik; Norway; Northern Norway
• Online Access: https://hdl.handle.net/10037/15250; https://doi.org/10.1093/intqhc/mzw054

This is a pre-copyedited, author-produced version of an article accepted for publication in the International Journal for Quality in Health Care following peer review. The version of record Mevik, K., Griffin, F.A., Hansen, T.E., Deilkås, E. & Vonen, B. (2016). Is inter-rater reliability of Global Trigger Tool results altered when members of the review team are replaced? International Journal for Quality in Health Care, 28 (4), 492-496 is available online at https://doi.org/10.1093/intqhc/mzw054 . Objective - To evaluate the inter-rater reliability of results from Global Trigger Tool (GTT) reviews when one of the three reviewers remains consistent, while one or two reviewers rotate. Design - Comparison of results from retrospective record review performed as a cross-sectional study with three review teams each consisting of two non-physicians and one physician; Team I (three consistent reviewers), Team II (one of the two non-physician reviewers or/and the physician from Team I are replaced for different review periods) and Team III (three consistent reviewers different from reviewers in Team I and Team II). Setting - Medium-sized hospital trust in Northern Norway. Participants - A total of 120 records were selected as biweekly samples of 10 from discharge lists between 1 July and 31 December 2010 for a 3-fold review. Intervention - Replacement of review team members was tested to assess impact on inter-rater reliability and adverse events measurment. Main Outcome Measure(s) - Inter-rater reliability assessed with the Cohen kappa coefficient between different teams regarding the presence and severity level of adverse events. Results - Substantial inter-rater reliability regarding the presence and severity level of adverse events was obtained between Teams I and II, while moderate inter-rater reliability was obtained between Teams I and III. Conclusions - Replacement of reviewers did not influence the results provided that one of the non-physician reviewers remains consistent. The experience of the consistent reviewer can result in continued consistency in interpretation with the new reviewer through discussion of events. These findings could encourage more hospital to rotate reviewers in order to optimize resources when using the GTT.