LAUSR

Critical Care Medicine www.ccmjournal.org e265 11. Nelson JC, Pepe MS: Statistical description of interrater variability in ordinal ratings. Stat Methods Med Res 2000; 9:475–496 12. Sim J, Wright CC: The kappa statistic in reliability studies: Use, interpretation, and sample size requirements. Phys Ther 2005; 85:257–268 13. Vanbelle S: A new interpretation of the weighted kappa coefficients. Psychometrika 2016; 81:399–410 14. Vanbelle S: Clinical agreement in qualitative measurements: The kappa coefficient in clinical research. In: Methods of Clinical Epidemiology, Springer Series on Epidemiology and Public Health. Doi S Williams G (Eds). Heidelberg, Germany, Springer, 2013, pp 3–38 15. Feinstein AR, Cicchetti DV: High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol 1990; 43:543–549 16. Byrt T, Bishop J, Carlin JB: Bias, prevalence and kappa. J Clin Epidemiol 1993; 46:423–429 17. Flight L, Julious SA: Practical guide to sample size calculations: An introduction. Pharm Stat 2016; 15:68–74 18. Kottner J, Audigé L, Brorson S, et al: Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol 2011; 64:96–106 DOI: 10.1097/CCM.0000000000004194