LAUSR

To the Editor: We appreciate the efforts by Drs Smith, Reichert, Ameling, and Meddings to analyze the methodology of the Leapfrog Hospital Safety Grade and offer suggestions on how hospitals can impact their grade. Analyses such as these help Leapfrog and our Hospital Safety Grade Expert Panel review and refine the methodology over time. However, it is unlikely to help hospitals. Although it is theoretically possible to “game” any ratings system, we find that hospitals attempting this with the Leapfrog Hospital Safety Grade are generally frustrated by the result. Two factors make it exceptionally difficult to predict the outcome of deliberate manipulation. First, the Leapfrog Hospital Safety Grade is composed of 30 measures, meaning that there are thousands of potential statistical outcomes that cannot be predicted with certainty. The researchers focused on 2 of those potential scenarios, because no study could encompass the complexity of configurations possible. Second, the grade is calculated on a curve, so even by using data from one point in time, hospitals cannot know how they will compare until the full range of hospitals are scored. The study’s authors also questioned the reliability of “self-reported” data. The Leapfrog Group goes to extreme lengths to verify data submitted to the Leapfrog Hospital Survey. In addition to an onsite verification conducted at a random selection of hospitals, Leapfrog’s experts analyze each survey response independently, including verifying the infection data cited by the researchers with National Health and Safety Network for selected hospitals. It seems that the researchers may not have aligned reporting periods and certain other factors when they suggested differences between Centers for Medicare and Medicaid Services and Leapfrog results. The researchers questioned why adherence to the Safe Practices do not always correlate with infection rates. We believe that hospitals that adhere to the Safe Practices are on the right path toward better outcomes. But there are many other steps along that path, so we have never assumed they correlate 100% with safety outcomes. Indeed, that is why we consider both processes and outcomes in scoring the Leapfrog Hospital Safety Grade. Although complex machinations to influence the Leapfrog Hospital Safety Grade rarely work, there are some good general strategies we find in hospitals that regularly earn an “A.” First, hospitals that sustain an “A” assure that Leapfrog’s grade composite contains as many measures as possible. This minimizes the impact of any 1 lagging measure, and gives the hospital the opportunity to tell its complete story. Second and most importantly, these consistently high-performing hospitals focus on patient safety as their priority, day in and day out. When hospitals work every day to protect their patients from harm, the results invariably show up in our data; these hospitals will likely be on track for an “A.” More significantly, their patients survive and thrive. We would encourage researchers to analyze hospitals that have achieved an “A,” to understand characteristics that drive safety excellence.