Blog rate

Blog: RAND 4.0 still riddled with methodological flaws and incomplete data

The latest hospital price report from the RAND Corporation is, unfortunately, more or less the same. The release comes amid historic financial losses to hospitals and health systems, and rapidly rising costs. Over the past two years of the COVID-19 pandemic, we have seen how essential hospitals and health systems are to their communities and patients as they have tried to cope with repeated surges of hospitalizations, launched life-saving new treatments and, in many cases, even took on more care responsibilities, such as rapidly setting up testing and vaccination centres. They’ve done all of this while facing unprecedented financial challenges throughout the pandemic. In this context, RAND has doubled down on its earlier flawed work, with the same preconceived notions and flawed methods.

The underlying premise is fundamentally dubious: as we have pointed out earlier, Medicare rates should not be used as a benchmark for commercial coverage. Medicare pays below the cost of providing care to Medicare patients – even the Medicare Payment Advisory Commission (MedPAC) recognizes this fact.1 In 2020, Medicare paid 84 cents for every dollar spent by hospitals providing care to Medicare patients.

Over time, chronic underpayment threatens the communities served by hospitals. Medicare underpayments were $75.6 billion in 2020 and have nearly doubled since 2014.2 Armchair critics argue that these underpayments incentivize hospitals to become more efficient. In the real world, a record number of rural hospitals closed in 2020, following years of record closures. These closures have a significant impact on people’s ability to access the care they need.3 Additionally, the RAND authors overgeneralize based on a limited and biased sample of self-funded employers who choose to participate from 11 claims databases of all payors. State and health plans opt in the same way..4 No matter how you slice it, this is a small chunk of what actually happens in real-world hospitals and healthcare systems:

  • The claims included in the report represent only 1.2% of total inpatient admissions and 0.5% of total outpatient visits in the United States during the three-year period.5
  • The authors claim to have increased their sample to 4,102 hospitals. However, they are unable to report inpatient data for more than 1,400 hospitals, or more than a third, because they have data covering fewer than 11 inpatient admissions to these hospitals.
  • The sample represents less than 1% of all hospitalizations in 30 states and less than 1% of all outpatient visits in 39 states.
  • For example, in West Virginia, a state the RAND authors call for having above-average commercial rates, the RAND analysis is based on 0.18% of all hospital admissions and 0.07 % of all outpatient admissions included over the three-year period.

Data at the hospital level are also disappointing. For about a third of the hospitals (33.6%) for which inpatient stays are reported, the authors had fewer than 100 inpatient stays over a three-year period on which to draw their conclusions.

Tellingly, in this year’s iteration, when RAND incorporated more claims compared to previous versions, the average price of hospital services actually went down. This suggests what we’ve long suspected and publicly pointed out: you simply cannot draw credible conclusions from such a limited and biased set of claims.

Likewise, RAND’s efforts to link price and quality are failing. RAND only uses one year of star ratings (2018) and Leapfrog scores (2019) despite the fact that these scores were refreshed several times during the measured period of 2018-2020. Additionally, due to normal data reporting timelines, some of the data included in Leapfrog Star Ratings and Scores may be up to five years behind the pricing data. Additionally, Star Ratings and Leapfrog scores are based on the metrics included in them and simply do not measure all aspects of patient care, or even those aspects of care that may be most important to patients.

It is also telling that once again RAND 4.0 ignores the role that another important set of stakeholders – health insurers and third-party administrators – play in healthcare spending. There is no discussion of the leverage insurers and third-party administrators often have in the markets they dominate. This omission occurs because RAND relies on data provided voluntarily by insurers, which RAND undertakes not to identify in its public reports.6 Additionally, RAND also ignores increasingly prevalent insurer policies and procedures that increase costs for hospitals and can lead to dangerous delays in care for patients.

The past two years have shown how vital hospitals and health systems are to every community. Unlike other entities, our doors are always open, regardless of a patient’s ability to pay. Many hospitals have taken on public health roles during the COVID-19 pandemic by holding testing operations and vaccination clinics. This is in addition to the non-clinical care that hospitals and health systems increasingly provide, often without additional resources, including programs that aim to improve housing stability and food security. No other part of the health care system has scaled up in the same way – and been on the front line – to meet these growing demands.

COVID-19 has changed the healthcare system, but RAND’s commentary – already flawed to begin with – remains stuck in the past.

Many hospitals and health systems are now struggling to make ends meet as they continue to care for patients in their communities day in and day out. Those who are already struggling are barely hanging on. Imposing cuts based on erroneous Medicare rates would remove desperately needed resources and funding from vital programs. Now more than ever, this should be a non-starter.

Aaron Wesolowski is the AHA’s Vice President for Policy Research, Analysis and Strategy. Benjamin Finder is the AHA’s director of research and policy analysis.


5 AHA Annual Survey, 2018-2020.