The Centers for Medicare & Medicaid Services (CMS) blames a codification error for the disappearance of the Tukey outlier deletion, a statistical method for removing outliers when calculating Star measure cut points, when the final rule for Medicare Advantage (MA) and Part D prescription drug programs was implemented on June 28. While we watch and wait for this to be fixed, here’s why “TukeyGate” matters and what to do next.
Now you see it, now you don’t
In the May 9 Final Rule that took effect on June 28, CMS quietly erased the Tukey Outer Fence Outlier Deletion statistical method from the regulatory text that describes its calculation methodology of Star ratings. You read that right. Hardly anyone noticed when it happened, not even some Stars folks at CMS. It was such a surprise that Stars and actuarial experts we consulted with were all left scratching their heads. Opinions as to what happened ranged from “nothing to see here… it’s just an error” to “someone tried to get away with something” to “CMS is trying to give itself a little more flexibility in case the timing needs to change.”
What does CMS say? They assure us they are implementing Tukey as announced for 2024 Stars and that they will fix the “codification error in the regulatory text” during the next regulatory update. While we watch and wait for this to be fixed, here’s why “TukeyGate” matters and what to do next.
RELATED: CMS issues the 2023 Medicare Advantage and Part D Final Rule: 10 things to know
So, what’s the big deal?
The Tukey method is expected to improve the stability of cut points and prevent cut points from being influenced by outliers beginning with the 2024 Star ratings. Through the use of the Tukey method, CMS intends to remove extreme outliers from measure scores prior to clustering to prevent these outliers from impacting cut points for all contracts. In doing so, CMS expects that it would be harder for plans to earn (or keep) their Star rating.
In the 2020 Final Rule where CMS finalized the use of Tukey, the agency explained the results of its modeling: “If Tukey outer fence outlier deletion and a 5 percent cumulative guardrail had been implemented for the 2018 Star ratings, 2 percent of MA–PD contracts would have seen their Star rating increase by half a star and 16 percent would have decreased by half a star.”
What will Tukey’s impact be?
It’s hard to predict in advance of the measurement period which measures will have extreme outliers, but in prior modeling, CMS determined that extreme outliers are more common in the lower end of the score distribution. As a result, the 1- to 2-Star thresholds often increased in the simulations when outliers were removed compared to the other thresholds which were not as impacted.
Plans with measure ratings at or near the lower edges of current cut points are scrambling to get measurement year 2022 rates up and bracing for the potential loss of half to a full Star. That’s because lower Star ratings would reduce bonus payments, rebates, and supplemental benefits offered to beneficiaries. For plans serving the most vulnerable, lower bonus and rebate dollars could make it even harder to address disparities. So, Tukey or no Tukey, plans shouldn’t rest on their laurels.
While CMS downplays the impact Tukey will have on 4- and 5- Star cut points, our math shows numerous instances where past cut points would have been higher after removal of outliers, significantly in some cases. While we await the full measurement year 2021 measure rate data, we’ve computed the impact Tukey and mean resampling would have had on measurement year 2020 scores, and it’s clear that current year goals are going to need to be adjusted this fall on many measures. Most plans either use prior year cut points as their basis for current year goal setting or use relatively rudimentary prediction methodologies to set goals.
Tukey’s impact on measurement year 2022 will not be inconsequential. Our advice is to take the time to create a modernized cut point governance model, and as painful as it will be, educate stakeholders on the mathematical technicalities they need to prepare for.
Threats beyond Tukey
There are also other significant threats to the measure scoring for Star ratings in 2023 and beyond, including:
- Rollback of the temporary expansion of Extreme and Uncontrollable Circumstances (EUC) due to COVID: Virtually all plans met the definition of “affected” contracts in the 2020 measurement year under the EUC policy. “Affected” plans enjoyed the better of current or prior year performance in almost all measures. This led to an artificial increase in the 2022 Star Ratings for about 50 percent of contracts, relative to what their Star rating without EUC would have been if the Star rating had been calculated on 2020 measurement year performance alone. Once EUC goes away for 2023 Stars, this artificial boost goes with it.
- Guardrails go into effect: To increase the predictability of the cut points used for measure-level ratings, CMS adopted guardrails for measures that have been in the program for more than three years. Guardrails are meant to ensure that the measure threshold-specific cut points for non-CAHPS measures don’t increase or decrease more than 5 percentage points from one year to the next. But the trade-off for the consistency provided by the bidirectional cap is the inability to keep pace with changes in achievement across the industry. While cut points that change less than the cap would be unaffected and keep pace with changes in the measure score trends, changes in the overall performance that are greater than the cap would not be reflected in the new cut points.
- Improvement measure “Hold Harmless” protection ends: The Star ratings calculation methodology includes a Hold Harmless provision that stipulates that if the inclusion of the Part C and Part D improvement measures reduced the overall Star ratings for plans with 4 or more Stars, they would be excluded from the overall rating calculation. For 2022 Stars, CMS expanded the Hold Harmless protection to all plans, not just those with 4 or more Stars. Once that protection is removed for 2023 Stars, the improvement measures can once again drag down the ratings of plans with less stellar performance.
- New measures TRC and FMC shine new light on common data challenges: The challenges associated with communication lapses between inpatient and outpatient providers must be resolved to succeed in these new measures. Transitions of Care (TRC) measures key points of transition after members leave inpatient facilities. Plans must report four separate rates for this measure:
- notification of inpatient admission
- receipt of discharge information
- patient engagement after inpatient discharge
- medication reconciliation post-discharge
Follow-Up after Emergency Department Visit for People with Multiple High-Risk Chronic Conditions (FMC) measures the percentage of emergency department visits for members 18 years or older with multiple high-risk chronic conditions who had a follow-up service within seven days of the visit. Plans must collaborate with inpatient and outpatient providers to achieve real-time or near-time communication to make headway.
- Member experience measures go to 4x weight: For 2023 Stars and beyond member experience measures will make up a majority of the overall Star ratings score. Plan received their CAHPS scores earlier this week, which make up about 50 percent of the member experience measures. The big takeaway was that plans underperformed from the prior year. That was a big surprise for many, given the concerted focus on these measures. Our colleagues across the industry think this result was due to widespread resistance to evolve the workflows, processes, and “old thinking” that’s so embedded within our organizations. Change management is hard! Success with member experience measures will depend on flawless execution and being “brilliant at the basics.” Plans must look at multiple journeys, identify all member touchpoints, and understand the drivers of abrasion and satisfaction. Only then can they shift from traditional gap closure efforts to a more holistic way to serve members to drive Star ratings achievement.
The bottom line for 2023 and beyond
The bottom line… 2023 Stars is just the beginning of the paradigm shift that will require incremental organizational change management for the foreseeable future.
As we are fond of saying, don’t let the perfect be the enemy of the good. Take small steps if you can’t take big ones… and by all means, ask for help. Plans that get this right will have a very significant competitive advantage over those that don’t. It’s go time!
About the authors
Ana Handshuh, principal at CAT5 Strategies, is a government programs executive with expertise in creating and implementing corporate programs for the health care industry. Her background includes quality, core measures, care management, benefit design and bid submission, accreditation, regulatory compliance, revenue management, communications, community-based care management programs and technology integration. Handshuh currently serves on the board of the Resource Initiative and Society for Education (RISE) and is the chair of RISE's Quality & Revenue Community.
She is a sought after speaker on the national health care circuit in the areas of quality, Star ratings, care management, member and provider engagement, and revenue management. Her recent consultancy roles have included assisting organizations create programs to address the unmet care management needs in the highest risk strata of membership, document their processes and procedures, achieve accreditation status, design and submit government program bids, institute corporate-wide programs and create communications strategies and materials.
Melissa Smith is the executive vice president of consulting and professional services at Healthmine, bringing over 25 years of experience in Star ratings, strategy, sales, and marketing for health plans, providers, pharmacy benefit managers, and industry vendors. She has extensive experience developing strategic and tactical solutions to meet client needs and a strong background of building productive partnerships across internal teams and with external vendors to improve performance on clinical, medication, patient survey, and administrative quality measures.
Most recently, Smith was senior vice president of sales, marketing, strategy, and Stars at Gorman Health Group. She is a well-known thought leader and healthcare strategist with proven success developing enterprise-wide solutions to improve Star ratings, quality performance, health outcomes, and the member experience. Her team helps clients improve performance within quality ratings systems, evaluate market dynamics and opportunities, optimize distribution channels, and support our clients’ strategic planning needs. Prior to Gorman Health Group, Smith served in a leadership capacity at Cigna-HealthSpring. Before working in Medicare Advantage and quality ratings systems, she was an associate director at Vanderbilt University Medical Center.
