This interactive is optimized for larger screens.
Please use a device with a minimum screen resolution of 1024x768px for the best viewing experience.

Mirror, Mirror 2017:International Comparison Reflects Flaws and Opportunities for Better U.S. Health Care

The United States Health System Falls Short

Exhibit 1. Health Care Spending as a Percentage of GDP, 1980–2014

Notes: GDP refers to gross domestic product.

Source: OECD Health Data 2016. Data are for current spending only, and exclude spending on capital formation of health care providers.

The United States spends far more on health care than other high-income countries, with spending levels that rose continuously over the past three decades (Exhibit 1). Yet the U.S. population has poorer health than other countries. 1 S. H. Woolf and L. Aron (eds.), U.S. Health in International Perspective: Shorter Lives, Poorer Health (National Academies Press, 2013). Life expectancy, after improving for several decades, worsened in recent years for some populations, aggravated by the opioid crisis. 2 A. Case and A. Deaton, “Rising Morbidity and Mortality in Midlife Among White Non-Hispanic Americans in the 21st Century,” Proceedings of the National Academy of Sciences of the United States of America, Dec. 8, 2015 112(49):15078–83. In addition, as the baby boom population ages, more people in the U.S.—and all over the world—are living with age-related disabilities and chronic disease, placing pressure on health care systems to respond.

Timely and accessible health care could mitigate many of these challenges, but the U.S. health care system falls short, failing to deliver indicated services reliably to all who could benefit. 3 L. T. Kohn, J. M. Corrigan, and M. S. Donaldson (eds.), To Err Is Human: Building a Safer Health System (National Academies Press, 2000); and D. Blumenthal, M. K. Abrams, and R. Nuzum, “The Affordable Care Act at 5 Years,” New England Journal of Medicine Online First, published online May 6, 2015. In particular, poor access to primary care has contributed to inadequate prevention and management of chronic diseases, delayed diagnoses, incomplete adherence to treatments, wasteful overuse of drugs and technologies, and coordination and safety problems.

This report uses recent data to compare health care system performance in the U.S. with that of 10 other high-income countries and considers the different approaches to health care organization and delivery that can contribute to top performance. We based our analysis on 72 indicators that measure performance in five domains important to policymakers, providers, patients, and the public: Care Process, Access, Administrative Efficiency, Equity, and Health Care Outcomes.

Our data come from a variety of sources. One is comparative survey research. Since 1998, The Commonwealth Fund, in collaboration with international partners, has supported surveys of patients and primary care physicians in advanced countries, collecting information for a standardized set of metrics on health system performance. Other comparative data are drawn from the most recent reports of the Organization for Economic Cooperation and Development (OECD), the European Observatory on Health Systems and Policies, and the World Health Organization (WHO).


Performance Varies Among Health Systems

The United States ranks last in health care system performance among the 11 countries included in this study (Exhibit 2). The U.S. ranks last in Access, Equity, and Health Care Outcomes, and next to last in Administrative Efficiency, as reported by patients and providers. Only in Care Process does the U.S. perform better, ranking fifth among the 11 countries. Other countries that rank near the bottom on overall performance include France (10th) and Canada (9th).

This analysis reveals striking variations in performance across the domains. No country ranks first consistently across all domains or measures, suggesting that all countries have room to improve. The U.S., France, and Canada score lower than the 11-country average across most of the five domains, but all three achieve above-average performance on at least one domain: France on Health Care Outcomes, Canada on Care Process and Administrative Efficiency, and the U.S. on Care Process (Appendix 1).

Top Performers

The top-ranked countries overall are the United Kingdom, Australia, and the Netherlands. In general, the U.K. achieves superior performance compared to other countries in all areas except Health Care Outcomes, where it ranks 10th despite experiencing the fastest reduction in deaths amenable to health care in the past decade. Australia ranks highest on Administrative Efficiency and Health Care Outcomes, is among the top-ranked countries on Care Process and Access, but ranks low on Equity. The Netherlands is among the top performers on Care Process, Access, and Equity; its performance on Administrative Efficiency stands out as an area for improvement.

New Zealand performs well on measures of Care Process and Administrative Efficiency, but below the 11-country average on other indicators. Norway and Sweden did better on Health Care Outcomes compared to the other countries, despite having relatively low rankings on Care Process. Switzerland performs well on measures of Equity and Health Care Outcomes, while Germany achieves a high rank only on measures of Access.

Exhibit 2. Health Care System Performance Rankings
Exhibit 3. Health System Performance Scores

Note: See How This Study Was Conducted for a description of how the performance scores are calculated.

Source: Commonwealth Fund analysis.

Exhibit 3 illustrates the countries’ overall performance score (as opposed to their overall performance rank). (See How This Study Was Conducted for a detailed explanation of how these performance scores are calculated). This exhibit makes evident the markedly lower performance of Canada (9), France (10), and the United States (11) compared to the other countries, which all group relatively closely above the 11-country average performance score.

Care Process

The United Kingdom ranks first and Sweden last on Care Process (Exhibit 2) based on the performance across the four subdomains of prevention, safe care, coordination, and patient engagement (Appendix 2). The United States ranks in the middle on Care Process (5th), with stronger performance on the subdomains of prevention, safety, and engagement. The U.S. performs slightly below the 11-country average in the coordination subdomain.

The U.S. tends to excel on measures that involve the doctor–patient relationship, performing relatively better on wellness counseling related to healthy behaviors, shared decision-making with primary care and specialist providers, chronic disease management, and end-of-life discussions (Appendices 2A–2D). The U.S. also performs above the 11-country average on preventive measures like mammography screening and older adult influenza immunization rates. However, the U.S. performs poorly on several coordination measures, including information flows between primary care providers and specialist and social service providers. The U.S. also lags other countries on avoidable hospital admissions.

The U.K., Australia, and New Zealand are the top performers in the Care Process domain. These three countries consistently perform above the 11-country average across all subdomains (except for Australia on coordinated care). The U.K. excels in safety, while Australia is the top performer in patient engagement. On the other end of the spectrum, Norway and Sweden’s performance is below average on each of the Care Process subdomains.

Access

Overall, the United States ranks last on Access (Exhibit 2). The U.S. has the poorest performance of all countries on the affordability subdomain, scoring much lower than even the second-to-last country, Switzerland (Appendix 3). The U.S. ranks ninth on the subdomain of timeliness (Appendix 3).

The Netherlands performs the best of the 11 countries on Access, ranking first on timeliness and in the middle on affordability (Appendix 3). Germany ranks second on Access, and is among the top-ranked countries on both subdomains. The United Kingdom, Sweden, and Norway are the other top-ranked performers on affordability.

Administrative Efficiency

The United States ranks 10th on Administrative Efficiency (Exhibit 2). Compared to the other countries, more U.S. doctors reported problems related to coverage restrictions (Appendix 4). Larger percentages of U.S. patients also reported Administrative Efficiency problems compared to those in other countries (except France). The top performers in this domain are Australia, New Zealand, the United Kingdom, and Norway. At the lower end of the range, respondents from France were most likely to report problems in this area among the surveyed countries.

Equity

The United Kingdom, the Netherlands, and Sweden rank highest on measures related to the equity of health systems with respect to access and care process (Exhibit 2). In these three countries, there are relatively small differences between lower- and higher-income adults on the 11 measures related to timeliness, financial barriers to care, and patient-centered care (Appendix 5).

In contrast, the United States, France, and Canada have larger disparities between lower and higher-income adults. These were especially large on measures related to financial barriers, such as skipping needed doctor visits or dental care, forgoing treatments or tests, and not filling prescriptions because of the cost.

Exhibit 4. Mortality Amenable to Health Care, 2004 and 2014

Source: European Observatory on Health Systems and Policies (2017). Trends in amenable mortality for selected countries, 2004 and 2014. Data for 2014 in all countries except Canada (2011), France (2013), the Netherlands (2013), New Zealand (2012), Switzerland (2013), and the U.K. (2013). Amenable mortality causes based on Nolte and McKee (2004). Mortality and population data derived from WHO mortality files (Sept. 2016); population data for Canada and the U.S. derived from the Human Mortality Database. Age-specific rates standardized to the European Standard Population (2013).

Health Care Outcomes

The United States ranks last overall in Health Care Outcomes (Exhibit 2). However, the pattern of performance across different outcomes measures reveals nuances. Compared to the other countries, the U.S. performs relatively poorly on population health outcomes such as infant mortality and life expectancy at age 60 (Appendix 6). The U.S. has the highest rate of mortality amenable to health care and has experienced the smallest reduction in that measure during the past decade (Exhibit 4). In contrast, the U.S. appears to perform relatively well on 30-day in-hospital mortality after heart attack or stroke. The U.S. also performs as well as several top performers on breast cancer five-year relative survival rate and close to the 11-country average on colorectal cancer five-year relative survival rate.

Australia has the best Health Care Outcomes overall. Sweden and Norway rank second and third in the domain. While the United Kingdom ranks 10th in the health care outcomes domain overall, it had the largest reduction in mortality amenable to health care during the past decade.


Causes of Poor Performance

Exhibit 5. Health Care System Performance Compared to Spending

Note: Health care spending as a percent of GDP.

Sources: Spending data are from OECD for the year 2014, and exclude spending on capital formation of health care providers; Commonwealth Fund analysis.

Based on a broadly inclusive set of performance metrics, we find that U.S. health care system performance ranks last among 11 high-income countries. The country’s performance shortcomings cross several domains of care including Access, Administrative Efficiency, Equity, and Health Care Outcomes. Only within the domain of Care Process is U.S. performance close to the 11-country average. These results are troubling because the U.S. has the highest per capita health expenditures of any country and devotes a larger percentage of its GDP to health care than any other country.

The U.S. health care system is unique in several respects. Most striking: it is the only high-income country lacking universal health insurance coverage. The U.S. has taken an important step to expand coverage through the Affordable Care Act. As a 2017 Commonwealth Fund report showed, the ACA has catalyzed widespread and historic gains in access to care across the U.S. 4 D. C. Radley, D. McCarthy, and S. L. Hayes, Aiming Higher: Results from the Commonwealth Fund Scorecard on State Health System Performance, 2017 Edition (The Commonwealth Fund, March 2017). More than 20 million Americans gained insurance coverage. Additional actions could extend insurance coverage to those who lack it. Furthermore, Americans with coverage often face far higher deductibles and out-of-pocket costs than citizens of other countries, whose systems offer more financial protection. 5 R. Osborn, D. Squires, M. M. Doty, D. O. Sarnak, and E. C. Schneider, “In New Survey of 11 Countries, U.S. Adults Still Struggle with Access to and Affordability of Health Care,” Health Affairs Web First, published online Nov. 16, 2016. Incomplete and fragmented insurance coverage may account for the relatively poor performance of the U.S. on health care outcomes, affordability, administrative efficiency, and equity.

Several new U.S. federal initiatives—notably the Affordable Care Act—have promoted actions to improve U.S. health care system performance. 6 D. Blumenthal, M. K. Abrams, and R. Nuzum, “The Affordable Care Act at 5 Years,” New England Journal of Medicine Online First, published online May 6, 2015; M. Gold and C. McLaughlin, “Assessing HITECH Implementation and Lessons: 5 Years Later,” Milbank Quarterly, Sept. 2016 94(3):654–87; and S. Findlay, Implementing MACRA (Health Affairs, published online March 27, 2017). In addition to extending insurance coverage to millions of Americans, recent legislation includes initiatives to spur innovation in health care delivery by changing payment incentives for providers. But health systems can be slow to change. Additional legislative and policy reforms may be needed to close the performance gap between the U.S. and other countries.

The U.S. could learn important lessons from other high-income countries (see Lessons for the United States). For example, the U.S. performs poorly in administrative efficiency mainly because of doctors and patients reporting wasting time on billing and insurance claims. Other countries that rely on private health insurers, like the Netherlands, minimize some of these problems by standardizing basic benefit packages, which can both reduce administrative burden for providers and ensure that patients face predictable copayments.

The U.K. stands out as a top performer in most categories except for health care outcomes, where it ranks with the U.S. near the bottom. In contrast to the U.S., over the past decade the U.K. saw a larger decline in mortality amenable to health care (i.e., a greater improvement in the measure) than the other countries studied. (The U.S. has had the smallest decline, or lowest level of improvement.) In the early 2000s, the U.K. made a major investment in its National Health Service, reforming primary care and cancer care in addition to increasing health care spending from 6.2 percent of GDP in 2000 to 9.9 percent of GDP in 2014 (Exhibit 1). 7 T. Doran and M. Roland, “Lessons from Major Initiatives to Improve Primary Care in the United Kingdom,” Health Affairs, May 2010 29(5):1023–29. The reforms and increased spending may have contributed to the rapid decline in mortality amenable to health care in the U.K.

There is a striking contrast between the U.S’s poor performance on infant mortality, life expectancy, and amenable mortality and its relatively better performance on in-hospital mortality after heart attack or stroke. Researchers have noted that the only modest decline in the rate of amenable mortality in the U.S. may be attributable to better management, once diagnosed, of hypertension and cerebrovascular disease that lead to cardiovascular mortality. 8 E. Nolte and C. M. McKee, “In Amenable Mortality—Deaths Avoidable Through Health Care—Progress in the U.S. Lags That of Three European Countries,” Health Affairs Web First, published online Aug. 29, 2012. These findings highlight the combined impact of a lack of universal insurance coverage and barriers to accessing primary care, and suggest that the U.S. could make gains by investing more in preventing chronic disease. The high level of inequity in the U.S. health care system intensifies the problem. For the first time in decades, midlife mortality for less-educated Americans is rapidly increasing. 9 A. Case and A. Deaton, “Rising Morbidity and Mortality in Midlife Among White Non-Hispanic Americans in the 21st Century,” Proceedings of the National Academy of Sciences of the United States of America, Dec. 8, 2015 112(49):15078–83.

In conclusion, the performance of the U.S. health care system ranks last compared to other high-income countries. Exhibit 5 shows how the U.S. health system is a substantial outlier when it comes to achieving value. Despite spending nearly twice as much as several other countries, the country’s performance is lackluster. This report points to several areas that the U.S. could improve, building on recent health reforms, to achieve better performance. The success of U.S. initiatives to reduce readmissions and hospital-acquired conditions suggest the country’s health care can be improved. To gain more than incremental improvement, however, the U.S. may need to pursue different approaches to organizing and financing the delivery system. These could include strengthening primary care, supporting organizations that excel at care coordination and moving away from fee-for-service payment to other types of purchasing that create incentives to better coordinate care. These steps should ensure early diagnosis and treatment, improve the affordability of care, and ultimately improve the health of all Americans.


Lessons for the United States

Comparing countries’ health care system performance using standardized performance data can offer benchmarks and other useful insights about how to improve care. Among the 11 countries we studied, the U.S. was ranked last in overall health system performance, while spending the most per capita on health care. The insurance, payment, and delivery system of the ACA have improved some aspects of health care system performance, but the U.S. still greatly lags countries with universal health insurance coverage. The top performing countries—the U.K., Australia, and the Netherlands—could offer important lessons to the U.S. and other countries.

The Health Systems Achieving Top Marks Do So in Diverse Ways

The three countries with the best overall health system performance scores have strikingly different health care systems. All three provide universal coverage and access, but do so in different ways, suggesting that high performance can be achieved through a variety of payment and organizational approaches.

Experts generally group universal coverage systems into three categories: Beveridge systems, single-payer systems, and multipayer systems. These three systems are represented among our highest performers.

The U.K.’s National Health Service

The Beveridge model takes its name from the creator of Britain’s modern welfare state, William Beveridge. In the NHS, initiated by Aneurin Bevan in 1948, health services are paid for through general tax revenue, as opposed to insurance premiums. Furthermore, the government plays a significant role in organizing and operating the delivery of health care. For example, most hospitals are publicly owned, and the specialists who work in them are often government employees. This is not true of all providers. Most general practitioner practices are privately owned. Health care in the U.K. and other Beveridge countries is centrally directed and has more direct management accountability to the government than in other health systems.

Australia’s Single-Payer Insurance Program

In Australia, everyone is covered under the public insurance plan, Medicare. Much like the NHS, Australia’s Medicare is funded through tax revenue. Medicare is distinguished, though, by lesser public involvement in care delivery. Many Australian hospitals are private, and roughly half the population purchases private health insurance to access care outside the public system. To put into an American context, Australia’s Medicare resembles Medicare in the U.S.

Netherland’s Competing Private Insurers

Unlike Australia and the U.K., the Dutch health system relies on private insurers to fund health services for its population. Dutch insurers are mainly financed through community-rated premiums and payroll taxes, which are pooled and then distributed to insurers based on the risk profile of their enrollees. All plans include a standard basic benefit package; subsidies are available for people with low incomes; adults are required to enroll in a plan or must pay a fine. Dutch health care providers are predominantly private. This multipayer system—partly inspired by the managed competition model—shares many similarities with the insurance marketplaces created under the Affordable Care Act. 10 A. C. Enthoven, “The History and Principles of Managed Competition,” Health Affairs, Jan. 1993 12(Suppl. 1):24–48.


How We Measured Performance

Care Process

Care Process encompasses four subdomains relevant to health care for the general population: preventive care, safe care, coordinated care, and engagement and patient preferences.

The preventive care measures include four survey items related to counseling by health professionals on healthy behaviors, two OECD measures of mammography screening and influenza vaccination, and three OECD measures of rates (age- and sex-standardized) of avoidable hospital admissions for three prevalent chronic conditions: diabetes, asthma, and congestive heart failure.

Safe care includes three survey items: two indicators of safe care based on patient reports of experiencing medical, medication, or laboratory mistakes, and failure to receive effective prescription medication management, as well as one measure indicating whether primary care doctors use electronic clinical decision supports in their practice to improve safety.

Coordinated care uses seven measures to summarize timely sharing of information among primary care clinicians, specialists, emergency departments, and hospitals. It includes three physician-reported measures of effective communication among primary care clinicians and home care and social service providers.

Engagement and patient preferences represents 10 measures that evaluate the degree to which doctors and other health professionals deliver patient-centered care, which includes effective and respectful clinician–patient communication and care planning that reflects the patient’s goals and preferences.

Access

Access encompasses two subdomains: affordability and timeliness. The six measures of affordability include patient reports of avoiding medical care or dental care because of cost, having high out-of-pocket expenses, facing insurance shortfalls, or having problems paying medical bills. One measure reflects primary care doctors’ views of the difficulty patients face in paying for care.

Timeliness includes nine measures (three of which are reported by primary care clinicians) summarizing how quickly patients can obtain information, make appointments, and obtain urgent care after hours. It also addresses the length of time needed to obtain specialty and elective nonemergency surgery.

Administrative Efficiency

Administrative Efficiency includes seven measures. Four measures evaluate barriers to care experienced by patients, such as limited availability of the regular doctor, medical records, or test results. Three indicators measure patients’ and primary care clinicians’ reports of time and effort spent dealing with paperwork, as well as disputes related to documentation requirements of insurance plans and government agencies.

Equity

Equity compares performance for higher- and lower-income individuals within each country, using 11 selected survey measures from the Care Process and Access domains. The analysis stratifies the surveyed populations based on reported income (above-average vs. below-average relative to the country’s median income) and calculates a percentage-point difference in performance between the two groups. A higher percentage-point difference—that is, a bigger gap—is interpreted as a measure of lower equity among income groups in that country.

Health Care Outcomes

The Health Care Outcomes domain includes nine measures of the health of populations. Taken together, they are intended to reflect outcomes that are attributable to the performance of the countries’ health care delivery systems. The measures fall into three categories: population health outcomes (i.e., those that reflect the chronic disease and mortality of populations, regardless of whether they have received health care), mortality amenable to health care (i.e., deaths under age 75 from specific causes that are considered preventable in the presence of timely and effective health care), and disease-specific health outcomes measures (i.e., mortality rates following stroke or heart attack and the duration of survival after a cancer diagnosis).

In the population health outcomes category, two measures compare countries on the mortality of populations defined by age (infant mortality and life expectancy after age 60) and one measure focuses on the proportion of surveyed nonelderly adults who report at least two of five common chronic conditions. For each country, mortality amenable to health care includes both the current rate of deaths amenable to care and the 10-year trend. In the disease-specific health outcomes category, two measures focus on 30-day in-hospital mortality following myocardial infarction and stroke, and two measures examine five-year relative survival for breast cancer and colon cancer.

How This Study Was Conducted

This edition of Mirror, Mirror reflects refinements to methods used in past reports. No report can claim to capture every aspect of the performance of health care systems. Health care systems are complex. Even if a report included thousands of measures, nuances would remain. In that spirit, the report underwent a thorough review by an advisory panel of international, independent performance measurement experts. 11 Members of the advisory panel include: Marc Elliott, M.A., Ph.D., Distinguished Chair in Statistics and Senior Principal Researcher, RAND Corporation; Niek Klazinga, M.D., Ph.D., Head of the Health Care Quality Indicators (HCQI) Project, Organisation for Economic Co-Operation and Development Health Division; Ellen Nolte, Ph.D., M.P.H., Hub Coordinator, European Observatory on Health Systems and Policies, London School of Economics and Political Science, London School of Hygiene and Tropical Medicine; Rosa Suñol, M.D., Ph.D., Director of Avedis Donabedian Research Institute, Universitat Autònoma de Barcelona and Red de Investigación en Servicios de Salud en Enfermedades Crónicas (REDISSEC), Spain; and Dana Gelb Safran, Sc.D., Chief Performance Measurement & Improvement Officer and Senior Vice President, Enterprise Analytics, Blue Cross Blue Shield of Massachusetts. The framework for Mirror, Mirror 2017 was developed in consultation with the advisory panel from January through December 2016.

Using data available from Commonwealth Fund international surveys of the public and physicians and other sources of standardized data on quality and health care outcomes, we identified 72 measures relevant to health care system performance, organizing them into five performance domains: Care Process, Access, Administrative Efficiency, Equity, and Health Care Outcomes. The criteria for selecting measures and grouping within domains included: that the measure be important, that the data to support the measure be standardized across the countries, and that the results be salient to policymakers and relevant to performance improvement efforts. Most of the measures are based on surveys designed to elicit the public’s experience of its health care system.

The indicators were carefully selected from among the best-available measures with comparable data across the included countries. The selected measures cover a wide range of performance domains. Mirror, Mirror is unique in its use of survey measures designed to gather the perspectives of patients and professionals—the people who experience health care directly in each country every day.

Data

The data for this report were derived from several sources. Survey data are drawn from the 2014, 2015, and 2016 Commonwealth Fund International Health Policy Surveys. Since 1998, in collaboration with international partners, the Commonwealth Fund has supported these surveys of the public’s and primary care physicians’ experiences of their health care systems. Each year, in collaboration with researchers in the 11 countries, a common questionnaire is developed, translated, adapted, and pretested. The 2016 survey was of the general population; the 2014 survey surveyed adults age 65 and older. The 2016 and 2014 surveys examined patients’ views of the health care system, quality of care, care coordination, medical errors, patient–physician communication, waiting times, and access problems. The 2015 survey was administered to primary care physicians, and examined their experiences providing care to patients, the use of information technology, and the use of teams to provide care.

The Commonwealth Fund International Health Policy Surveys (2014, 2015, 2016) are nationally representative samples drawn at random from the populations surveyed. The 2014 and 2016 surveys sampling frames were generated using probability-based overlapping landline and mobile phone sampling designs and in some countries, federal registries; the 2015 survey was drawn from government or private company lists of practicing primary care doctors in each country. Appendix 7 presents the number of respondents and response rates for each survey, and further details of the survey methods are described elsewhere. 12 R. Osborn, D. Squires, M. M. Doty, D. O. Sarnak, and E. C. Schneider, “In New Survey of 11 Countries, U.S. Adults Still Struggle with Access to and Affordability of Health Care,” Health Affairs Web First, published online Nov. 16, 2016; R. Osborn, D. Moulds, E. C. Schneider, M. M. Doty, D. Squires, and D. O. Sarnak, “Primary Care Physicians in Ten Countries Report Challenges in Caring for Patients with Complex Health Needs,” Health Affairs, Dec. 2015 34(12):2104–12; and R. Osborn, D. Moulds, D. Squires, M. M. Doty, and C. Anderson, “International Survey of Older Adults Finds Shortcomings in Access, Coordination, and Patient-Centered Care,” Health Affairs Web First, published online Nov. 19, 2014.

In addition to the surveys, other standardized comparative data were drawn from the most recent reports of the Organization for Economic Cooperation and Development (OECD), the European Observatory on Health Systems and Policies, and the World Health Organization (WHO). Our study included data from the OECD on screening, immunization, preventable hospital admissions, population health, and disease-specific outcomes. The WHO and European Observatory data were used to measure population health.

Analysis

We used the following approach to calculate performance scores and rankings for comparison:

Measure performance scores: For each measure, we converted each country’s result (e.g., the percentage of survey respondents giving a certain response or a mortality rate) to a measure-specific performance score. This score was calculated as the difference between the country result and the 11-country mean, measured in standard deviations. Normalizing the results based on the standard deviation accounts for differences between measures in the range of variation among country-specific results. A positive performance score indicates the country performs above the 11-country average; a negative score indicates the country performs below the 11-country average.

The 11 measures in the equity domain were derived from the 2016 population survey and calculated by stratifying the population samples based on reported income (above-average vs. below-average relative to the country’s median income). Performance scores were based on the difference between the two groups, with a wider difference interpreted as a measure of lower equity between the two income strata in each country.

Domain performance scores and ranking: For each country, we calculated the mean of the measure performance scores in that domain. Then we ranked each country from 1 to 11 based on the mean domain performance score, with 1 representing the highest performance score and 11 representing the lowest performance score.

Overall performance scores and ranking: For each country, we calculated the mean of the five domain-specific performance scores. Then, we ranked each country from 1 to 11 based on this summary mean score, again with 1 representing the highest overall performance score and rank 11 representing the lowest overall performance score.

Sensitivity Analyses

We tested the stability of this ranking method by running two tests based on Monte Carlo simulation to observe how changes in the measure set or changes in the results on some measures would affect the overall rankings. For the first test, we removed three measure results from the analysis at random, and then calculated the overall rankings on the remaining 69 measure results, repeating this procedure for 1,000 combinations selected at random. For the second test, we reassigned at random the survey measure results derived from the Commonwealth Fund international surveys across a range of plus or minus 3 percentage points (approximately the 95 percent confidence interval for most measures), recalculating the overall rankings based on the adjusted data, and repeating this procedure 1,000 times.

The sensitivity tests showed that the overall performance scores for each country varied, but that the ranks clustered within three groups (Exhibit 3). Among the simulations, the U.K., Australia, and the Netherlands were nearly always ranked among the three top countries; the U.S., France and Canada were nearly always ranked among the three bottom countries. The other five countries varied order between the 4th and 8th ranks.

Limitations

This report has several limitations. Some are related to the particulars of our analysis and some inherent in any effort to assess overall health system performance.

First, as described above, our sensitivity analyses suggest that the overall country rankings are somewhat sensitive to small changes in the data or indicators included in the analysis.

Second, despite improvements in recent years, the availability of cross-national data on health system performance remains highly variable. The Commonwealth Fund surveys offer unique and detailed data on the experiences of patients and primary care physicians. However, they do not capture important dimensions that might be obtained from medical records or administrative data. Furthermore, patients’ and physicians’ assessments might be affected by their expectations, which could differ by country and culture. In this report, we augment our survey data with other international sources, and include several important indicators of population health and disease-specific outcomes. However, in general, the report relies predominantly on patient experience measures. Moreover, there is little cross-national data available on mental health services and on long-term care services.

Third, we base our assessment of overall health system performance on five domains—Care Process, Access, Administrative Efficiency, Equity, and Health Care Outcomes—which we weight equally in calculating each countries’ overall performance score. In the past some have argued there are other important elements of system performance that should be considered as well, such as innovativeness or value. After consideration, and based on discussions with our advisory panel, we decided not to add new domains to the report. We believe our current five domains capture a sufficiently broad and comprehensive view of health system performance. In addition, there was a lack of meaningful data to assess these new domains.