The controversy of composite indices

GDP has long been the single number that is most used to gauge living standards around the globe. As the world becomes increasingly conscious of the shortcomings of our best current measures of progress, efforts are underway to create new ways of measuring the human condition—ones that are more conscious of social and environmental factors but can still be represented in a single, attention-capturing number. Composite indices like UNDP’s Human Development Index (HDI), the OECD Better Life Index and Yale’s Environmental Performance Index attempt to do just that—capturing a wider range of factors that contribute to human quality of life. But Martin Ravallion of the World Bank is deeply skeptical of such indices. In his 2010 article, Ravallion argues that:

Countries are increasingly being ranked by some new “mashup index of development,” defined as a composite index for which existing theory and practice provides little or no guidance to its design. Thus the index has an unusually large number of moving parts, which the producer is essentially free to set. The parsimony of these indices is often appealing — collapsing multiple dimensions into just one, yielding unambiguous country rankings, and possibly reducing concerns about measurement errors in the component series.[1]

A tool to start the conversation

Ravallion’s concern is justified. He notes that the tradeoffs implied in such indices can sometimes be troubling–in a previous article he finds that the implied value of human life in the HDI ranges from unacceptably low in poor countries to many times GDP in wealthier ones. But I’d argue that so long as composite indices are used as the jumping off point for a deeper conversation, much can be gained from harnessing their attention-capturing power.

Composite indices have an important role in getting the conversation started: in the age of twitter, attention spans tend towards 140 characters and few policymakers have the time to sit down and read a policy brief. However, in a few seconds one number from a composite index—the country’s rank among peers—sends a powerful message. Dig just one layer deeper into an index’s component scores and a it flags areas where a country does relatively poorly, so that policymakers can focus efforts on improving their worse performance vis a vis peer countries. Composite indices are unique among research products in their ability to capture even short attention spans and catalyze policy action.

Composite indices are nonsense

In a certain way, composite indices are nonsense. They bring together an assortment of issues into a single, unitless measure. But they reflect the nonsense of our reality: that policymakers are expected to balance many competing priorities determined by an abstract concept of social welfare and that are essentially non-substitutable. The extent to which composite indices are nonsense is only the extent to which they reflect the real, difficult choices that policymakers must make. And if composite indices are used as one tool among many—as a snapshot of policymaker performance across the issues that experts think matter most, and a tool for focusing efforts—they can provide valuable context for policy action.

[1] Ravallion, M. (2010). “Mashup indices of development.”


Closing the gap at the top: What should the Gender-GEDI’s high scoring countries do to improve?

New Picture

During one of our expert meetings for the Gender-GEDI female entrepreneurship index we were talking with Anne Simmons-Benton of DAI about how we can explain the gender gap in entrepreneurship in developed nations. One theory that was offered based on her experience training female entrepreneurs is that women frequently lack the confidence to succeed – often based on a lack of role models and support – which then becomes a deeply culturally embedded phenomenon.

This is something we see in the low scores for Germany and France on willingness to start a business: both countries are economic leaders, but for some reason a relatively large portion of women who could otherwise become entrepreneurs aren’t choosing to start businesses for fear of failure.

This shows that even in high-income, relatively progressive countries social norms can still exert a powerful influence, holding back innovators that would otherwise choose to become entrepreneurs.

For some developing nations it’s easy to look to institutional factors like access to finance and equal legal rights to explain some of the entrepreneurship gender gap. But in Western Europe women for the most part enjoy institutional equality. Instead, in countries where the tangible barriers to entrepreneurship have been knocked down, the less-tangible, internal and socially constructed barriers create the bottleneck. Where to go from here?

Interestingly, both France and Germany also score relatively low on “know an entrepreneur,” a measure of the percentage of the female population who personally know an entrepreneur who started a business within the last two years. One way for these countries to make progress would be to foster female entrepreneurial mentoring and support networks. As the numbers of female entrepreneurs grow, so too will the networks that support and inspire them, creating positive feedback that encourages more women to become entrepreneurs and that fuels economic growth.

The Gender-GEDI was sponsored by Dell and launched at the Dell Women’s Entrepreneurship Network meeting in Istanbul on June 4, 2013.

The EPI informs new measures of progress and helps countries gauge performance at the Rio+20 Earth Summit

The environment doesn’t count. Or at least this is the message that GDP numbers send. But as the world becomes increasingly conscious of the shortcomings of our best current measures of progress, efforts are underway to create new ways of measuring the human condition—ones that are more conscious of social and environmental factors. The EPI is part of this effort and is helping policymakers understand the environmental conditions of their countries, highlighting areas where they need to focus efforts to improve. At the Rio+20 Earth Summit last month, the EPI and YCELP were an active part of the dialogue

While the final Outcome Document of the conference recognizes the need for “broader measures of progress to complement GDP” so as to “better inform policy decisions,” it lacks specifics as to what future measures could or should be.   In Rio, Armenia hosted a side event, “Sustainable Development Indices – possible options,” to discuss potential options, including the EPI.  Armenia has also been working since 1995 to incorporate a sustainable development component to the Human Development Index (HDI) – a widely used counter-measure of human progress. The HDI attempts to create a summary measure of human development across three basic dimensions of human development: health, education, and income but lacks mention of the environment or sustainable development.   You can read here about EPI Project Director Angel Hsu’s report on the EPI participation in the Armenian government’s side event.

In addition to defining new measures of progress, the EPI played an important role in Rio in terms of allowing countries to evaluate both current and past environmental results. At the high-level ministerial plenary sessions June 20-22, Latvia’s Minister of Environment referenced their country’s ranking on the 2012 EPI to support statements made that they have been implementing environmentally-sound policies to address declining growth and high rates of unemployment. UNDP Administrator Helen Clark named the EPI as one of several model efforts for measuring sustainability during her opening statement at the UNDP’s Beyond GDP: Measuring the Future We Want event on June 20.

In these ways, we saw that the EPI and similar efforts are helping countries begin to think about ways of measuring progress and decline.  This ability of countries to establish clear metrics of performance will be critical for tracking achievement of Sustainable Development Goals (SDGs), a set of universal targets that are meant to integrate development and the environment in a way that the SDGs’ predecessor, the Millennium Development Goals, do not.

Measuring environmental performance is only the first step in catalyzing progress towards the SDGs. YCELP is also seeking a deeper understanding of how countries measure and achieve progress. On the ground this meant that the team was on the lookout for examples for the Indicators in Practice project—an effort to compile and present examples of best practices in the use of indicators around the world to catalyze environmental progress. The team met Ian Drysdale and Jennifer Myton at the Healthy Reefs Initiative, who discussed their efforts to improve reef stewardship in the Caribbean basin. The team also met Daniele Giovannucci at COSA (the Committee on Sustainability Assessment), “a consortium of institutions developing and applying an independent measurement tool to analyze the distinct social, environmental and economic impacts of any agricultural practices, particularly those associated with the implementation of specific sustainability programs.” From reefs to agriculture, environmental indicators are helping stakeholders understand the changes in the world around them, and to work towards bettering environmental conditions.

While the conference as a whole received mixed reviews, participants demonstrated high-level commitment to tracking not just the money in our bank accounts, but the natural resources that support both economies and human lives. It is our hope that the EPI can continue to play an important role in the process of achieving sustainable development goals and defining new measures of progress.

Digging deeper into time trends in international trade and CO2 emissions

In a recent study on trade and the environment, the Yale Center for Environmental Law & Policy conducted a pilot time trend analysis, painting a clearer picture of the complex relationship between country-level CO2emissions and trade intensity. The analysis examines changes in trade intensity (as seen through trade as a percent of GDP) alongside two measures of CO2 emissions—CO2 per capita and CO2 per GDP. The former CO2 measure indicates emissions intensity per person, while the latter indicates the emissions intensity of the economy.

Time trends show that the most common phenomenon is a decline in emissions per GDP with increasing trade intensity, an indication that economies become more carbon efficient as trade becomes a greater portion of GDP. However, the data also reveal a troubling trend among countries: emissions per capita most commonly increase with increasing trade, an indication that trade might be harmful in terms of emissions.

co2 per capita co2 per gdpco2 per gdp

The exciting news—at least for those interested in trade and environmental quality—is that this trend is by no means universal. Many countries have managed decreasing emissions per capita with increases in trade intensity. This elite group includes many European nations—Germany, France, the UK, Sweden and Denmark—as well as several countries in the Americas—Belize, Colombia, and Cuba.

While considerable work is still needed help deepen understanding of the complicated relationships between trade and the environment and while the statistical findings do not support drawing conclusions about causation, it is interesting to pose the question – what are these nations doing differently?

Materials for the study can be downloaded below:
Executive Summary.
Press Release.

Full Report.