The United States has long prided itself as being an exceptionally fluid society with respect to social class and economic mobility. The American Dream holds that anyone who works hard can achieve economic success – perhaps even rise from rags to riches.

Underlying this belief is the assumption of abundant opportunity and meritocracy. Arriving immigrants often believe they have come to a land of opportunity, with a level playing field allowing for advancement and success. Those who fail to do so tend to blame themselves.

Yet according to recent research, the United States has far less mobility and equality of opportunity today than the European Union or other Organization for Economic Co-operation and Development (OECD) countries.

The amount of economic advantage passed down from one generation to the next is much higher in the United States. Approximately 50% of a father’s income position is inherited by his son. In contrast, the amount in Norway or Canada is less than 20%.

What about rising from rags to riches? In the U.S., 8% of children raised in the bottom 20% of the income distribution are able to climb to the top 20% as adults, while the figure in Denmark is nearly double at 15%.

Equality of opportunity is also much less viable in the U.S. than in other OECD countries. American life expectancy varies by up to 20 years depending on the zip code of residence. Quality of education also differs widely depending on the wealth of the neighborhood that families reside in. And the chances of being victimized by a crime, exposed to environmental toxins or having unmet healthcare needs is far greater for America’s poor than those impoverished in all other OECD countries.

One of the reasons for lower U.S. mobility is that the ladder of opportunity has become much harder to climb – because the rungs of the ladder have grown further apart. This is evidenced by the rising levels of income and wealth inequality.

Currently, those in the top 20% of the income distribution earn nearly nine times more than those in the bottom 20%. This difference is far greater than in the European Union or the United Kingdom. Wealth inequality is even more skewed. In the United States, the top 5% of the population own three-quarters of the entire financial wealth of the country, while the bottom 60% possess less than 1%.

We discuss one explanation for these trends in our book, Poorly Understood: What America Gets Wrong about Poverty. The United States has traditionally viewed economic success and failure as the result of individual effort. Rugged individualism and self-reliance have been defining qualities of the American character.

On the other hand, European nations are much more likely to attribute poverty to structural factors such as social class or the lack of jobs. As a result, other OECD countries are much more willing to invest in a robust social welfare state designed to help ameliorate some of these structural inequities.

In addition, the United States has frequently viewed poverty and inequities through the lens of race and ethnicity. Poverty is often seen as a “Black problem” rather than as an “American problem.” Race has been used to divide poor blacks and whites from seeing their common economic interests.

As President Lyndon Johnson once explained to an aide in 1960, “I’ll tell you what’s at the bottom of it. If you can convince the lowest white man that he’s better than the best colored man, he won’t notice you picking his pocket. Hell, give him somebody to look down on, and he’ll empty his pockets for you.”

Research has shown that more racially heterogeneous societies tend to be less generous in their economic redistribution policies to address structural inequities. The reason for this may be that we tend to be less concerned about the needs of others when they look different from us. On the other hand, countries that are racially homogeneous tend to have much more robust social safety nets.

With a new president in the White House, let us hope that more progressive policies will begin to restore the viability of the American Dream and reverse the trends towards less mobility and opportunity. The American Dream should reflect a reality where every child has the opportunity to pursue their passion in order to reach their full potential. Such a dream is well worth investing in.

Mаrk R. Rаnk and Lаwrеncе M. Еppаrd

Kаrim Mаnjrа

Help deliver the independent journalism that the world needs, make a contribution of support to The Guardian.