Blog

Inequality of Knowledge Acquisition and Learning Assessments: Insights from the Data

Authors

Image of Luis Crouch

Luis Crouch

RISE Directorate

RTI International (Emeritus)

The RISE Programme sees equity in education as fundamental. Key stakeholders in the international education community are re-focusing on learning and the equality (or inequality) of its distribution. The Sustainable Development Goals, more so than the Millennium Development Goals, put explicit emphasis on equity. The UK’s Department for International Development, the main sponsor of RISE, has also had a long-term interest in equity.

Arguably, the equity aspects that matter most in education are those that pertain to the life chances that education creates. Getting children into school and staying there is a first, relatively easy, step. But getting children to learn a lot, and reasonably equally, is a tough act. It is an important one, though, because it helps drive equality of life chances.

It is natural that the RISE Programme would be interested in what we know about the equality or inequality of learning in societies. Does inequality increase as average learning levels increase, because it is just inevitable that some will be left behind as the average goes up? Or on the contrary, is it that without “bringing up the bottom” it is really hard to push up the average? Caine Rolleston and myself took a first look at those issues for RISE, in an Insight piece whose title carries the key message: “Raising the Floor on Learning Levels: Equitable Improvement Starts with the Tail.”

A new RISE working paper, Worldwide Inequality and Poverty in Cognitive Results: Cross-sectional Evidence and Time-based Trends (which I co-authored with Martin Gustafsson), now digs deeper into the question, using almost all of the available evidence across all grades and subjects in the most recent round of the widely-used learning assessments: PISA, TIMSS, PIRLS, SACMEQ, and TERCE.*

It is hard to say whether inequality increases or decreases as average performance goes up. Some countries with high average performance have more inequality, others less. So, the relationship is often weak or statistically unreliable. Furthermore, on some assessments there is a positive relationship: the higher the average score, the greater the inequality of scores. But on other assessments, the opposite is the case. This pattern seems somewhat to depend on the testing institution. There may be methodological reasons, related to testing, that partly account for this ambiguity of results. This is a topic for further research.

However, the relationship between the average performance of countries, and the number of children “left behind” in the very low levels of performance, was very clear, and strongly confirms the findings in the RISE Insight noted above. We found a strong empirical pattern that was common across almost all of the assessments: in progressing from very low overall levels of performance to middling ones, successful countries mostly make sure that very few children are left at the lowest levels of performance. The number of children with very high levels of performance is hardly increased. Interestingly, in moving from middling overall performance to high overall performance, the opposite is the case: the number of children with high performance increased significantly, while the number of children with very low performance was not reduced by much. Perhaps this is because for the really successful countries, it is difficult to reduce even further the numbers of children with extremely low performance, as these numbers are so low already.

We found that this pattern holds both when comparing countries to each other at a point in time (letting differences at a point in time stand in for changes over time) and when comparing countries to themselves over time. Countries that progress do so mostly by making sure that pockets of very low performance are reduced to zero or nearly zero.

The analogy with economic growth would be that the number of children with very low performance generalizes to economic poverty. In the case of education, one could say by analogy, poverty reduction and growth go hand in hand. What the successful countries have actually done to generate this result should be of great interest to an education systems research programme like RISE.

We also found that these results hold regardless of whether one measures learning outcomes using relatively simple methodologies such as the percent-correct on tests, or more sophisticated techniques that take into account how hard different questions are.

Finally, we found that a little less than half of the worldwide inequality in learning is between countries, and a bit more than half is within countries. This means that, for international agencies, it makes sense to target certain countries as a whole, but it makes just as much sense (if not more) to help those countries address their own (large) pockets of very low performance.

In short, the paper’s results are fairly optimistic in reinforcing the message that averages can be (and most importantly empirically tend to be) improved by taking care of those learning least, but also raise important questions for further analysis.

 

* PISA (Programme for International Student Assessment), TIMSS (Trends in International Mathematics and Science Study), PIRLS (Progress in International Reading Literacy Study), SACMEQ (The Southern and Eastern Africa Consortium for Monitoring Educational Quality), TERCE (Third Regional Comparative and Explanatory Study)

Author bios:

RISE blog posts and podcasts reflect the views of the authors and do not necessarily represent the views of the organisation or our funders.