Inequality of Knowledge Acquisition and Learning Assessments: Insights from the Data
The RISE Programme sees equity in education as fundamental. Key stakeholders in the international education community are re-focusing on learning and the equality (or inequality) of its distribution. The Sustainable Development Goals, more so than the Millennium Development Goals, put explicit emphasis on equity. The UK’s Department for International Development, the main sponsor of RISE, has also had a long-term interest in equity.
Arguably, the equity aspects that matter most in education are those that pertain to the life chances that education creates. Getting children into school and staying there is a first, relatively easy, step. But getting children to learn a lot, and reasonably equally, is a tough act. It is an important one, though, because it helps drive equality of life chances.
It is natural that the RISE Programme would be interested in what we know about the equality or inequality of learning in societies. Does inequality increase as average learning levels increase, because it is just inevitable that some will be left behind as the average goes up? Or on the contrary, is it that without “bringing up the bottom” it is really hard to push up the average? Caine Rolleston and myself took a first look at those issues for RISE, in an Insight piece whose title carries the key message: “Raising the Floor on Learning Levels: Equitable Improvement Starts with the Tail.”
A new RISE working paper, Worldwide Inequality and Poverty in Cognitive Results: Cross-sectional Evidence and Time-based Trends (which I co-authored with Martin Gustafsson), now digs deeper into the question, using almost all of the available evidence across all grades and subjects in the most recent round of the widely-used learning assessments: PISA, TIMSS, PIRLS, SACMEQ, and TERCE.*
It is hard to say whether inequality increases or decreases as average performance goes up. Some countries with high average performance have more inequality, others less. So, the relationship is often weak or statistically unreliable. Furthermore, on some assessments there is a positive relationship: the higher the average score, the greater the inequality of scores. But on other assessments, the opposite is the case. This pattern seems somewhat to depend on the testing institution. There may be methodological reasons, related to testing, that partly account for this ambiguity of results. This is a topic for further research.
However, the relationship between the average performance of countries, and the number of children “left behind” in the very low levels of performance, was very clear, and strongly confirms the findings in the RISE Insight noted above. We found a strong empirical pattern that was common across almost all of the assessments: in progressing from very low overall levels of performance to middling ones, successful countries mostly make sure that very few children are left at the lowest levels of performance. The number of children with very high levels of performance is hardly increased. Interestingly, in moving from middling overall performance to high overall performance, the opposite is the case: the number of children with high performance increased significantly, while the number of children with very low performance was not reduced by much. Perhaps this is because for the really successful countries, it is difficult to reduce even further the numbers of children with extremely low performance, as these numbers are so low already.
We found that this pattern holds both when comparing countries to each other at a point in time (letting differences at a point in time stand in for changes over time) and when comparing countries to themselves over time. Countries that progress do so mostly by making sure that pockets of very low performance are reduced to zero or nearly zero.
The analogy with economic growth would be that the number of children with very low performance generalizes to economic poverty. In the case of education, one could say by analogy, poverty reduction and growth go hand in hand. What the successful countries have actually done to generate this result should be of great interest to an education systems research programme like RISE.
We also found that these results hold regardless of whether one measures learning outcomes using relatively simple methodologies such as the percent-correct on tests, or more sophisticated techniques that take into account how hard different questions are.
Finally, we found that a little less than half of the worldwide inequality in learning is between countries, and a bit more than half is within countries. This means that, for international agencies, it makes sense to target certain countries as a whole, but it makes just as much sense (if not more) to help those countries address their own (large) pockets of very low performance.
In short, the paper’s results are fairly optimistic in reinforcing the message that averages can be (and most importantly empirically tend to be) improved by taking care of those learning least, but also raise important questions for further analysis.
* PISA (Programme for International Student Assessment), TIMSS (Trends in International Mathematics and Science Study), PIRLS (Progress in International Reading Literacy Study), SACMEQ (The Southern and Eastern Africa Consortium for Monitoring Educational Quality), TERCE (Third Regional Comparative and Explanatory Study)
Luis Crouch is the Chief Technical Officer at RTI’s International Development Group and a member of the RISE Intellectual Leadership Team. He specialises in education policy, decentralised finance (e.g., funding formulas), political economy of reform, education statistics, planning, and projections. He has experience in all key areas of education data analysis, from the generation of primary data via surveys and citizen input, to statistical and econometric analysis, to evidence-based, Cabinet-level policy dialogue. He has previously worked at the World Bank and at the Global Partnership for Education. He has worked closely on South Africa’s education sector funding reforms, Egypt’s decentralisation experiments, and decentralisation and other policy reforms in Peru and Indonesia. His more recent work is in early grade reading and Early Childhood Development, as key entry-points to improving quality. He has worked in more than 25 countries in a 30-year career in development, and is the author of reports, technical papers, and books.
RISE blog posts reflect the views of the authors and do not necessarily represent the views of the organisation or our funders.