RISE in Ethiopia: Research Overview (Technical)
Principal Investigator: Tassew Woldehanna (Addis Ababa University)
Contracting Organisation: University of Cambridge
Other Key Institutions: Ethiopian Development Research Institute, Institute of Educational Research at Addis Ababa University, Ethiopian Education Strategy Centre, Research for Equitable Access and Learning Centre (REAL), University College London Institute of Education, Cornell University, Stockholm University.
Primary school enrolment in Ethiopia has soared in recent years – increasing from 25 percent in 1994, to 94 percent in 2015, according to government figures. The number of students in all grades doubled over the same period, from 10 million to 20 million students – a number that is roughly equivalent to the entire population of Romania. Though the gains in enrolment have been characterised as remarkable, inequalities are evident, with urban areas better served than rural ones. Certain populations, such as the poorest girls, are particularly vulnerable, and disadvantages tend to compound.
The Ethiopia Country Research Team’s analysis will focus on the General Education Quality Improvement Package (GEQIP), an evolving, flagship education programme of reforms aimed at improving educational quality. Started in 2009, the reform package is now in a second phase, with a third anticipated. The complex and multi-faceted reforms underway in Ethiopia lend themselves well to the ‘systems-focused’ approach of the RISE initiative.
An important question is why, despite such an ambitious package of reforms, backed up by political will, learning outcomes in Ethiopia are still very low. This compels a better understanding of the effects of the changing composition of children in the classroom, and the extent to which these reforms are working at scale, and how to overcome any impediments to their success in order to raise learning outcomes for all.
Broadly, the research team will explore the nature of accountability within a large educational system, and among key actors – administrators at national, regional and local levels, teachers, parents, and students. Three key aspects are worth highlighting:
- Emphasising equity - One important feature of the research team’s plan is that it will closely examine the issue of equity, with an emphasis on understanding what improves learning for all populations. This is a particularly important challenge for Ethiopia, which has a very diverse population, among which some 83 different languages and 200 dialects are spoken.
- Using mixed methods - A core aspect of this research project is its combination of political analysis, and school-based case studies. This mix is intended to give the team the research tools to closely examine the issues that are crucial to gaining an understanding of Ethiopia’s education system and the levers that bring about – or stall – change.
- Taking a longitudinal approach - This research project is designed to be longitudinal in nature. Thus, it will allow the team to take a long-term view to understand the path that led Ethiopia to embark on such a wide-ranging reform package, and, further, to examine what the effects of such a large-scale programme are. Researchers plan to follow the same children in the same schools over a five-year period to measure changes in foundational skills in reading, vocabulary, numeracy, and non-cognitive skills, such as non-verbal reasoning and self-efficacy (self-belief in the ability to succeed or master certain tasks).
The research plan draws on the RISE conceptual framework outlined in Pritchett (2015), which emphasises the importance of coherence across accountability relationships within an education system. Individually, the related components of the Ethiopian reform package may provide only relatively marginal contributions to the quality of education offered in a school site, and to the overall management of the system. The theory is, however, that these come together to raise learning outcomes and to ensure both equitable improvement and increased access. These synergistic effects will be examined in the Ethiopia research.
Ethiopia’s large population lives in diverse agro-climatic conditions including pastoral, highland, and lowland areas. Ethiopia is divided into 10 regional states and two city administrations, based on the language people speak. In general, regions can be grouped into four categories: urban, pastoral emerging regions, western emerging regions and relatively developed rural regions. The sampling approach will aim to represent these four categories of regions in the country, and will include households, children, schools and communities selected from each of these categories, with a total of seven regions being sampled: Addis Ababa (representing the urban regions), Afar (representing the pastoralist emerging regions), BeniShangul Gumuz (representing other emerging regions) and Tigray, Amhara, Oromia, and SNNP (representing the more developed regions, where the majority of rural Ethiopian children live). These regions will be further stratified into woredas (the third-level administrative divisions of Ethiopia) based on rural/urban settings and distance from the main road or regional capital.
In studying the Ethiopian reform package, RISE researchers have three main objectives: to understand the reform-creation process, to evaluate the reforms’ success, and to inform future policy.
To understand the mechanisms through which the reforms were selected, designed and implemented, it is important to study the political economy context in which it the reform package was devised. The team aims to understand sources of path dependence using a political economy framework, interviews and documentary analysis. First, the team will conduct a ‘systems diagnostic’ to map out features and actors of reforms, and the potential for these features and actors to support - or impede -learning. The team will place reforms associated with the package within a wider context of one of five approaches – specifically looking at whether the changes involve a federal-level approach, sector-level approach, spatial approach (between levels), temporal approach (changes over time), and/or problem-focused approach (focus on obstacles).
Beyond the diagnostic work, the team will complement the findings with insights offered by analysis of existing data sources on the status of learning outcomes. Sources include: household and school surveys conducted by Young Lives, a long-term project investigating the changing nature of childhood poverty in four developing countries, including Ethiopia; household surveys from Ethiopia’s Productive Safety Net Programme, part of the World Food Programme; and data from Ethiopia’s National Learning Assessment (NLA); the World Bank’s Living Standards Measurement Study (LSMS); and the government’s Education Management Information Systems (EMIS). These detailed datasets will help inform the next stage of the research plan, particularly focusing on how learning outcomes differ across population groups and where these differences lie.
Rigorous evaluation of programmes often aim to emulate experimental designs as much as possible. The research aims to evaluate the effectiveness of the reforms. In designing a methodology to meet this objective, the team faces several challenges: first, the reforms are ongoing, and implementation is not expected to be complete until July 2018. Second, the individual reforms that make up the package are not mutually exclusive. Thus, identifying any individual effects from bundled interventions may not be possible. In any case, the purpose of the analysis is to assess the effects of a package of reforms, recognising this is not straightforward methodologically. However, by having detailed interviews at both community and individual levels, it is possible to know which reform may be contributing to any observed impact.
The team will test two cohorts of students in grade one and grade four at the beginning and end of the 2018/9 school year, and will use these data to capture ‘value added’ in learning. Each cohort will have been exposed to all or some of the reforms. Researchers aim to survey a total of 8,000 students (4,000 boys and 4,000 girls) across 100 schools in grades one and four. This will be repeated with the same cohorts of children in 2021/2.
School and household-survey data would allow for controls of time-varying factors, such as child illness, income shocks (such as, say, due to droughts), or changes in infrastructure (roads/accessibility to schools). Although longitudinal studies can be marred by sample attrition, the team’s researchers have substantial collective experience with such studies to ensure the risk will be as low as possible, and the analysis plan accounts for such possible issues. To augment their research plan, the researchers will also look at the feasibility of using the Young Lives survey data in the analysis as a comparison group – that is, including in their analysis children who were not exposed to the reforms in 2012.
The team will take steps to address the possible issue that reforms are not mutually exclusive, and the difficulty in extricating individual effects given the bundled nature of the reforms. To complement the quantitative data, researchers will carry out detailed interviews at both community and household levels to understand which reforms were, in fact, selected and implemented, and the extent of each cohort’s exposure to given reforms. These case studies will enable the team to dig deeper into the possible mechanisms behind the estimated effects.
Ultimately, the goal of the research plan is to understand and evaluate the reforms to inform future policy. The team considers detailed case studies as the best means to identify the most consequential elements of the reform package. The team will use case studies to map out the conditions on which reforms depend, and to understand how the relationship between different actors influences the success or failure of various reforms’ implementation. Ideally, the research will help identify the necessary conditions for exceptional outcomes, and will pinpoint the system-level changes that can support replication of these conditions elsewhere.
The case studies will focus particular attention on the processes of the political economy of reform. It is crucial to understand how and why certain strategies were chosen, the political interests driving these choices, and the conditions under which different components of the reform package are – or are not - implemented successfully. The team will conduct a set of qualitative interviews with key stakeholders to collect the necessary data for this exercise.
The qualitative interviews will consider the different perspectives of three categories of actors at the school level: principals, teachers and students. A suite of qualitative approaches will be employed, including individual interviews, focus groups, time-use diaries, community mappings, and related methods used by the Young Lives project.
Key relationships of accountability within Ethiopia’s education system
In summary, the Ethiopia Country Research Team proposes a multi-disciplinary approach including mixed methods to take a holistic view of the reforms. The team intends to make use of available datasets, and to engage in primary data collection where necessary. The researchers plan to tackle a challenging set of questions in the quest to understand ‘what works’ for education reform in the context of Ethiopia, and to gain insights that may benefit the quest to improve learning worldwide.
- Teacher education, development and retention - In June 2016, the Government of Ethiopia announced salary increases for all teachers – increasing average salaries by roughly 25 percent. The increase was intended to boost teachers’ motivation, and to bring skilled people who might have otherwise sought other employment into the teaching force. In addition, the government established new training programmes to see whether it would lead to improved student learning.
- Pre-school to prepare for formal schooling - Nationally, all primary schools are required to introduce a year of pre-primary school in hopes of helping more children to prepare. The change was prompted by research showing that more than a quarter of Grade 2 students were unable to read a single word correctly.
- New teaching methods – Ethiopia initiated reforms intended to improve reading skills of young elementary school children (Grades 1-4) in some 2,600 schools. The reforms replaced traditional lectures with more active teaching methods, and regularly tracked students’ progress. In addition, teachers received reading materials and training in new techniques. .
- Feeding children at school – In Ethiopia and throughout the world, low learning levels are closely linked with poverty and malnutrition. To help retain students in school, the government is implementing a programme to feed children who might otherwise go to school hungry. Though the humanitarian aid agency, the World Food Programme, has been in operation in Ethiopia for many years, the Ethiopian government’s national programme, which began in the 2016/17 school year, was expected to reach millions more students. A key determinant of the programme’s success – or failure – will stem from headteachers’ decisions regarding how to use funds provided to ensure the provision of food for the most disadvantaged. The programme has the potential to work through two routes: by leading to more consistent attendance, and by improving children’s ability to concentrate and learn.
- School grants – As part of the school improvement programme, schools receive cash grants for investments and purchases based on schools’ enrolment levels. The reforms also offer local school administrators training in leadership skills, guidance on evidence-based planning, and information about the importance of transparency, in particular, regarding the sharing of information on student achievement with the local community.