RISE in Pakistan: Research Overview (Technical)

Principal Investigators: Tahir Andrabi (Pomona College), Jishnu Das (World Bank), Asim Khwaja (Harvard Kennedy School)

Contracting organisation: Presidents and Fellows of Harvard College

Student test scores in Pakistan remain low, with students in Grade 5 barely at Grade 2 competency. This is happening despite increases in government spending, enrolment and parental involvement in education. The Pakistan team proposes a set of studies and activities to shed light on the nature of the frictions within the Pakistani education system, and to help build local capacity for the sustainability of continued monitoring and long-term improvement of the system. The team has strong local networks with established and well-regarded partners in Pakistan that will help the implementation of this ambitious research project. 

The team sees the system as a market, and seeks to study different frictions along five key dimensions, depicted in Figure 1: (a) access to information; (b) access to resources and financing; (c) knowledge and innovation markets; (d) labour market incentives; and (e) regulatory and governance structure.

For each of the frictions there is a set of studies proposed to understand how an alleviation of this friction affects student learning and the relationships of accountability between the actors in the system. Crucially, the greatest advantage of conducting this set of studies with RISE support is that it breaks with conventional funding schemes that force projects to be sequential. In this instance, all proposed studies can happen concurrently rather than sequentially, allowing for a study design that exploits synergies between the studies, and gives a more system-level outcome.

A large part of this proposal is also an ambitious data collection project. The new data collected will enable researchers to consider issues raised in this proposal and beyond, including focusing on gender gaps in learning and across socio-economic backgrounds (as there is a focus on both public* and private schools). 

The proposed research for each market friction is detailed below, under the relevant heading:

(a) Access to information: provide key information that helps parents and schools to recognise and rewards the best schools.

Studies show that public information on school quality improves school performance across public and private systems. This study seeks to understand how this happens and the impact of different types of information.

  1. Long-term impact of report cards: look at the long-term impact of information provision on educational markets, especially school and child impacts, taking advantage of a previous intervention providing child and school level report cards in 2004.

(b) Access to resources and financing: help schools access the tools they need to improve quality and innovate. Schools need resources to implement learning innovations. This proposal includes three projects to look at this question.

  1. Public School Funding: examine the direct (short- and long-term) impact of a reform by the Punjab Education Department that started in 2004, which brought about a large increase in locally available funds in public schools (on the order of USD 2-4,000 per school). Preliminary RCT results show positive direct impacts in public schools, as well as positive spillover effects to private schools. 

  1. Low Cost Private Schools Grants Study: village-level RCT of cash grants (USD 500) to low-cost private schools that examines intent-to-treat effects of grants on school revenues, fees, enrolment and child test scores. In addition, we will examine whether the returns to capital differ based on how many schools in a village have access to increased resources. Preliminary results suggest a positive return on the grants and a “crowd-in” effect when grants are offered to all schools in a village. 

  2. Education Financing: sustainable impact in alleviating financial resources faced by schools can only be addressed by having mainstream financial providers offer products that are designed to serve the needs of these schools. The team has created two bespoke financial products in collaboration with Tameer Microfinance Bank, and will evaluate the likely take-up rates of these products and affecting factors, as well as the impact of these products on schools’ revenues, fees, enrolment, child learning and the educational ecosystem as whole through an RCT. 

(c) Alleviating knowledge and innovation failures to increase access to education support services: provide the knowledge and innovation markets that lead to mature markets for educational support services (ESS).

A resource for growth is access to knowledge and innovations. There are no existing markets for soft investments such as curriculum development, school assessments, and teacher training designed to fit the needs of schools in Pakistan. Thus, even schools committed to improving quality find it difficult to access these ESS. The team proposes one study to look at this question.

  1. Education Support Services: Partner with leading ESS providers to create, test and refine a marketplace for ESS to study the impact of alleviating the ESS market failure. To take a system-approach, the team proposes to cross-randomise with the education financing study above. This way they can evaluate the take-up given the current financial constraints and when these constraints are alleviated via credit products.

(d) Focus on the labour market for teachers: ensure the labour market incentives link educational returns to investments, and produces well-trained and motivated teachers.

Reforms thus far have not been tested in meaningful ways. The team proposes to partner with local stakeholders responsible for the training and hiring of teachers to carry out three studies.

  1. Labour market returns to education: use longitudinal datasets from the LEAPS study, including the ethnographic study of the families involved to examine the impact of schooling on early adult labour force outcomes, occupational choice and family formation stratified by gender and access to schooling heterogeneity. 

  2. Teacher labour market: examine the connection between pay and productivity using LEAPS data and teacher value-added modelling. Use a natural experiment involving hiring public sector contract teachers at half the standard wage.

  3. Teacher recruitment, compensation and retention: evaluate Beachonhouse administrative database and conduct focus groups to produce a descriptive study examining the details of the teacher evaluation process. This will include the performance feedback system, teacher reward structures, teacher recruitment, movements within the school system, and attrition. 

(e) Enabling a Conducive Political Economy Environment: examine governance and political economy dynamics and their effects on the system.

Although the themes explored in the other headings look at a diverse set of system frictions, it is also critical to understand the broader political economy encapsulating the system. Thus, to examine the dynamics of governance and the political economy, the team proposes a two-part study.

  1. Historical documentation: use key informant interviews and archival work to document the contrasting educational governance structures for Punjab and Khyber Pakhtunkhwa.

  2. Institution case study: Look at the regional examination system and examination boards to understand how they work. Include dialogue with government officials to foster debate around these issues.

The results from this set of studies will have immediate impact on policies in similar environments, particularly in South Asia, where private schools are a rapidly growing presence. But more broadly, the questions that are addressed in this research agenda respond to the age-old issue of what we ultimately want from our schooling systems, and whether one way to improve schooling is to reduce the frictions in the basic relationship between the parent, the school, and the child.

 

* In the RISE lexicon a public school refers to one operated by a government at little or no cost to students. This is to distinguish the definition of public schools as used in England and Wales, where the term refers to selective, and expensive, independent secondary schools.