Principal Investigators: Dr. Asim Ijaz Khwaja (Harvard Kennedy School), Dr. Ali Cheema (Institute of Development and Economic Alternatives), Dr. Farooq Naseer (Lahore University of Management Sciences), Dr. Jacob Shapiro (Princeton University)
Implementing Partner: Punjab Skills Development Fund (PSDF)
Start Date: July 2011
Project Type: Large Scale
Goal: Increase the rate of income growth in poor and vulnerable households in high poverty districts of Southern Punjab – Bahawalpur, Bahawalnagar, Lodhran and Muzaffargarh.
Objectives: Augment the skills-base of low income, poor and vulnerable families through vocational training.
The Government of Punjab supported by Department for International Development (DFID) introduced a bold poverty alleviation initiative in July 2011. Its aim was to increase incomes and economic stability in low income families residing in the high poverty districts of Punjab of Southern Punjab.
The vocational training and skills component of PEOP is being implemented by the Punjab Skills Development Fund (PSDF), which is a not-for-profit organization set up by the Government of Punjab in collaboration with DFID. PSDF has been created to provide access to low income members of the society to vocational training and skills acquisition programs with an aim to implement the ‘Theory of Change’.
CERP is responsible for providing evidence based recommendations towards program design along with evaluating the training and skills component. Our methodology is based on Smart Policy Design.
CERP’s engagement is divided in the following two phases:
- Baseline phase: Collect baseline data and provide analysis to enhance the program-design effort which is being undertaken by PSDF. At the same time, pilot evaluations will be carried out of the initial PSDF schemes and learnings from these evaluations will impact future schemes
- Evaluation phase: Conduct rigorous scientific impact evaluation using the Randomized Controlled Trials (RCT) methodology to evaluate the final redesigned schemes designed and implemented by PSDF.
With the successful completion of the baseline phase in the high poverty districts of Southern Punjab (Bahawalpur, Bahawalnagar, Lodhran and Muzzafargarh), CERP has successfully transitioned to the evaluation phase.
PEOP Theory of Change
PEOP’s theory of change has the following premises:
- Skills acquisition will provide critical human capital to enable the working-age population with poor human capital to transition into better quality jobs and improved earnings.
- Widening the skills base in agriculture, manufacturing and services is essential to stimulate productivity, which is an important pre-requisite for an increase in working class earnings.
- Access to functional training programs and resultant improvements in labor market opportunities will strengthen citizen trust in the state and contribute to civic and democratic political engagement.
- Focusing on women as an essential target group will result in improvements in female labor force participation and gender empowerment.
The RCT methodology, as illustrated by Figure 2, is a method of impact assessment that exploits random assignment in order to compare recipients of a treatment group to a counterfactual group of non-recipients who differ from recipients only in that they do not receive the treatment. This approach allows for a precise estimate of the causal impact of the program on the population included in the trial. Essentially, the RCT design allows us to assess the differences in outcomes between those offered the opportunity to train (treatment group) versus those not offered training (control group).
The baseline stage conducted in-depth surveys of 31, 495 households and 6800 employers that provided detailed insights into the demand and requirements for skills and the constraints faced in skill acquisition and employability. The evidence generated in this stage helped PSDF to calibrate its program design while providing evidence on:
- What type of training best matches the demand of households and employers?
- How should training be delivered to the target population to reduce access constraints (such as distance and safe and reliable transportation to the training centre, lack of information regarding trainings, financial and credit constraints, etc.) that prevent them from enrolling for training?
- Which types of post-training linkages need to be strengthened to broaden the set of economic opportunities for the target population?
These baseline phase studies showed that maximizing impact requires addressing constraints along three margins: (a) supply of quality training, (b) access to training for the target population and (c) post-training linkages with economic opportunities that enable skilled workers to benefit from existing opportunities.
They also showed that the nature of constraints along these margins is very different for the rural and urban populations. The rural population exists in a context of low employer density, thin markets for training with a limited menu and severe mobility constraints. In contrast, the urban context is different as employer density is thicker as is the market for training and the menu it supports. Therefore, specific solutions to address constraints need to differ by the type of population and for each population the solutions need to focus on different margins.
CERP initiated its research for PEOP by evaluating PSDF’s Skills for Employability scheme in March 2012. The aim of the evaluation was to provide early assessment of the impact of PSDF training on socio-economic outcomes of the target population.
Future interventions were designed based on the outcomes of SFE by focusing on particular issues and constraints in acquiring skills. It aimed to assess whether delivering vouchers to households increased uptake of training.
Key Findings of SFE
- Uptake is low in the general population – A small fraction of inpiduals randomly selected for this evaluation from the Program districts opted to enroll in the free training courses. The uptake for PSDF supported training was 5% in a representative sample of the population.
Uptake is low in the general population – A small fraction of inpiduals randomly selected for this evaluation from the Program districts opted to enroll in the free training courses. The uptake for PSDF supported training was 5% in a representative sample of the population.
- Uptake was low among the Program’s target population – While the uptake was low in the general population, it was even lower among some members of the Program’s target population such as the asset poor, women, less educated etc.
- A range of factors constrained training demand – Analysis indicated that low program uptake was not due to lack of demand among the general population. A large fraction of baseline households repeatedly expressed demand for vocational skills training at four different stages of the enrollment process. Instead, several socioeconomic factors were associated with low enrollment such as distance from the training provider, low stipend allowance, low household wealth and limited number of courses offering a trainee’s preferred skills.
- Uptake among women is further inhibited by household level constraints – Training uptake among women is further constrained by household factors such as large number of dependents, small household size and labor force participation by other household members.The figure below shows the reasons for refusal collected from those who did not select a course:
This scheme (February 2013-August 2013) was designed to study the elasticity of stipend to enrolling in vocational training programs. We wanted to see if additional stipend led to an increase in voucher acceptance and eventually enrollment in training courses. The design of this scheme was based on evidence collected from baseline household surveys and the SFE evaluation.
Because we had already tested for other constraints for women in SFM 2012-13, SFJ sample was restricted to urban males. The target group for this scheme was males of Urban Bahawalpur and Lodhran.
The results of this phased experiment suggested that enrollment was not very responsive to increased stipend. A modest increase in uptake was observed among urban males when PSDF?s base stipend was increased two to three-fold. Furthermore, despite high demand for skills training, uptake was low among males that were actively engaged in the labor market (with relatively higher uptake among students and the unemployed). These findings implied that stipend alone, unless significantly increased, would be unlikely to resolve the uptake problem among specific sub-groups of urban males.
The findings also suggested that the current model was not well suited to train urban workers who were actively engaged in the labor market, thereby presenting a strong case for integrating job placement with skills training. it also presented the need for experimenting with on-the-job training and employer-based training, as well as flexible training models. Later discussions on these findings led to a common understanding that job placement services were critical for increasing uptake.
SFM 2012-13 (November 2012 – May 2013) was a pilot project designed to test for two factors that affect voucher uptake, namely social mobilization and distance to training center. The results of the initial assessment of SFE showed a very low uptake. This scheme was designed to understand the possible causes of this. The scheme was designed for women exclusively as we believed that distance was a primary issue for women, whereas uptake amongst males could potentially be triggered by a higher stipend to compensate for lost income. It involved approximately 1,000 inpiduals from 52 villages from the baseline survey sample. The target districts for this scheme were females of rural Bahawalpur and Lodhran.
The results indicated substantial returns to locating the training facility within the village: villages that received a training center saw take-up rates on average 30 percentage points higher than those that did not. This evidence helped answer an open question about the effectiveness of locating the training center in the village, but it also raised practical concerns. Although locating the training center in the village significantly increased uptake, it did so at a high cost. Given PSDF’s current capacity and budget, it was not feasible to locate training centers in every village.
After the conclusion of SFM (2012-2013), in April 2013 the research team began design innovations and stakeholder engagement in preparation for a broader study (SFM 2013-14) on more cost-effective schemes that alleviate distance related access constraints for villages not hosting a training center.
Valuable insights gained regarding uptake:
- Social mobilization matters. Social mobilization efforts, including household visits and other forms of encouragement, had a positive impact on course enrollment. Villages that participated in social mobilization activities had significantly higher course uptake rates than villages that only received information about course offerings.
- Distance matters. Distance to the training center was a commonly cited constraint among prospective voucher-holders who did not enroll in or complete a course. Nearly half of the households that did not have a training center located within their village stated that distance to training was the most important barrier to uptake.
- Choice of course matters. Women who participated in the PSDF scheme were more likely to complete Tailoring courses than other course offerings. While 36% of participating women with in-village training centers completed Tailoring courses, only 29% completed Home Decoration and 9% completed Dairy Products courses.
SFM 2013-14 was launched in October 2013 as a follow up to the first phase of Skills for Market Scheme (SFM 2012-13). As revealed by the baseline surveys and previous schemes, access constraints hindered voucher uptake and the enrollment of women in skills’ training programs. Therefore, SFM 2013-14 was designed to measure and evaluate the impact of acquiring the popularly demanded training in tailoring on the economic and non-economic returns to rural women in the pilot districts. In addition, the scheme also aimed to assess the effect of design calibrations intended to address salient access hurdles that adversely impact uptake and economic returns in a social context where women face severe mobility constraints. The target districts for this scheme were Bahawalpur, Bahawalnagar and Muzaffargarh.
Baseline surveys and interviews following up on previous schemes indicated that women in the rural areas faced a number of problems. Distance from training centers, lack of safe and reliable transport, insufficient information regarding training programs, opportunity cost of enrollment and restrictive social norms were considered to be some of the major obstacles being faced by rural females.
Further analysis revealed that these constraints were interrelated and tackling them would require combining schemes to address the issues at hand. CERP in collaboration with PSDF organized and implemented different treatments, which broadly included village-based training (VBT) in which training centers are located within the villages; non-village-based training (non-VBT) where training centers are at a distance from the villages; provision of a group transport to trainees where the option of group transport to and from the training centers is also given; and providing course information about the location as well as inpidual and village-level social mobilization.
Preliminary analysis on the impact of treatment on uptake has recently been completed.
To briefly sum up:
- Stipend raises enrollment but is a relatively expensive method
- Social mobilization is challenging to do effectively as it requires much time and thought to develop
- In-village training is most effective to raise uptake – 36% take-up
- Group Transport is most effective to raise outside village uptake – 20% take-up
Future post-treatment trackers are planned for SFM 2013-14 that will provide for a rich understanding of the household level impacts on income, consumption, political attitudes, and well-being.
It is worth noting that based on the results forSFM 2012-13, distance to training location was stated as one of the most significant reasons for voucher refusal; almost 46% of the households didn’t accept vouchers because of distant location of training centers. For SFM 2013-14, distance to the training center was only reported by 9% of the households as the reason for voucher refusal.
Figure 6shows the various reasons for refusing vouchers presented by sample households. It can be observed that the most prevalent reason this time around (stated by almost 40% of the households) for refusing vouchers is commitment to household work and responsibilities.
Field visits and interviews with SFM 2013-14 graduates revealed that these women find it difficult to engage with the market post-training thereby limiting their income generating opportunities. Additionally, evidence gathered earlier from our baseline surveys revealed that women are predominantly engaged in home based work due to the reluctance of local employers to hire women and low labor mobility of women in general.
Market Linkages, an add-on component to SFM 2013-14 aims at alleviating these constraints (such as restrictive social norms, distance from markets and lack of market knowledge and organization) by linking these women to sales agents who will provide them with raw materials and designs for clothes that are in demand and bring orders for these clothes from a variety of markets. This will enable these women to engage with the market while allowing them to generate stable sources of income. This scheme will be launched in mid-April in the districts of Bahawalpur and Bahawalnagar.
The objective of conducting an evaluation of Market Linkages will be to measure the impact of this scheme and gauge how effective it has been in increasing the returns from SFM training. This will help inform PSDF of the feasibility of this add-on scheme and allow it to decide whether it should become a permanent part of its portfolio of schemes.
This scheme aims to increase the productivity of rural agricultural communities through saturating villages with intensive training in agriculture, livestock, and veterinary practices throughout the value chains.
The objective is to identify whether saturating villages with human capital in high employability sectors that will catalyze scale economies and complementarities can act as an engine of growth. This is very much in line with ideas such as Sach’s Millennium Villages and consistent with “big push” theories of economic growth.
The scheme will be based on a review of local and international evidence gained as a result of the success of different designs of training and technical advisory (TA) service models for villages. The design involves PSDF developing a comprehensive menu of trainings in frontier skills and practices related to the entire value chain in these sectors. The design of this menu will involve an engagement with a consortium of leading private sector agricultural companies and input suppliers as well as progressive farmers to share knowledge on frontier skills and practices across the value chain.
This scheme is currently in the design phase.
The previous SFJ evaluation indicated that uptake of SFJ training was low among urban males actively participating in the labor market and is unresponsive to doubling and trebling stipend levels. Moreover, according to an employer survey, most employers hire people through their own local networks rather than through advertisements and job fairs.
This scheme aims to use the next round of SFJ trainings to generate evidence on both the effectiveness of the program as it currently stands and ways it can be improved in the future.
Thus, the objective of the SFJ evaluation is two-fold and is geared towards:
- Evaluating the Impact of Vocational Training: This component aims to understand the skills and employment gains from existing skills trainings.
- Improving Job Placement: This component aims to understand which strategies can effectively increase job placement and economic outcomes.
SFJ/JP is currently in the design phase.