The employment effects of job-creation schemes in Germany: A microeconometric evaluation

TLDR: In this paper, the employment effects of job-creation schemes (JCS) on the participating individuals in Germany are evaluated and the overall results are rather discouraging, since employment effects are negative or insignificant for most of the analysed groups.

Abstract: In this chapter, we evaluate the employment effects of job-creation schemes (JCS) on the participating individuals in Germany. JCS are a major element of active labour market policy in Germany and are targeted at long-term unemployed and other hard-to-place individuals. Access to very informative administrative data of the Federal Employment Agency justifies the application of a matching estimator and allows us to account for individual (group-specific) and regional effect heterogeneity. We extend previous studies for Germany in four directions. First, we are able to evaluate the effects on regular (unsubsidised) employment. Second, we observe the outcomes of participants and non-participants for nearly three years after the programme starts and can therefore analyse medium-term effects. Third, we test the sensitivity of the results with respect to various decisions that have to be made during implementation of the matching estimator. Finally, we check if a possible occurrence of a specific form of ‘unobserved heterogeneity’ distorts our interpretation. The overall results are rather discouraging, since the employment effects are negative or insignificant for most of the analysed groups. One exception are long-term unemployed individuals who benefit from participation at the end of our observation period. Hence, one policy implication is to address the programmes to this problem group more closely.

Figures

Table 6 contains the results for the eleven sub-groups. The first column in the table refers to the MSB before matching and the second column shows the MSB after matching, when matching is done with the estimated ‘overall’ propensity score as shown in table A.4. This propensity score specification, which we label P1, has been done separately for the four main groups. However, it is very clear that the matching procedure based on the overall scores is not able to balance the covariates between treated and matched non-treated individuals in the sub-groups. For example, the bias after matching for men in West Germany reaches a level of 13.23% for rehabilitation attendants. Even though this is a reduction compared to the MSB before matching, it is not acceptable. For women in West Germany in this group the bias after matching is 18.69%. In East Germany the bias after matching is not much lower, reaching levels of 13.11% for young men and 15.11% for young women. Even though there are some sub-groups for which the bias is acceptable, the overall matching quality in the sub-groups is not. Hence, alternative strategies are called for. One way to do so is to re-define the propensity score estimation. Whereas the ‘overall’ propensity score estimation has only been done separately for men and women in West and East Germany, we estimate in a second step ‘group-specific’ propensity scores. The basic idea behind that is to capture the varying influence of the variables for certain sub-groups more accurately. Since we have eleven sub-groups for both gender and regions, we are left with 44 propensity score estimations.22 Based on these ‘group-specific’ estimations, labelled P2, we re-run the matching procedure and estimate

Table 3 contains the results for the main groups for the last month of the observation period. Bold letters indicate significance at the 1%-level, italic letters refer to the 5%-level, standard errors are bootstrapped with 50 replications. The estimates illustrate two points: First of all, the results are not sensitive to the chosen matching algorithm. For men in West Germany the effects are insignificant

Table 4 contains the number of treated individuals lost in each of the sub-groups. It can be seen that the number of lost individuals is fairly low for three of the main groups. For men in West Germany we lose 0.79% of the observations, for men (0.03%) and women (0.16%) in East Germany the proportion is even smaller. However, for women in West Germany we cannot find similar non-participants for around 6.84% of the treated population and have to discard these individuals. Figure A.2 makes very clear that the overlap between participating and non-participating women in West Germany is fairly limited in the sub-groups, too. As a consequence, we there lose up to 20.95% of the treated population (for women with placement restrictions). Hence, interpretation of the effects has to be made careful.

Full text

IZA DP No. 1512
The Employment Effects of Job Creation Schemes
in Germany: A Microeconometric Evaluation
Marco Caliendo
Reinhard Hujer
Stephan L. Thomsen
DISCUSSION PAPER SERIES
Forschungsinstitut
zur Zukunft der Arbeit
Institute for the Study
of Labor
March 2005

The Employment Effects of Job
Creation Schemes in Germany:
A Microeconometric Evaluation
Marco Caliendo
DIW Berlin and IZA Bonn
Reinhard Hujer
University of Frankfurt,
ZEW Mannheim and IZA Bonn
Stephan L. Thomsen
University of Frankfurt
Discussion Paper No. 1512
March 2005
IZA
P.O. Box 7240
53072 Bonn
Germany
Phone: +49-228-3894-0
Fax: +49-228-3894-180
Email: iza@iza.org
Any opinions expressed here are those of the author(s) and not those of the institute. Research
disseminated by IZA may include views on policy, but the institute itself takes no institutional policy
positions.
The Institute for the Study of Labor (IZA) in Bonn is a local and virtual international research center
and a place of communication between science, politics and business. IZA is an independent nonprofit
company supported by Deutsche Post World Net. The center is associated with the University of Bonn
and offers a stimulating research environment through its research networks, research support, and
visitors and doctoral programs. IZA engages in (i) original and internationally competitive research in
all fields of labor economics, (ii) development of policy concepts, and (iii) dissemination of research
results and concepts to the interested public.
IZA Discussion Papers often represent preliminary work and are circulated to encourage discussion.
Citation of such a paper should account for its provisional character. A revised version may be
available directly from the author.

IZA Discussion Paper No. 1512
March 2005
ABSTRACT
The Employment Effects of Job Creation Schemes in
Germany: A Microeconometric Evaluation
∗
In this paper we evaluate the employment effects of job creation schemes on the participating
individuals in Germany. Job creation schemes are a major element of active labour market
policy in Germany and are targeted at long-term unemployed and other hard-to-place
individuals. Access to very informative administrative data of the Federal Employment
Agency justifies the application of a matching estimator and allows to account for individual
(group-specific) and regional effect heterogeneity. We extend previous studies in four
directions. First, we are able to evaluate the effects on regular (unsubsidised) employment.
Second, we observe the outcome of participants and non-participants for nearly three years
after programme start and can therefore analyse mid- and long-term effects. Third, we test
the sensitivity of the results with respect to various decisions which have to be made during
implementation of the matching estimator, e.g. choosing the matching algorithm or estimating
the propensity score. Finally, we check if a possible occurrence of 'unobserved heterogeneity'
distorts our interpretation. The overall results are rather discouraging, since the employment
effects are negative or insignificant for most of the analysed groups. One notable exception
are long-term unemployed individuals who benefit from participation. Hence, one policy
implication is to address programmes to this problem group more tightly.
JEL Classification: J68, H43, C13
Keywords: evaluation, matching, sensitivity analysis, job creation schemes, long-term
unemployed
Corresponding author:
Marco Caliendo
Department of Public Economics
DIW Berlin
Königin-Luise-Str. 5
14195 Berlin
Germany
Email: mcaliendo@diw.de
∗
This paper emerged within the research project ‘Effects of Job Creation and Structural Adjustment
Schemes’ financed by the Institute for Employment Research (IAB).

1 Intro duction
The German labour market is plagued by persistently high unemployment in combination with a
clearly separated situation on the labour markets in West and East Germany. This gets clear looking
at the unemployment rate for 2003, which has been 9.3% in West and 20.1% in East Germany. The
Federal Employment Agency (FEA) spends substantial amounts to overcome this unemployment
problem. A particular emphasis is laid on active labour market policies (ALMP), as the spendings
of 12.3 bn Euro in West Germany and 8.9 bn Euro in East Germany reflect. The main goal of
ALMP is the permanent integration of unemployed persons into regular employment. ALMP were
first introduced in Germany in the late 1960s and have been gradually adjusted to important changes
on the labour market since then. One major reform step was the introduction of the Social Code III
(Sozialgesetzbuch III) in 1998 as the legal basis for ALMP. Within that reform new instruments were
intro duced, competencies were decentralised and a more flexible allocation of funds has been made
possible. Mayb e the most important change from an evaluator’s point of view was the legal anchoring
of a mandatory output evaluation for all ALMP measures. As a consequence new administrative
datasets have been made accessible for scientific research.
1
A major element of ALMP in Germany over the last years have been job creation schemes (JCS),
even though their importance is currently decreasing. JCS have often been criticised because they
lack explicit qualificational elements and they might involve ‘stigma effects’.
2
However, it can also be
argued that they are a reasonable opportunity for individuals who are not able to re-integrate into
the first labour market themselves or who do not fit the criteria for other programmes, e.g. long-term
unemployed or other hard-to-place individuals. The evaluation of JCS has been impossible for a
long time, since datasets have either not been available or been to small to draw policy relevant
conclusions. However, with the intro duction of the Social Code III things have changed and give us
access to a very rich administrative dataset containing more than 11,000 participants in JCS and
a comparison group of nearly 220,000 non-participants. We use this data to answer the question,
if JCS enhance the employment chances of participating individuals. The extensive set of available
individual characteristics in combination with information on the regional labour market situation,
makes the application of a matching estimator possible. Additionally, the large number of participants
allows to account for several sources of effect heterogeneity.
The importance of effect heterogeneity for the evaluation of JCS in Germany has been well doc-
umented in Hujer, Caliendo, and Thomsen (2004). Basically, there are two shortcomings to that
study. The first one refers to an unsatisfying outcome variable, which allows only to monitor if
the individual is registered unemployed or not and does not allow to draw conclusions about the
re-integration success into regular (unsubsidised) employment. A second restriction relates to the
relatively short observation period after programme start, namely two years. This paper extends the
1
The reform process on the German labour market is still ongoing. More reforms are implemented gradually (see
‘Mo dern Services on the Labour Market’, Bundesministerium f¨ur Wirtschaft und Arbeit (2003)). Since we focus in our
empirical analysis on the time period 2000-2002, we are not going to discuss the current reforms here.
2
If the programme is targeted at people with ‘disadvantages’, there is always a risk that a possible employer takes
participation in such schemes as a negative signal concerning the expected productivity or motivation.
1

previous analyses in four directions. First, we are able to evaluate the re-integration effects of JCS
into regular (unsubsidised) employment. Second, we can monitor the employment status of partici-
pants and non-participants nearly three years after programme start. Third, we test the sensitivity of
the results to various decisions which have to be made whilst implementing the matching estimator,
like the choice of the matching algorithm or the estimation of the propensity score. Finally, we check
if a possible occurrence of ‘unobserved heterogeneity’ or ‘hidden bias’ distorts interpretation of our
results.
The focus of our analysis will be the identification of individual (group-specific) and regional ef-
fect heterogeneity.
3
To do so, we separate the analysis by several characteristics and carry out the
matching analysis on sub-populations.
4
Men and women in West and East Germany will be the ‘main
groups’ of our analysis. In addition, we estimate effects for eleven ‘sub-groups’ defined by age and
unemployment duration as well as by specific characteristics indicating disadvantages on the labour
market like the lack of professional training or the existence of placement restrictions due to health
problems. The situation on the regional labour market might be a further source of effect hetero-
geneity as programme impacts may differ in regions with high underemployment when compared to
prospering regions. We take that into account by evaluating the programme effects with respect to
regional differences using the classification of similar and comparable labour office districts of the
FEA (Blien et al. (2004)).
The remainder of this paper is organised as follows. In the following section we describe the insti-
tutional background of JCS in Germany, introduce the dataset used and present some descriptive
statistics. Section 3 explains the general framework for microeconometric evaluation analysis and
section 4 deals with the empirical implementation of the matching estimator. In particular we discuss
the justification of the matching estimator, the estimation of the propensity scores (section 4.1) and
the choice of the proper matching algorithm (section 4.2) for our situation. Section 4.3 deals with
common support issues, whereas section 4.4 presents some quality indicators for the chosen matching
algorithm. In section 5 we present the results for the main and sub-groups as well as for the re-
gional clusters. Additionally, we also test the sensitivity of our estimates with respect to unobserved
heterogeneity. The final section concludes and gives some policy recommendations.
2 Institutional Background, Dataset and Selected Descriptives
2.1 Institutional Background
JCS have been the second most important programme (after vocational training) of ALMP in Ger-
many in the last years with respect to the number of participants. JCS can be supported if they
provide the last chance to stabilise and qualify unemployed individuals for later re-integration into
3
In a companion paper, we concentrate on sectoral heterogeneity (Caliendo, Hujer, and Thomsen, 2004b).
4
There are basically two ways to put greater emphasis on specific variables. One can either find variables in the
comparison group who are identical with respect to these variables (see e.g. Puhani (1998) or carrying out matching
on sub-populations (see e.g. Heckman, Ichimura, and Todd (1997) or Heckman, Ichimura, Smith, and Todd (1998)).
2

Frequently asked questions

Q1. What have the authors contributed in "The employment effects of job creation schemes in germany: a microeconometric evaluation" ?

In this paper the authors evaluate the employment effects of job creation schemes on the participating individuals in Germany. Second, the authors observe the outcome of participants and non-participants for nearly three years after programme start and can therefore analyse midand long-term effects. 

Q2. Why do the authors expect to find stronger effects for the other groups?

Due to the worse ‘outside options’ of the non-participants, the authors expect to find weaker locking-in effects for older participants and stronger effects for the other groups (young and middle-aged persons). 

Q3. How long did the participants have to be unemployed before the start of the program?

Before January 2002, potential participants had to be long-term unemployed (for more than one year) or unemployed for at least six of the last twelve months before participation. 

Q4. What is the effect of the socio-demographic variables on the participation probability of men in west?

The coefficients of the socio-demographic variables show that the participation probability of men in West Germany decreases with age, while in East Germany older men and women are more likely to participate. 

Q5. What is the effect of the JCS on the long-term unemployed?

for the short-term unemployed persons and women with high degree the authors lose substantial amounts of individuals due to failure of the common support, indicating that it is problematic to find short-term unemployed who did not participate and have similar characteristics. 

Q6. What is the effect of the previous unemployment duration on the participants?

With regard to the previous unemployment duration, it can be assumed that re-integration into the labour market is generally easier for persons with only a short duration of unemployment (‘negative duration dependence’). 

Q7. What is the argument against excluding a variable from the analysis?

They argue that a variable should only be excluded from analysis if there is consensus that the variable is either unrelated to the outcome or not a proper covariate. 

Q8. What is the effect of the unemployment duration on the participation probability of unemployed women?

It can be assumed that it is for unemployed women with higher qualification harder than for higher qualified unemployed men to end their unemployment and so they are more likely to participate in JCS.