We study trends in the partisanship of Congressional speech from 1873 to 2009. We define partisanship to be the ease with which an observer could infer a congressperson’s party from a fixed amount of speech, and we estimate it using a structural choice model and methods from machine learning. The estimates reveal that partisanship is far greater today than at any point in the past. Partisanship was low and roughly constant from 1873 to the early 1990s, then increased dramatically in subsequent years. Evidence suggests innovation in political persuasion beginning with the Contract with America, possibly reinforced by changes in the media environment, as a likely cause. Naive estimates of partisanship are subject to a severe finite-sample bias and imply substantially different conclusions.Institutional subscribers to the NBER working paper series, and residents of developing countries may download this paper without additional charge at www.nber.org.

/pdf/measuring-polarization-in-high-dimensional-data-method-and-wk5maj2nm6.pdf

Measuring Polarization in High-Dimensional Data: Method and Application to Congressional Speech

We study the problem of measuring group differences in choices when the dimensionality of the choice set is large. We show that standard approaches suffer from a severe finite‐sample bias, and we propose an estimator that applies recent advances in machine learning to address this bias. We apply this method to measure trends in the partisanship of congressional speech from 1873 to 2016, defining partisanship to be the ease with which an observer could infer a congressperson's party from a single utterance. Our estimates imply that partisanship is far greater in recent years than in the past, and that it increased sharply in the early 1990s after remaining low and relatively constant over the preceding century.

/pdf/measuring-group-differences-in-high-dimensional-choices-3czkbbqoz0.pdf

Measuring Group Differences in High‐Dimensional Choices: Method and Application to Congressional Speech

The most widely used measure of segregation is the so‐called dissimilarity index. It is now well understood that this measure also reflects randomness in the allocation of individuals to units (i.e. it measures deviations from evenness, not deviations from randomness). This leads to potentially large values of the segregation index when unit sizes and/or minority proportions are small, even if there is no underlying systematic segregation. Our response to this is to produce adjustments to the index, based on an underlying statistical model. We specify the assignment problem in a very general way, with differences in conditional assignment probabilities underlying the resulting segregation. From this, we derive a likelihood ratio test for the presence of any systematic segregation, and bias adjustments to the dissimilarity index. We further develop the asymptotic distribution theory for testing hypotheses concerning the magnitude of the segregation index and show that the use of bootstrap methods can improve the size and power properties of test procedures considerably. We illustrate these methods by comparing dissimilarity indices across school districts in England to measure social segregation.

More reliable inference for the dissimilarity index of segregation

We address empirically the issues of the optimality of simple linear compensation contracts and the importance of asymmetries between firms and workers. For that purpose, we consider contracts between the French National Institute of Statistics and Economics (Insee) and the interviewers it hired to conduct its surveys in 2001, 2002, and 2003. To derive our results, we exploit an exogenous change in the contract structure in 2003, the piece rate increasing from 20.2 to 22.9 euros. We argue that such a change is crucial for a structural analysis. It allows us, in particular, to identify and recover nonparametrically some information on the cost function of the interviewers and on the distribution of their types. This information is used to select correctly our parametric restrictions. Our results indicate that the loss of using such simple contracts instead of the optimal ones is no more than 16%, which might explain why linear contracts are so popular. We also find moderate costs of asymmetric information in our data, the loss being around 22% of what Insee could achieve under complete information.
				
Incentives asymmetric information optimal contracts nonparametric identification C14 D82 D86

/pdf/the-provision-of-wage-incentives-a-structural-estimation-31m4dnb2pf.pdf

The provision of wage incentives: A structural estimation using contracts variation

In human geography and the urban social sciences, the segregation literature typically engages with five conceptual dimensions along which a given society may be considered segregated: evenness, isolation, clustering, concentration and centralization (all of which can incorporate or omit spatial context). Over the last several decades, dozens of segregation indices have been proposed and studied in the literature, each of which is designed to focus on the nuances of a particular dimension, or correct an oversight in earlier work. Despite their increasing proliferation, however, few of these indices remain used in practice beyond their original conception, due in part to complex formulae and data requirements, particularly for indices that incorporate spatial context. Furthermore, existing segregation software typically fails to provide inferential frameworks for either single-value or comparative hypothesis testing. To fill this gap, we develop an open-source Python package designed as a submodule for the Python Spatial Analysis Library, PySAL. This new module tackles the problem of segregation point estimation for a wide variety of spatial and aspatial segregation indices, while providing a computationally based hypothesis testing framework that relies on simulations under the null hypothesis. We illustrate the use of this new library using tract-level census data in two American cities.

/pdf/an-open-source-framework-for-non-spatial-and-spatial-3q8i2j2842.pdf

An open-source framework for non-spatial and spatial segregation measures: the PySAL segregation module

We consider the issue of measuring segregation in a population of small units, considering establishments in our application. Each establishment may have a different probability of hiring an individual from the minority group. We define segregation indices as inequality indices on these unobserved, random probabilities. Because these probabilities are measured with error by proportions, standard estimators are inconsistent. We model this problem as a nonparametric binomial mixture. Under this testable assumption and conditions satisfied by standard segregation indices, such indices are partially identified and sharp bounds can be easily obtained by an optimization over a low dimensional space. We also develop bootstrap confidence intervals and a test of the binomial mixture model. Finally, we apply our method to measure the segregation of foreigners in small French firms.

Segregation small units partial identification C13 C14 J71

http://www.hec.unil.ch/documents/seminars/deep/1283.pdf

Xavier D'Haultfœuille

Papers

Measuring segregation on small units: A partial identification analysis