Process evaluation is an essential part of designing and testing complex interventions. New MRC guidance provides a framework for conducting and reporting process evaluation studies

https://www.bmj.com/content/bmj/350/bmj.h1258.full.pdf

Process evaluation of complex interventions: Medical Research Council guidance

In the past few years several major reports highlighted the gap between our knowledge of effective treatments and services currently being received by consumers. These reports agree that we know much about interventions that are effective but make little use of them to help achieve important behavioral health outcomes for children families and adults nationally. This theme is repeated in reports by the Surgeon General (United States Department of Health and Human Services 1999; 2001) the National Institute of Mental Health [NIMH] National Advisory Mental Health Council Workgroup on Child and Adolescent Mental Health Intervention Development and Deployment (2001) Bernfeld Farrington & Leschied (2001) Institute of Medicine (2001) and the Presidents New Freedom Commission on Mental Health (2003). The authors call for applied research to better understand service delivery processes and contextual factors to improve the efficiency and effectiveness of program implementation at local state and national levels. Our understanding of how to develop and evaluate evidence-based intervention programs has been furthered by on-going efforts to research and refine programs and practices to define "evidence bases" and to designate and catalogue "evidence-based programs or practices". However the factors involved in successful implementation of these programs are not as well understood. Current views of implementation are based on the scholarly foundations prepared by Pressman & Wildavskys (1973) study of policy implementation Havelock & Havelocks (1973) classic curriculum for training change agents and Rogers (1983; 1995) series of analyses of factors influencing decisions to choose a given innovation. These foundations were tested and further informed by the experience base generated by pioneering attempts to implement Fairweather Lodges and National Follow-Through education models among others. Petersilia (1990) concluded that "The ideas embodied in innovative social programs are not self-executing." Instead what is needed is an "implementation perspective on innovation--an approach that views postadoption events as crucial and focuses on the actions of those who convert it into practice as the key to success or failure". (excerpt)

Implementation research: a synthesis of the literature.

In this review, we examine the oft-made claim that peer-relationship difficulties in childhood predict serious adjustment problems in later life. The article begins with a framework for conceptualizing and assessing children's peer difficulties and with a discussion of conceptual and methodological issues in longitudinal risk research. Following this, three indexes of problematic peer relationships (acceptance, aggressiveness, and shyness/withdrawal) are evaluated as predictors of three later outcomes (dropping out of school, criminality, and psychcpathology). The relation between peer difficulties and later maladjustment is examined in terms of both the consistency and strength of prediction. A review and analysis of the literature indicates general support for the hypothesis that children with poor peer adjustment are at risk for later life difficulties. Support is clearest for the outcomes of dropping out and criminality. It is also clearest for low acceptance and aggressiveness as predictors, whereas a link between shyness/withdrawal and later maladjustment has not yet been adequately tested. The article concludes with a critical discussion of the implicit models that have guided past research in this area and a set of recommendations for the next generation of research on the risk

Peer relations and later personal adjustment: Are low-accepted children at risk?

A developmental model of antisocial behavior is outlined. Recent findings are reviewed that concern the etiology and course of antisocial behavior from early childhood through adolescence. Evidence is presented in support of the hypothesis that the route to chronic delinquency is marked by a reliable developmental sequence of experiences. As a first step, ineffective parenting practices are viewed as determinants for childhood conduct disorders. The general model also takes into account the contextual variables that influence the family interaction process. As a second step, the conduct-disordered behaviors lead to academic failure and peer rejection. These dual failures lead, in turn, to increased risk for depressed mood and involvement in a deviant peer group. This third step usually occurs during later childhood and early adolescence. It is assumed that children following this developmental sequence are at high risk for engaging in chronic delinquent behavior. Finally, implications for prevention and intervention are discussed.

A developmental perspective on antisocial behavior.

Progress and issues in the study of coping with stress during childhood and adolescence are reviewed. Definitions of coping are considered, and the relationship between coping and other aspects of responses to stress (e.g., temperament and stress reactivity) is described. Questionnaire, interview, and observation measures of child and adolescent coping are evaluated with regard to reliability and validity. Studies of the association of coping with symptoms of psychopathology and social and academic competence are reviewed. Initial progress has been made in the conceptualization and measurement of coping, and substantial evidence has accumulated on the association between coping and adjustment. Problems still remain in the conceptualization and measurement of coping in young people, however, and aspects of the development and correlates of coping remain to be identified. An agenda for future research on child-adolescent coping is outlined.

https://psycnet.apa.org/journals/bul/127/1/87.pdf

Coping with stress during childhood and adolescence: problems, progress, and potential in theory and research.

Potential roles of parental self-efficacy in parent and child adjustment: a review.

Treatment fidelity in outcome studies

A battery of measures was used to assess conflict between mothers and young adolescents (females and males, 11 to 15 years of age). Two groups of families, one composed of a distressed clinical sample (N = 38), the other a nondistressed normative sample (N = 40), participated. The assessment battery included retrospective judgments, frequency estimates, self-monitored home recording, and tape-recorded discussion of a home problem. Content of assessment measures tapped aspects of parental control, decision-making, self-reported interaction behavior, arguments, interaction behavior rated by independent "blind" observers, frequency and anger-intensity of specific problematic issues, and perceptions of positive and negative behaviors of the other family member. Based on univariate analyses, 21 of the 26 defined variables discriminated significantly in the predicted direction. Maternal and adolescent reports of behavior and independent ratings of tape-recorded interaction emerged as strong and consistent discriminators. Stepwise multivariate discriminant analysis provided successful classification of 100% of the families based on the inclusion of nine variables. In a cross-validation sample, 84% of the families were correctly classified. Implications for systematic outcome research as well as clinical application are discussed.

/pdf/multivariate-assessment-of-conflict-in-distressed-and-3p2l7e99x2.pdf

Multivariate assessment of conflict in distressed and nondistressed mother‐adolescent dyads

The prevention of child maltreatment necessitates a public health approach. In the U.S. Triple P System Population Trial, 18 counties were randomly assigned to either dissemination of the Triple P—Positive Parenting Program system or to the services-as-usual control condition. Dissemination involved Triple P professional training for the existing workforce (over 600 service providers), as well as universal media and communication strategies. Large effect sizes were found for three independently derived population indicators: substantiated child maltreatment, child out-of-home placements, and child maltreatment injuries. This study is the first to randomize geographical areas and show preventive impact on child maltreatment at a population level using evidence-based parenting interventions.

/pdf/population-based-prevention-of-child-maltreatment-the-u-s-559qnnt2lx.pdf

Ronald J. Prinz

Papers

Potential roles of parental self-efficacy in parent and child adjustment: a review.

Treatment fidelity in outcome studies

Multivariate assessment of conflict in distressed and nondistressed mother‐adolescent dyads

Population-based prevention of child maltreatment: the U.S. Triple p system population trial.

Parental alcoholism and childhood psychopathology.