Primer.
Understanding “Evidence-Based”
School-based efficacy studies are difficult to arrange as there are numerous layers of bureaucracy, encroachment on school and family life and significant expenses in coordination and data collection. They are different than medical trials where patients and controls are easily identified and the trial conditions can be closely monitored.
School-based efficacy studies are also designed in different ways and have varying levels of rigor. When evaluating efficacy results of any educational program it’s important to understand the difference in rigor.
Randomized-Controlled Trial
The gold standard for rigor is the Randomized-Controlled Trial (RCT). This trial randomly assigns one school to receive the program, and a similar school does not receive the program (the control condition). This ensures that any differences observed in a program school were due to the program and not some other cause(s). Positive Action has completed two RCT's, in Chicago and Hawaii.
Quasi-Experimental Trial
The more common type of trial found in educational programs is the Quasi-Experimental Trial (QET) where there is no randomization in the sample choice. A QET can include a comparison group but this is not the same as a control group in a RCT. This forces the researchers to rule out alternative plausible explanations of the observed effects in other ways which is very difficult to do with any certainty.
Case Study
The least reliable trial type is a Case Study. They're considered pre-experimental as they focus on one school or district, and do not include a control or comparison group. Some consider Case Studies with multiple waves of data a QET but only those where there is very systematic manipulation of cause and effect can be so considered. Most reports established after the fact usually do not involve this careful manipulation.
Feeder Effect
Analysis from one trial has found evidence of the feeder effect, where students who participated in an elementary program school were tracked through middle school and high school.
Some secondary schools had far fewer Positive Action graduates. These are called low-feeder schools, whereas high- feeder schools had higher proportions of Positive Action graduates in the student body. The results consistently indicated far more positive results in high-feeder schools.
Comprehensive, Integrated, Three Tiered Model of Prevention
Another set of results describes a three-tier intervention model, with Positive Action being tested as a Tier 2 program. Tier 1 focuses on the needs of all students. Tier 2 is intended for students who have not responded to Tier 1 intervention. Tier 3 is for students who require intensive intervention and are at risk.
Outcomes List
The outcomes of the Positive Action trials are organized and displayed by subject. Each result includes a link to the trial summary, its design type and the referring journal article.
