A meta-analysis exploring the long-term effects of school-based reading interventions shows that programs that teach comprehension strategies are much more effective than programs that focus only on phonics (sound/symbol correspondence). 1 Although phonics programs for younger children seem to have a measurable impact in the short term (immediately after completion of the intervention), these effects fade significantly in the ensuing months. In contrast, the effect size of interventions focused on comprehension strategies actually increases over time after completion of the underlying program.
This study is particularly important because most studies of reading instructional approaches or interventions do not track long term progress. Instead, most studies report only on immediate gains. But if, as appears from this study, the gains from some forms of teaching are not sustained, the value of the intervention is called into question.
Previous Studies of Pre/Post Program Effects
Before this study of longer-term outcomes, the same researcher, Sebastion Suggate, completed a meta-analysis of studies of 116 treatment-control groups, comparing immediate post-program effect sizes (degree of improvement) of phonics programs with programs geared to aiding reading comprehension. 2
The comprehension or meaning-based approaches included strategies such as meta-cognitive skills, reflective listening, mental imagery, story structure, and summarization. The analysis showed that the comprehension-focused interventions were more effective for children in grades 2 and above, although the phonics programs seemed helpful at the preschool level through grade 1. The cross-over point at which comprehension-focused programs showed better results was at about mid-first grade.
These findings were consistent with earlier findings of the U.S. National Reading Panel (NRP), based on its own meta-analysis of available data, which concluded, “Phonics instruction taught early proved much more effective than phonics instruction introduced after 1st grade.” 3
Study of Long-Term Outcomes
Suggate’s study of long-term outcomes evaluated results from 71 treatment-control groups from studies published between 1980 and 2010. Each study included pre- and post- program testing results, with subsequent followup testing averaging about 12 months after the intervention post-test. Actual periods for the followup testing ranged from 3 months post-program to 48 months. The programs geared to comprehension, or mixed programs which included a comprehension component, showed the best long-term results. This was in part because the comprehension programs showed an average increase in effect size between the initial post-test and long-term followup testing, whereas the phonics programs showed a significant decrease in effect size over time.
|Type of Intervention||Post-Test Effect Size||Follow-Up Effect Size|
The analysis also showed that older children fared better over time than younger students, with students in grades 3-6 showing the strongest sustained gains: “kindergarten and preschool follow-up effect sizes were small, those in Grades 1 to 2 were small to moderate, and those in Grades 3 to 6 were large to moderate.” The researcher noted, “the younger the intervention sample, the lower the effect size at follow-up.” In fact, at the point of follow-up, the reading interventions “were more than 3.5 times more effective for older children.”
|Grade Level at Intervention||Post-Test Effect Size||Follow-Up Effect Size|
|Preschool / Kindergarten||0.34||0.12|
|Grades 1 and 2||0.40||0.26|
|Grades 3 to 6||0.35||0.43|
Effect size in research is a measure of the degree of difference between a treatment group in contrast with a control or comparison group. An effect size of 0.20 is considered small; 0.50 is medium; and an effect size of 0.80 or above is considered to be large. An effect size of less than 0.20 is considered trivial; that is, the level of change is too small to justify the use of the intervention producing that effect. Thus, overall long-term effect sizes for the phonics-only interventions and for the programs for preschool and kindergarten proved too low over time to reach the level of even “small” improvement.
What do these findings mean?
A meta-analysis can identify patterns, trends, and problems by compiling, comparing, and scrutinizing data from multiple studies. However, because the specific interventions used and the findings of the separate underlying studies are varied, the findings of the meta-analysis cannot be universally applied. Fortunately, Suggate’s research articles provide meticulous details of the methodology employed and explore many distinguishing factors among the studies relied on, and also include a complete list of each study and summary tables of the individual findings.
However, the overall degradation of effect sizes between post-test and follow-up raises important questions about the timing and evaluation of reading intervention programs, particularly among those exclusively focused on phonics teaching. Much of educational policy and adoption of programs as being “evidence-based” is based on studies that showed only short-term results. Such programs may appear to be effective at the time they are given, but if the benefit fades over time, then their value needs to be reassessed, especially if instructional time could be better spent on approaches that are more likely to foster long-term improvement.
Similarly, this work suggests that an emphasis on intensive, early intervention might be misplaced. It may be more effective to follow a gradual approach which provides limited, developmentally appropriate teaching early on, and is structured to continue through later elementary school years.
It is possible that the phonetic decoding skills taught to young children simply do not transfer well as reading becomes more complex and less predictable, especially given the inconsistency of English spelling. Because of the important role of word morphology and derivation in English spelling, the students who have been taught to focus on word meaning may have an edge as reading difficulty increases. The comprehension strategies taught to young children are tied to higher-order reading skills which are readily transferable to more advanced reading material. Students with strong comprehension skills are probably more likely to find reading enjoyable and informative, and to be motivated to tackle more challenging books. The increase in effect size over time for the comprehension-only programs may be a result of skills being reinforced through practice with independent reading.
- Suggate, Sebastian P. A Meta-Analysis of the Long-Term Effects of Phonemic Awareness, Phonics, Fluency, and Reading Comprehension Interventions. Journal of Learning Disabilities, Volume 49, Issue 1, 2014 | DOI: 10.1177/0022219414528540
- Suggate, Sebastian P. Why what we teach depends on when: Grade and reading intervention modality moderate effect size. Developmental Psychology . 46(6), 1556-1579. 2010 | DOI: 10.1037/a0020612
- National Institute of Child Health and Human Development, NIH, DHHS. (2000). Report of the National Reading Panel: Teaching Children to Read: Reports of the Subgroups (00-4754). Washington, DC: U.S. Government Printing Office.