Summary
Besides funding basic and special education, government has provided revenue to help students whose academic achievement is below average for their grade in school. During fiscal year 1997, Minnesota school districts statewide received about $255 million in state and federal revenue for K-12 students' remedial education. Through the school funding formula, the Minnesota Legislature earmarked $132 million as "compensatory revenue" and another $39 million as "targeted needs aid." The federal government provided an additional $80 million in Title I funds through the Improving America's Schools Act and about half a million dollars for homeless and new immigrant students.
Despite these large expenditures, more than one-fourth of Minnesota's public students failed either the eighth grade reading or math skills test given in 1996 and 1997 that the state now requires for graduation. As a result, in May 1997, the Legislative Audit Commission directed us to examine the remedial education services that school districts have provided.
Our evaluation addressed public school remedial education in grades K-12, regardless of funding source. For the purposes of our study, we defined remedial education broadly to refer to all strategies, programs, and services that schools routinely used to bring low-achieving K-12 students' academic performance closer to the standards for their grade in school. <FN We excluded special education services delivered to students with individual education plans unless schools provided them with remedial education services in addition to special education.> We focused on the following major research questions:
To answer these questions, we used information from a variety of sources. We analyzed average test scores for schools that received Title I funds during the 1995-96 school year and for schools that administered the Minnesota Basic Standards Tests in 1996 and 1997. We surveyed a sample of 659 elementary, middle, and secondary schools from around the state to learn what remedial education services have been provided to students, how many students participated, and whether there was any evidence of effectiveness. In addition, we interviewed staff from the Department of Children, Families & Learning, the University of Minnesota, and the U.S. Department of Education, and school administrators and teachers about remedial education. Finally, we reviewed the research literature to learn more about effective remedial practices and the experiences of other states.
The federal government became involved in remedial education in 1965 when it created Title I of the Elementary and Secondary Education Act. A component of President Johnson's War on Poverty, Title I was designed to address economic inequality by improving educational opportunities for children of poverty.
Minnesota's efforts to address the educational problems associated with poverty began in 1971 when the Legislature created the forerunner of what is now known as "compensatory revenue." Since that time, the Legislature has refined and expanded its efforts to help ameliorate problems of poverty and the low achievement that is often associated with it. Over the last 10 years, the Legislature generally has expanded compensatory revenue to allow more districts to receive such aid and to change the basis of how that aid is calculated. Today, it represents the largest single source of state funds for remedial purposes.
In addition to compensatory revenue, at least three federal and three other state funding sources provided revenue to school districts primarily for remedial purposes during the 1996-97 school year. Federal revenue sources included: Title I, emergency immigrant grants, and homeless students. Other state revenue sources were: targeted needs revenue (which combines assurance of mastery, limited English proficiency, and integration grants), low-income concentration grants, and first grade preparedness. We found that:
From fiscal year 1988-89 through 1996-97, total school district operating revenue in inflation-adjusted dollars grew 31 percent compared with a growth in remedial funds of 64 percent. Remedial revenue as a percent of total operating revenue increased about 27 percent, going from 4.1 percent to 5.2 percent.
From fiscal year 1989 through 1997, total state and federal aid for remedial education increased 64 percent in constant dollars, going from about $155 million in 1989 to $255 million in 1997. During this same period, state revenue grew more than twice as fast as federal revenue: 83 percent compared with 35 percent.
In reviewing how districts could spend this revenue, we learned that:
Although school districts generally received remedial funds based upon the number of students in poverty, the funds did not have to be spent on low-income students. Rather, statutes generally require that the money be spent on low-achieving students. While it was originally hoped that providing additional funds to districts would help offset or compensate for the effects of poverty on low-income students, the additional money must instead be used to compensate for regular instruction's inability to move all students along at grade level. However, most state funding for remedial education--compensatory revenue--did not even have to be spent on low-achieving students until the 1996-97 school year. Prior to that, districts could spend it for whatever they saw fit.
Not only do remedial funds not have to be spent on low-income students, but state laws allocating remedial education dollars give school districts considerable flexibility in how to spend remedial funds. Likewise, the federal government has loosened some of its restrictions on Title I expenditures. For the most part, districts can use remedial funds to provide a wide variety of services that may be directed at specific, low-achieving students or at the school as a whole. Services may be mainly academic, such as extra math or reading instruction either inside or outside the regular classroom or one-to-one tutoring, but may also include health, attendance, counseling, and safety programs.
What works?
In general, research has shown that remedial services funded through Title I have not been effective in closing the achievement gap between disadvantaged students and their nondisadvantaged peers. Most recently, the U.S. Department of Education issued its 1997 evaluation of Title I nationwide and reported that:
The department found that most of the variation among students in their level of achievement was related to individual or family characteristics, including family income, parental expectations, membership in a racial or ethnic group, limited-English proficiency, frequent changes in schools, disability, health problems, and having a single parent. Student participation in remedial activities that were paid for by Title I did not seem to have an effect on student achievement.
The U.S. Department of Education also looked at the characteristics of several Title I schools that performed better than other Title I schools. It found that the more successful schools usually grouped students by ability from first through sixth grades. They also had more experienced principals and less turnover among teachers, and there was more support for the school's mission by the community, parents, and teachers. In reading instruction, teachers emphasized comprehension along with the basics.
Another recent study looked at the effectiveness of several exemplary programs in a small number of Title I schools in high-poverty areas to see if these programs actually worked as expected. <FN U.S. Department of Education, Special Strategies for Educating Disadvantaged Children-- Final Report (Washington, D.C., April 1997).> The evaluation examined several nationally known programs, including the Comer school reform model, the Coalition of Essential Schools restructuring model for secondary schools, Success for All, Reading Recovery, the Paideia program, a computer-assisted program from the Computer Curriculum Corporation (CCC), and the METRA tutoring program. The evaluation also looked at a locally originated extended-year program and an extended-day program. Although the Reading Recovery programs involved too few students to analyze, results for the remaining programs showed that:
The remaining programs produced meager student progress at best, and in some schools student achievement declined. The evaluators also noted that student progress in any program was usually limited to the earliest grades.
Our review of educational research showed that only a few programs or strategies have consistently proven their worth in helping low-achieving students. We found that:
Several highly structured reading programs for the early grades, such as Reading Recovery, Success for All, and Direct Instruction, have a tutoring component and have consistently demonstrated effectiveness. Small class sizes were also effective and seemed to be the only strategy where increased funding for schools had a demonstrable impact.
However, even in effective programs, most of the gains were made by students in the earliest grades; much less is known about the effectiveness of remedial programs at the middle school or secondary school level. The effectiveness of any program also depends on how faithfully it is implemented as designed, and the effectiveness of a program might be limited inadvertently by other situations in a school.
Using data obtained from our survey of elementary, middle, and secondary principals in Minnesota, we estimated that:
As shown in the table below, elementary, middle, and secondary school principals estimated remedial education participation rates of 33, 26, and 19 percent, respectively, during the 1996-97 school year. Elementary, middle, and secondary schools with higher rates of student poverty reported participation rates of 37, 31, and 21 percent respectively, compared with rates of 26, 20, and 17 percent in schools with lower rates of student poverty.
Also, remedial students in schools with higher rates of student poverty were more likely than remedial students in schools with lower rates of student poverty to have limited-English proficiency, frequent school changes, poor attendance records, little home support, or were likely to have received inadequate instruction earlier in their educational careers. These problems became more pronounced, that is, were reported affecting more students, as school level increased, regardless of students' poverty.
To learn how students were actually served by remedial programs, we asked school principals to estimate the percentage of their low-achieving students who received remedial services in a variety of methods. We learned that:
Seventy-five percent of elementary schools reported that half or more of their remedial students received help from instructional aides in the regular classroom and 73 percent reported using small group instruction in the regular classroom. Also, 53 percent reported that half or more of their remedial students received individual tutoring by instructional aides, 48 percent reported having individual learning plans, and 46 percent reported that half or more of their students received small group instruction outside the regular classroom. In addition, elementary schools with higher rates of student poverty reported using significantly less individual tutoring by adult volunteers or peers (15 percent compared with 29 percent), but more individualized computer labs (49 percent compared with 39 percent) than schools with fewer students in poverty. As discussed earlier, one-to-one tutoring has been shown to be one of the most effective remedial methods, while the effectiveness of individualized computer instruction has been largely unproved.
In middle schools, 64 percent of the principals reported that half or more of their remedial students were assigned to an advisor, 57 percent reported that over half of remedial students had instructional aides in the classroom, and 53 percent reported that over half of remedial students received small group instruction within the regular classroom. Few middle school principals reported widespread use of individual tutors or computerized instruction.
No single method of remediation was dominant in secondary schools. Forty-seven percent of secondary principals reported that half or more of their remedial students had individual learning plans and 44 percent reported that the majority of remedial students received help via small group instruction in the regular classroom. Like their middle school counterparts, few secondary school principals indicated that individual tutoring was commonplace.
As discussed earlier, national research has shown that schools that used proprietary, research-based remedial strategies, such as Reading Recovery and Success for All, were more successful in remediating students than schools that relied on locally developed models. We found that:
Forty-three percent of the elementary schools compared with 29 percent of the middle and 8 percent of secondary schools reported using special, proprietary programs developed by others for remedial education. Also, elementary schools serving large proportions of students in poverty were also significantly more likely to be using such programs. Most frequently cited were: Reading Recovery, Higher Order Thinking Skills, Read Naturally, Computer Curriculum Corporation, and Success for All. Some of these programs, like Reading Recovery and Success for All, have been shown to be effective in national studies, while the effectiveness of others has yet to be proven on a large scale.
Finally, we questioned principals in schools that had students who failed one or more of the state's basic skills tests about what strategies, if any, they used during the 1996-97 school year to help these students. We found that:
More than half of the schools reported giving students practice tests (81 percent), spending more time on basic skills (77 percent), sharing students' test scores with teachers (60 percent), holding summer schools (59 percent), and meeting with students and parents (56 percent).
Elementary schools that had eighth-grade students were more likely than middle and secondary schools to develop individual learning plans for students. Along with middle schools, they were also more likely than secondary schools to extend the school day or have summer school to provide remedial services. On the other hand, secondary and middle schools were more likely than elementary schools to give their students practice tests to help them pass the basic skills tests.
The Department of Children, Families & Learning does not collect data on the effectiveness of remedial programs in general. It does, however, collect limited data on remedial programs that are funded through Title I. To analyze the effectiveness of these programs in Minnesota, we compared the average pre- and post-test scores of Title I schools in Minnesota with national averages. Until 1996, schools receiving Title I funds had to test participating students before and after they received remedial services. We looked at test results for the three most recent school years for which data were available (from 1992-93 to 1994-95) for grades 3 to 5 for both reading and mathematics. <FN Department of Children, Families & Learning, System Performance Measure Report (St. Paul, November 15, 1996); U.S. Department of Education, State Chapter I Participation and Achievement Information 1992-93 (Washington, D.C., 1994); State Chapter I Participation and Achievement Information 1993-94 (Washington, D.C., 1996), and State Chapter I Participation and Achievement Information 1994-95 (Washington, D.C., forthcoming).> This analysis showed that:
The largest difference between Minnesota and the national averages in grades 3 to 5 over three years was 3.7 NCEs. <FN Test results are expressed in terms of "normal-curve equivalents" or NCES, which range from 1 to 99 and can be compared across tests, years, and student populations.> In only 4 of 18 comparisons by subject, year, and grade did the difference between Minnesota and the national averages exceed 2 NCEs. While Title I students did show some progress, their test scores increased only slightly--not nearly enough to bring them up to grade-level standards. Overall, Title I programs have been judged to be ineffective nationally and the same can be said about Title I programs in Minnesota.
Nevertheless, some Minnesota schools have reported better results than others. In looking at what distinguished the more successful schools from the less successful ones, we found that:
The average reading test NCEs of third graders were about 1.5 higher in a school for every percentage point higher rate of school attendance; results for mathematics were similar. While this result does not mean that schools' test scores will automatically go up if attendance improves, on average, schools with better attendance had better test scores. The percentage of students eligible for a free or reduced-priced lunch was not significantly related to the Title I test scores.
We did a similar analysis on school and district average test scores on the Minnesota Basic Standards Tests in reading and math and found that:
Attendance had the strongest relationship with average school test scores of the variables that we examined. For every percentage point higher attendance rate, average school and district reading and mathematics scores were about 0.8 points higher. We found a much weaker relationship for poverty. For every percentage point higher rate of students eligible for subsidized lunch, average scores were about 0.1 points lower.
In general, we found that:
According to our review of the research literature, two remedial reading programs currently used by a small number of Minnesota schools have a proven record of effectiveness: Reading Recovery and Success for All. Our own analysis of Title I post-test scores also found that Reading Recovery had a positive impact. Yet our survey results, weighted to reflect statewide numbers, showed that only about 11 percent of elementary schools were using either of these programs during the 1996-97 school year. While another 10 percent of schools were using other "packaged" programs that may hold some promise, we did not have sufficient information on their effectiveness.
In contrast to the findings of national research, we observed that:
Ineffective tutoring might be due to the fact that schools generally used instructional aides rather than licensed teachers for one-to-one tutoring, and these aides may have needed more training and supervision. A 1997 survey by the Department of Children, Families & Learning of over 1,800 paraprofessionals in schools throughout the state found that 49 percent of remedial aides had no degrees beyond a high school diploma or its equivalent. <FN Department of Children, Families & Learning, Characteristics and Training Needs of Paraprofessionals in Minnesota (St. Paul, March 1997).> Moreover, only 39 percent of remedial paraprofessionals reported that they had any non-student contact planning time with licensed staff, even though about half of remedial aides reported that their typical activities included designing and preparing student instructional activities, modifying or adapting classroom curriculum, and designing individualized instructional plans for students.
We also examined average school scores on the basic skills tests in relation to remedial practices identified on our survey, while taking into account other variables, such as the school's attendance and poverty rates. We found that:
Schools reporting a higher percentage of students in classes with instructional aides in regular classrooms had slightly higher average scores on the basic standards tests, as did schools that gave their students practice tests. We found that 52 percent of schools had instructional aides serving half or more of the low-achieving students, and practice tests were given in 81 percent of schools that had students who failed the basic standards tests. Among schools with a majority of students failing the basic standards test in reading, 56 percent were offering a summer remedial program, compared with 45 percent where the majority of students passed the test. Schools where more students chose not to participate in the remedial program had slightly lower reading scores on the basic standards tests.
At the elementary level, extra instructional time was "strongly" emphasized in 35 percent of schools in our survey. Our analysis found that this practice was positively related to the reading progress of third-grade students receiving Title I services. Schools that had a Reading Recovery program also had a positive relationship with reading progress in Title I programs. There were too few schools in the survey that used Success for All or other reading programs, however, to do a statistical analysis of their effectiveness.
Finally, we found that:
For example, the St. Paul School District voted in 1997 to spend up to $500,000 on staff and programs to increase attendance at all school levels. St. Paul had recently discovered that about 40 percent of students had missed at least 15 days of school in the 1995-96 school year. <FN Maureen Smith, "St. Paul Schools Decide How to Spend Attendance Money," Star Tribune, October 21, 1997, B5.>
Remedial education is both a state and local responsibility. The state and federal governments have long provided extra funds to school districts to help low-achieving students, usually based upon some measure of student poverty. However, there has generally been no state requirement to provide remediation to students, no state definition of who must receive such help, and no consistent measure of achievement to identify low-achieving students.
Although schools are supposed to use remedial education revenue to increase student achievement, schools and districts do not receive remedial funds based on a direct measure of student achievement. Currently, most remedial aid, both state and federal, is based on a measure of poverty; that is, the percentage of students who are eligible to receive a subsidized lunch. Our analysis showed that this measure of poverty had, at most, a moderate, negative relationship with student achievement, as measured by average Title I and Minnesota Basic Standards Tests scores.
Furthermore, we did not find a strong relationship between poverty and participation in remedial programs. In elementary and middle schools, only 25 percent of the variation in the percentage of students who were receiving remedial services was related to the level of student poverty in schools. The link between poverty and remedial students dropped to 4 percent in secondary schools.
Finally, for schools administering the Minnesota Basic Standards Tests, we found no relationship between student participation in remedial programs and student achievement, as measured by average test scores or passing rates. Schools with low passing rates on the basic skills tests did not tend to provide remedial services to a greater percentage of their enrollment than schools with higher passing rates.
Taken together, the overall lack of strong relationships between poverty, participation, and achievement suggest that, if revenue for remediation is allocated strictly in terms of student poverty at the building level, particularly at the secondary level, schools may not receive remedial revenue in proportion to their students' needs, as currently identified. Therefore, we recommend that:
This might be done by means of students' scores on standardized achievement tests, such as those planned under the state's new education accountability system. The 1997 Legislature directed the Department of Children, Families & Learning to develop a statewide testing and reporting system that includes testing all third, fifth, and eighth grade students annually. <FN Minn. Laws (1997), ch. 138.> The department expects to begin testing third and fifth graders in February and March of 1998 using the Minnesota Comprehensive Assessment Test, which will measure student progress along state standards. Thus, consistent statewide data will be available to help identify how many students may need remediation, at least beginning in the third grade.
It should be noted that the Department of Children, Families & Learning is currently examining various ways to link performance and funding. The 1997 Legislature directed that the department, in consultation with the State Board of Education and other stakeholders, recommend to the Legislature performance funding options for successful and at-risk schools, to be implemented during the 1999-2000 school year. <FN Minn. Laws (First Special Session, 1997), ch.4, art. 5, sec. 25.>
Regardless of how districts or schools receive remedial education funds, the money must be spent effectively if the state hopes to raise the academic achievement of students performing below grade-level standards. Although a small number of schools have implemented remedial programs of proven effectiveness, much more could be done statewide. Thus, we recommend that:
Our evaluation did not examine the Department of Children, Families & Learning's role in assuring that schools identify low-achieving students and provide them with effective remedial services. However, with the advent of the state's new education accountability system, the department will be in a unique position not only to track school performance, but to also identify schools that are making better or worse than expected progress in having all students performing at grade-level standards. The department should be able to use these data to encourage schools to adopt promising remedial methods.
Finally, we recommend that:
We encourage the Department of Children, Families & Learning to provide technical assistance to schools to help them judge the merits of various remedial strategies that have been shown to be effective elsewhere and help schools implement those that seem appropriate. Also, the department should help schools routinely evaluate the effectiveness of their remedial programs.
We do not think it is necessary to provide increased funding for remedial programs to encourage greater use of proven remedial methods. As we pointed out earlier, districts already receive most remedial revenue with few strings attached. Adopting proven methods of remediation and discarding unproved or ineffective methods are possible within current funding levels. As noted earlier, 10 to 20 percent of elementary schools used some proprietary remedial packages of proven effectiveness during the 1996-97 school year, with more schools planning to implement them during the 1997-98 school year.
Finally, our study, as well as other recent reports on school districts' use of compensatory revenue, have pointed out the wide array of activities for which school districts may spend remedial revenue. Our review of the literature suggests that the list of activities may be excessively broad, especially at the elementary level where considerable research has already been done on effective remedial programs. However, because less is known about the effectiveness of various remedial strategies for older students, we think that it may be difficult for the Legislature to mandate specific remedial services or programs for all grade levels. At the same time, if the Department of Children, Families & Learning uses its new education accountability system to monitor and report on schools' progress in assuring that all students are meeting state standards, school districts will be under greater pressure to adopt proven methods and discard unproven ones.
More InformationThe Program Evaluation Division was directed to conduct this study by the Legislative Audit Commission on May, 1997. For a copy of the full report, entitled "Remedial Education", (98-02), 76 pp., published on January 21, 1998, please call 651/296-4708, e-mail Legislative.Auditor@state.mn.us, or write to Office of the Legislative Auditor, 658 Cedar St., St. Paul, MN 55155.
Staff who worked on this project were Jo Vos (project manager), and Stephen Coleman.
