OFFICE OF THE LEGISLATIVE AUDITOR STATE OF MINNESOTA

EVALUATION REPORT

MARCH 2004 PROGRAM EVALUATION DIVISION Centennial Building -Suite 140 658 Cedar Street -St. Paul, MN 55155 Telephone: 651-296-4708 Fax: 651-296-4712 E-mail:legislative.auditor@state.mn.us Web Site: http://www.auditor.leg.state.mn.us

Photo Credits: The photograph on the No Child Left Behind report cover was provided by the Minnesota Department of Education.

Program Evaluation Division The Minnesota Office of the Legislative Auditor was established in 1973, replacing the century-old Public Examiner's Office. Its role is to audit and evaluate public programs and ensure accountability for the expenditure of public funds. In 1975, the Legislature created the Program Evaluation Division within the auditor's office. The division's mission, as set forth in law, is to determine the degree to which activities and programs entered into or funded by the state are accomplishing their goals and objectives and utilizing resources efficiently.

Topics for evaluation are approved by the Legislative Audit Commission (LAC), a 16-member joint, bipartisan commission. The division's reports, however, are solely the responsibility of the Office of the Legislative Auditor. Findings, conclusions, and recommendations do not necessarily reflect the views of the LAC or any of its members. A list of recent evaluations is on the last page of this report. A more complete list is available at OLA's website (www. auditor. leg. state. mn. us), as are copies of evaluation reports.

The Office of the Legislative Auditor also includes a Financial Audit Division, which annually conducts a statewide audit of the 25 largest agencies, an audit of federal funds, and approximately 40 financial and compliance audits of individual state agencies. The division also investigates allegations of improper actions by state employees.

Professional Staff James Nobles, Legislative Auditor, Joel Alter, Valerie Bombach, David Chein, Jody Hauer, Adrienne Howard, Daniel Jacobson, Deborah Junod, Carrie Meyerhoff, John Patterson, Judith Randall, Jan Sandberg, Jo Vos, John Yunker

Support Staff Barbara Wing This document can be made available in alternative formats, such as large print, Braille, or audio tape, by calling 651-296-8976 Voice, or the Minnesota Relay Service at 651-297-5353 or 1-800-627-3529.

e-mail:legislative.auditor@state.mn.us

Reports of the Office of the Legislative Auditor are available at our Web Site: http://www.auditor.leg.state.mn.us

OFFICE OF THE LEGISLATIVE AUDITOR State of Minnesota • James Nobles, Legislative Auditor February 26, 2004 Members Legislative Audit Commission The No Child Left Behind (NCLB) Act imposes new requirements for states that accept federal money for certain education programs. Some Minnesota policy makers have expressed concerns about NCLB, and the Legislative Audit Commission asked the Office of the Legislative Auditor to examine the law's impacts on Minnesota schools.

We found that local education officials in Minnesota generally embrace NCLB's goal of helping all children succeed in school, but many told us they think the law is costly, unrealistic, and punitive. We also found that it will be very difficult for schools and school districts to comply with the law's specific targets for student achievement. Even assuming substantial improvement in student achievement, we estimate that most Minnesota schools will not meet NCLB's goals for student proficiency by 2014, and many will be subject to significant consequences.

Minnesota is still in the early stages of implementing NCLB, and various factors will affect the law's eventual fiscal impact. However, school districts will face significant new costs to implement NCLB, including new requirements regarding student assessment, staff qualifications, and sanctions and services for underperforming schools.

Minnesota policy makers could "opt out" of the NCLB law, but the state would lose a large amount of federal education revenue by taking this action. Alternatively, if Minnesota continues to participate in NCLB, state officials could seek changes in the federal law that would make its goals more achievable.

Our report was researched and written by Joel Alter and John Patterson (project co-managers) and Adrienne Howard, with research assistance from the University of Minnesota's Office of Educational Accountability. We received the full cooperation of the Minnesota Department of Education.

Sincerely, /s/ James Nobles James Nobles Legislative Auditor

Room 140, 658 Cedar Street, St. Paul, Minnesota 55155-1603 • Tel: 651/296-4708 • Fax: 651/296-4712 E-mail:legislative.auditor@state.mn.us • TDD Relay: 651/297-5353 • Website: www.auditor.leg.state.mn.us

Table of Content
SUMMARY
INTRODUCTION 1 1.
BACKGROUND
Targeting "Achievement Gaps" Among Students
NCLB Requirements and Fund Allocation
NCLB Revenues
2. NCLB's IMPACT ON MINNESOTA'S EDUCATION ACCOUNTABILITY SYSTEM
Consistency of NCLB with Existing Minnesota Policy
Perceptions of Education Officials
3. COMPLIANCE WITH "ADEQUATE YEARLY PROGRESS" (AYP) REQUIREMENTS
Determining AYP in Minnesota
AYP Determinations for the 2002-03 School Year
AYP Determinations for Future Years
Other AYP and Account ability Issues
4. FISCAL IMPACTS
Back ground
Over view of NCLB Costs
Costs Related to Assessment Development and Administration
Costs of NCLB-Prescribed Consequences for Low Performance
Costs Related to Teacher and Paraprofessional Qualifications
Costs Related to Curriculum Alignment
Other Costs
Will NCLB's New Revenues Cover Its New Costs?
School District Responses to NCLB Costs
"Opting Out" of NCLB
Tracking NCLB Costs in the Future
SUMMARY OF RECOMMENDATIONS
FURTHER READING
AGENCY RESPONSE
RECENT PROGRAM EVALUATIONS

List of Tables and Figures Tables Page 1.1 Factors Associated With School Achievement 5 1.2 Significant Title I, Part A Requirements 6 1.3 Major NCLB Programs and Funding 9 2.1 Comparison of Key NCLB Accountability Requirements with Minnesota's Pre-NCLB Requirements 17 2.2 Components of "Adequate Yearly Progress" 19 2.3 NCLB Requirements for Schools Failing to Make Adequate Yearly Progress 20 2.4 Superintendents' Perceptions About Using Uniform Standards to Measure Students' Academic Proficiency 23 2.5 Superintendents' Perceptions About NCLB-Prescribed Consequences for Schools Failing to Make Adequate Yearly Progress 25 2.6 Superintendents' Perceptions About NCLB-Prescribed Assessments 27 3.1 Achievement Levels on the Minnesota Comprehensive Assessments (MCAs) 31 3.2 Calculation of a Minnesota School's Proficiency Rate 32 3.3 Minnesota's Annual Proficiency Targets 32 3.4 NCLB's "Safe Harbor" Provision 33 3.5 Schools Failing to Make Adequate Yearly Progress, by Various Measures, 2002-03 School Year 34 3.6 Schools Held Accountable on Proficiency Measures, by Subgroup, 2002-03 School Year 35 3.7 Schools Failing to Make Adequate Yearly Progress in Proficiency, by Subgroup, 2002-03 School Year 36 3.8 Scenarios Used to Simulate Schools' Future AYP Status 39 3.9 AYP Status of Minnesota Elementary Schools, 2008 and 2014 42 3.10 Percentage of Elementary Schools Accountable for the Proficiency of Each Subgroup by District Type, 2014 43 3.11 Proficiency Failure by Sub group, 2014 -Modest Improvement Scenario 44 4.1 Fiscal Impact of NCLB-Initiated Activities 61 4.2 Superintendent's Opinions Regarding Which NCLB Requirements Will be the Most Costly 62 4.3 Minnesota's Schedule for Initiating Assessments for Performance Reporting 64 4.4 NCLB's Consequences for Repeated Failure to Make AYP 67 4.5 Estimates of the Percentage of Minnesota Title I Schools Offering School Choice and Supplemental Services, 2008 and 2014 71 4.6 NCLB Teacher and Paraprofessional Qualification Requirements 74 4.7 New ESEA Formula Funding 82 4.8 New Title I, Part A Costs

Tables Page 4.9 Ways That School Districts Have Funded (or Intend to Fund) New Requirements of the NCLB Act 85

4.10 Changes that School Districts Made (or Will Likely Make) As a Direct Result of NCLB 86

Figures 3.1 Percentage of Elementary Schools Failing to Make AYP in Proficiency 41

Summary Major Findings The federal No Child Left Behind (NCLB) Act imposes rigorous new requirements on Minnesota's education accountability system (pp. 16-21).

While most education officials in Minnesota embrace the underlying goals of NCLB, many school district superintendents believe that NCLB is costly, unrealistic, and punitive (pp. 21-28). Local officials have particular concerns about holding students with disabilities and limited English skills to the same standards as other students (p. 23).

Even if Minnesota students' math and reading test scores improve significantly in coming years, there will likely be large increases in the number of schools failing to make "adequate yearly progress" (AYP), as defined by NCLB. More than 80 percent of Minnesota elementary schools would not make AYP by 2014, according to a simulation conducted for our office, and many of these schools would face the prospect of restructuring or other serious sanctions prescribed by NCLB (pp. 40-44).

NCLB has had limited state and local fiscal impacts so far, but many school districts will likely bear significant new costs in future years for student assessments, sanctions for low-performing schools, and compliance with stricter requirements for staff qualifications. These costs cannot be estimated with precision, but it is quite possible that NCLB's new costs will exceed the increase in NCLB revenues (pp. 60 and 84).

However, Minnesota could lose the majority of its projected $216 million in federal funding for state fiscal year 2005 if it "opts out" of the accountability provisions of NCLB. While federal NCLB funding is less than 4 percent of school districts' operating budgets, relatively few school district superintendents favor opting out (pp. 86 and 90).

Recommendations Changes in the federal NCLB law may be necessary for states to have a realistic chance of complying with the law's goals for student achievement. At the state level, we recommend:

The Minnesota Department of Education should provide the Legislature with (1) a plan for how measures of individual student achievement growth could be incorporated into the state's AYP determination process (p. 51), and (2) an assessment of the overall validity and reliability of Minnesota's educational accountability system (p. 54).

The Legislature should require the department to (1) annually report on school district expenditures related to sanctions for low-performing schools (p. 90), and (2) specify how it will monitor the quality and effectiveness of supplemental educational services providers (p. 56).

It is likely that most Minnesota schools will not be able to meet the goals of No Child Left Behind, and this could trigger expensive sanctions.

Report Summary In January 2002, President Bush signed into law the No Child Left Behind (NCLB) Act. This law is the latest version of the Elementary and Secondary Education Act, first passed in 1965. The law's stated purpose is to close the "achievement gaps" between high-and low-performing students. It proposes to accomplish this through improved accountability, expanded educational choices, and more funding.

NCLB requires public reporting on the extent to which schools are making "adequate yearly progress" (AYP) toward the goal of having all students proficient in reading and math by the 2013-14 school year. The Minnesota Department of Education makes these AYP determinations, based on reading and math test scores, test participation rates, attendance rates, and graduation rates. Presently, Minnesota schools test students' reading and math in grades 3, 5, and 7, and they assess reading in grade 10 and math in grade 11. NCLB requires annual reading and math assessments in grades 3 through 8, plus in one year of high school.

Individual schools are held accountable for their overall performance and for the performance of various student subgroups within the school population. 1 NCLB prescribes up to 37 performance targets that may be used to assess a school's performance. Most schools are not subject to all of these targets, but a school's failure to meet any of the applicable targets results in a state determination that the school has failed to make AYP.

The law specifies sanctions for schools that fail to make AYP for at least two consecutive years. School districts may have to offer parents the options of transferring their children to other schools or enrolling them in "supplemental educational services" (such as after-school tutoring). If schools continue to under-perform, NCLB subjects them to "corrective actions" (such as replacement of staff or curriculum) or "restructuring."

To help meet the law's education goals, Minnesota is projected to receive $216 million in NCLB formula grants in state fiscal year 2005. This is 24 percent more than Minnesota received three years earlier (after adjusting for inflation), although Minnesota's funding increase was smaller than the increase received by most other states.

Schools Will Have Increasing Difficulty Meeting NCLB's Performance Targets

In the 2002-03 school year, about 8 percent of Minnesota's schools did not make AYP, and only 5 of Minnesota's 342 school districts were required by NCLB to offer school choice or supplemental educational services to parents. But the number of schools subject to NCLB sanctions will likely grow, for at least two reasons. First, the proficiency targets used to measure school performance will begin increasing in the 2005-06 school year; by 2013-14, NCLB will expect 100 percent of students to be proficient. Second, as Minnesota implements assessments in more grades, many schools will be held accountable for the performance of more NCLB-specified student subgroups. 2

With the help of the University of Minnesota's Office of Educational Accountability, we simulated the likelihood that Minnesota elementary schools will fail to make AYP in coming years. This analysis was based on 2003 statewide test data, using assumptions ranging from "no improvement" to "high improvement" in the future achievement levels of students. Under these various scenarios, the simulations showed that between 80 and 100 percent of Minnesota's elementary schools would fail to make AYP by 2014. In addition, the simulations showed that 35 to 76 percent of Minnesota's elementary schools that receive federal "Title I" funding for disadvantaged students would be subject to NCLB-prescribed restructuring within the next decade. In sum, even if there are large, sustained improvements in student achievement, many Minnesota schools will likely struggle to comply with the ambitious targets set by NCLB.

NO CHILD LEFT BEHIND NCLB imposes new accountability requirements on states.

1 Minnesota schools are held accountable for the following student subgroups: white, black, Asian, American Indian, Hispanic, limited-English, special education, and low income students.

2 A school is held accountable for a subgroup of its population only if the number of tested students in the subgroup exceeds a minimum number designated by the state. Consequently, as the number of tested students rises, a growing number of subgroups will surpass this minimum threshold.

Many aspects of the AYP determination process are prescribed by the federal NCLB law and are not subject to change by individual states. Thus, it is questionable whether Minnesota policy makers could, through state action alone, significantly improve schools' likelihood of making AYP. Minnesota Department of Education officials told us they would like to find ways to recognize year-to-year growth in individual student achievement levels during the AYP determination process, in addition to measuring achievement against an absolute standard. But the department has not yet specified how it would do this, and it is questionable whether such an approach would meet federal requirements. We recommend that the department outline how it proposes to incorporate measures of individual achievement growth into the AYP process. We also recommend that the department assess the overall validity and reliability of Minnesota's education accountability system.

School Districts Face Growing NCLB-Related Costs

Many of NCLB's new requirements have not yet been fully implemented in Minnesota. Thus, the implementation costs borne by the Minnesota Department of Education and local school districts have been modest, so far.

Although Minnesota had implemented (or planned to implement) several statewide tests before NCLB passed, the costs of some forthcoming Minnesota assessments are attributable to NCLB— specifically, reading and math assessments in grades 4, 6, and 8, three science assessments, and listening and speaking assessments for limited-English students. The state and local costs to administer these assessments will total roughly $19 million annually.

School districts will bear other NCLB-related costs in coming years, although they are difficult to accurately forecast. Districts could spend up to $20 million of federal or other revenues annually to comply with NCLB requirements for school choice and supplemental services, depending partly on the number of schools failing to make AYP. In addition, many schools may be subject to "corrective actions" or "restructuring" because of persistent under-performance, although it is unclear what specific actions will be pursued by school districts and the Minnesota Department of Education. 3 Also, due to NCLB, schools must comply with more stringent requirements regarding teacher and paraprofessional qualifications, and some school districts will incur higher costs to attract or retain staff who meet these standards. Furthermore, schools are expected to ensure that all students are proficient by 2014, although it is unclear what strategies and resources this might require.

SUMMARY xi The Minnesota Department of Education should specify how measures of individual student achievement growth could be integrated with NCLB-prescribed measures of school progress.

3 The department has assembled a committee to advise it on NCLB sanctions, including legislative changes that may be required in 2005.

It is plausible that new, NCLB-related costs will exceed the $42 million (inflation-adjusted) increase in annual revenues that Minnesota is expected to receive under NCLB, but this will be unclear until school districts proceed further with NCLB implementation. In a statewide survey, less than 3 percent of Minnesota superintendents said that they expected their school district's share of the increased federal revenues to cover the cost of new spending required by NCLB.

Key NCLB Provisions Lack the Support of Local School Officials

Minnesota was implementing its own education accountability system at the time that NCLB became law. The Legislature had adopted academic standards, mandated statewide tests in several grades, and required the measurement of progress by schools and individual students.

NCLB reinforced some parts of Minnesota's emerging accountability system, but it also imposed new, more rigorous requirements. Compared with previous requirements, NCLB set more ambitious goals, required more tests and performance measures, specified stronger sanctions, and held schools more accountable for the performance of student subgroups. Officials with the Minnesota Department of Education strongly believe that NCLB will improve student achievement and close achievement gaps among student subgroups.

Meanwhile, although many Minnesota school district superintendents support the act's emphasis on improving achievement levels of all children, most superintendents view the act as unrealistic, costly, and punitive. Only 17 percent of superintendents said that it is "likely" or "very likely" that their districts could help all students become proficient by 2013-14. Nearly three-fourths of superintendents said that, contrary to NCLB requirements, special education and limited-English students should not be held to the same standards of academic proficiency as other students. For each of the various NCLB subgroups, a majority of superintendents said that schools should not be required to face NCLB-prescribed consequences for the subgroup's persistent failure to make AYP. Only 7 percent of superintendents said that the educational benefits of NCLB will outweigh any adverse impacts the act will have on their districts.

Some legislators have asked whether Minnesota should simply ignore the federal NCLB requirements. This report offers no recommendation, and policy makers weighing this issue might consider various factors— such as the appropriateness of the federal government's role in education, the fiscal implications of noncompliance for the state, and the overall impact of NCLB on schools. But, by "opting out" of NCLB, Minnesota would risk losing the majority of its funding under the Elementary and Secondary Education Act ($ 216 million), and it is unclear whether the cost savings from opting out would offset the revenue losses. Less than 20 percent of superintendents said they would favor Minnesota opting out of NCLB.

NO CHILD LEFT BEHIND At this time, it is unclear whether the cost savings from "opting out" of NCLB w ould offset the state's revenue losses.

Introduction In January 2002, President Bush signed into law the No Child Left Behind (NCLB) Act, which is the most recent reauthorization of the 1965 Elementary and Secondary Education Act. The act establishes ambitious education goals and prescribes mechanisms for holding schools, school districts, and states accountable for their performance. Under NCLB, all students are expected to be proficient in reading and math by the 2013-14 school year. For state fiscal year 2005, the federal government expects to grant Minnesota $216 million to implement the provisions of the act and provide services to students.

The law is in its early stages of implementation. Still, there was considerable discussion during the 2003 legislative session about the law's potential cost implications, and the Legislative Audit Commission asked our office to assess the law's likely impact. Our evaluation addressed the following questions:

What costs will NCLB impose on the state and school districts, and what factors are likely to affect the magnitude of those costs?

What is the likelihood that Minnesota school districts and schools will achieve "adequate yearly progress" toward the goal of 100 percent proficiency?

Will the increase in federal revenues that Minnesota receives under NCLB cover the new costs imposed by the act?

What would be the fiscal implications if Minnesota "opted out" of NCLB? To what extent do Minnesota education officials support the goals and approaches of NCLB? We used several research methods to address these questions. To help us evaluate NCLB costs, we interviewed staff from the Minnesota Department of Education and nine school districts. 1 We also asked the department and these districts to estimate their past and future expenditures in more than 20 NCLB-related categories, covering seven broad areas: (1) administering NCLB's general provisions, (2) establishing academic content standards and aligning curricula, (3) assessing student proficiency, (4) monitoring and reporting school district and school performance, (5) sanctioning low-performing schools, (6) improving teacher and paraprofessional qualifications, and (7) increasing parental involvement. 2 We considered these estimates informative but not definitive, as discussed in Chapter 4. Our assessment of fiscal impacts focused on the costs associated with requirements in Title I, Part A of the NCLB Act, mainly because people we interviewed did not express concerns about the costs of other portions of the act.

1 The school districts were Minneapolis, St. Paul, Osseo, Bloomington, Rochester, Shakopee, Willmar, Detroit Lakes, and Mahnomen. During site visits, we usually talked with district superintendents and staff familiar with curriculum, assessments, personnel issues, and Title I services.

We reviewed the requirements and funding provisions of the NCLB Act, and we examined the requirements of federal and state laws that existed prior to its enactment. We also interviewed representatives of Minnesota education advocacy groups, as well as education officials in selected other states.

In November and December 2003, we conducted a statewide survey of Minnesota school district superintendents and charter school directors. The primary purpose of the survey was to document general perceptions regarding NCLB's goals, implementation, and impacts. We received responses from 95 percent of the superintendents and 86 percent of the charter school directors.

By analyzing current and projected school performance, we assessed the likelihood that Minnesota schools and school districts will make "adequate yearly progress" (as defined by NCLB) toward the goal of 100 percent proficiency. Specifically, we obtained and analyzed school-specific data on the academic performance of all Minnesota public schools for the 2002-03 school year. In addition, we contracted with the Office of Educational Accountability at the University of Minnesota to simulate the number of schools that might be expected to make "adequate yearly progress" in future years, using various assumptions that we identified. 3

To assess the consequences of opting out of NCLB, we talked with officials from the U. S. and Minnesota departments of education. We also interviewed staff with the National Conference of State Legislatures regarding other states' investigations into this issue.

Chapter 1 provides a brief overview of the NCLB Act and the related revenues that Minnesota receives. Chapter 2 discusses how NCLB's provisions affected the direction of Minnesota's emerging educational accountability system and how these provisions have been perceived by state and local education officials. Chapter 3 looks at Minnesota's implementation of the NCLB-mandated concept of "adequate yearly progress," a key component of the act's accountability provisions. Chapter 4 examines the fiscal impacts of the NCLB Act and the implications if the state "opted out" of NCLB.

2 NO CHILD LEFT BEHIND

2 For all cost categories, we requested estimates for fiscal years 2002 through 2005; for some categories, we also requested estimates for fiscal years 2006 through 2008.

3 The Office of Educational Accountability was created by the 1997 Legislature as an independent agency to advise legislative committees and the education commissioner on education accountability issues.

Background SUMMARY In January 2002, President Bush signed into law the No Child Left Behind (NCLB) Act with the goal of closing the educational achievement gap among students. In Minnesota and across the country, low income and minority students have not had the same level of academic success as their counterparts. By increasing educational funding and demanding greater accountability, the federal government hopes to close this gap and make every child proficient with respect to state academic standards in reading and math by the 2013-14 school year. However, while Minnesota received an increase in funding during the first two years of NCLB, the state's NCLB funding is projected to decline in state fiscal year 2005.

The federal "No Child Left Behind" Act was passed by Congress in late 2001 and signed into law by President George W. Bush on January 8, 2002. The act is the most recent reauthorization of the Elementary and Secondary Education Act (ESEA), enacted in 1965. 1 This chapter reviews the provisions of NCLB and addresses the following questions:

What is the purpose of the NCLB Act? What are the main requirements of the NCLB Act, particularly those related to standards, accountability, and staff qualifications?

How much funding does Minnesota receive under the act, and how has Minnesota's federal funding changed over time?

TARGETING " ACHIEVEMENT GAPS" AMONG STUDENTS The stated aim of the NCLB Act is "to close the achievement gap with accountability, flexibility, and choice, so that no child is left behind." 2 The act says that this "achievement gap" refers to the differences between high-and low-performing children, "especially the achievement gaps between minority and nonminority students, and between disadvantaged children and their more advantaged peers." 3

The federal law called "No Child Left Behind" (NCLB) took effect in January 2002.

1 By periodically "reauthorizing" an act, Congress extends the act and may amend its provisions. 2 No Child Left Behind Act of 2001, Pub. L. No. 107-110. This law's full title is "An Act to close the achievement gap with accountability, flexibility, and choice, so that no child is left behind."

The only nationally representative assessment of American students— the National Assessment of Educational Progress (NAEP)— has consistently documented the existence of such achievement gaps. For example, the average 2003 NAEP reading score for 4th -grade students nationally was 229 for white students, 226 for Asian students, 202 for American Indian students, 200 for Hispanic students, and 198 for black students. In addition, 4th -grade students from lower income families (specifically, students eligible to receive free or reduced-price lunches) had an average reading score of 201 in 2003, well below the average score of 229 for other students. 4

Similarly, results from standardized tests given to Minnesota students have shown persistent achievement gaps. On the basic skills test that Minnesota students must pass to graduate from high school, the percentage of 8th -grade students passing the 2003 reading test ranged from 87 percent for white students to 49 percent for black students. 5 Likewise, the average 3rd -grade reading score on the 2003 Minnesota Comprehensive Assessment (MCA) was 1548 for white students, compared with 1442 for American Indian students, 1435 for Asian students, 1382 for black students, and 1375 for Hispanic students. Third-grade students eligible for free and reduced-price lunches had an average MCA reading score of 1425, compared with an average score of 1559 for other students. Such results led Minnesota's education commissioner to conclude: "The sad fact is that Minnesota, while a national leader in overall student achievement, ranks near the bottom [among states] in terms of the achievement gap. This is unacceptable." 6

Recently, a respected educational research and testing organization issued a report summarizing previous research about factors associated with educational achievement. As shown in Table 1.1, the report identified 14 "correlates of achievement"— some that are within the control of a school system, and some that are not. The report said that "gaps in school achievement… have deep roots— deep in out-of-school experiences and deep in the structures of schools. Inequality is like an unwanted guest who comes early and stays late." 7

4 NO CHILD LEFT BEHIND Nationally and in Minnesota, the achievement levels of some categories of students have persistently lagged behind their peers.

3 No Child Left Behind Act, §1001. 4National Center for Education Statistics (NCES), "Average Reading Scale Scores, by Race/ Ethnicity, Grades 4 and 8: 1992-2003," http://nces.ed.gov/nationsreportcard/reading/results2003/raceethnicity.asp, accessed December 19, 2003; NCES, "Average Reading Scale Scores, by Student Eligibility for Free/Reduced-Price School Lunch, Grades 4 and 8: 1998-2003," http://nces.ed.gov/nationsreportcard/reading/results2003/lunch.asp, accessed December 19, 2003.

5These white and black rates only include non-Hispanic persons. Other passage rates were 62 percent for Asians, 59 percent for American Indians, and 55 percent for Hispanics. See Minnesota Department of Education, "State of Minnesota Data Analysis,"

http://education.state.mn.us/html/intro_ schools_ mde_ analysis.htm, accessed January 26, 2004. 6Commissioner Cheri Pierson Yecke, "Closing the Achievement Gap: Why Minnesota's Accountability Plan Must Address Achievement Disparities Among Our Students," Presentation to Citizen's League Forum, Minneapolis, May 22, 2003, http://education.state.mn.us/stellent/groups/public/documents/translatedcontent/pub_ 041625. jsp, accessed February 2, 2004. 7Paul E. Barton, Parsing the Achievement Gap: Baselines for Tracking Progress( Princeton, NJ: Educational Testing Center, October 2003), 36.

The NCLB Act sets an ambitious goal for closing student achievement gaps. The portion of the act that focuses on educational services for disadvantaged students (called "Title I") says:

The purpose of [Title I] is to ensure that all children have a fair, equal, and significant opportunity to obtain a high-quality education and reach, at a minimum, proficiency on challenging state academic achievement standards and state academic assessments. 8

The act requires each state to define "proficiency" and set a timeline for achieving proficiency among all students. 9 However, each state's timeline must ensure that all students are proficient by the 2013-14 school year. 10

NCLB REQUIREMENTS AND FUND ALLOCATION

In order to achieve the goal of 100 percent proficiency for all children by the 2013-14 school year, NCLB outlines extensive educational activities that all states receiving NCLB funding must carry out. The act itself is large— encompassing 670 pages, 10 titles, and 43 parts. Yet, the heart of the act is contained in Title I, Part A, which funds educational services for disadvantaged students. As we discuss later, Title I, Part A accounts for roughly half of the funding that Minnesota receives under NCLB. In addition, Title I, Part A establishes NCLB's k ey accountability requirements— shown in Table 1.2— to help ensure that all students become proficient. While other titles and parts of the act provide additional grants to improve educational performance, they largely support the efforts of schools under Title I, Part A. In addition, based on a review of NCLB summaries and interviews with stakeholders, including state and district officials, we believe that requirements in sections of NCLB other than Title I, Part A will likely have relatively minor impacts. 11 Consequently, our study focused on Title I, Part A.

BACKGROUND 5 Table 1.1: Factors Associated With School Achievement School-Related Factors Rigor of curriculum Teacher preparation

Teacher attendance and experience Class size Use of technology-assisted instruction School safety

External Factors Parental involvement in children's schooling Student mobility

Birth weight Lead poisoning Hunger/nutrition Reading to young children Television watching One-vs. two-parent families

NOTE: The author identified these "correlates of achievement" based on a review of existing research. He concluded that research has documented differences between minority students and other students in each of these areas. SOURCE: Paul E. Barton, Parsing the Achievement Gap: Baselines for Tracking Progress( Princeton, NJ: Educational Testing Service, October 2003), 7.

NCLB aims to ensure that all children meet challenging academic standards, bu t each state is allowed to set its own standards.

8No Child Left Behind Act, §1001. 9Minnesota defines "proficiency" as a level of performance where students are working successfully on materials at their grade level. In Chapter 3, we describe the scoring levels on Minnesota assessments that indicate "proficiency."

10 No Child Left Behind Act, §1111( b)( 2)( F).

Table 1.2: Significant Title I, Part A Requirements Plan a single, statewide accountability system that will track each school district's and school's progress toward 100 percent proficiency. Develop statewide content standards in reading, math, and science that identify what students are expected to know. Develop and administer reading and math assessments in grades 3, 4, 5, 6, 7, and 8 and once in high school that measure each student's proficiency with respect to the state's content standards. Develop and administer science assessments for grade spans 3-5, 6-9, and 10-12 that measure each student's proficiency with respect to the state's content standards.

Develop and administer assessments of English proficiency in reading, writing, listening, and speaking. Annually collect, verify, and analyze test scores to determine if school districts and schools are making "adequate yearly progress" (AYP) toward 100 percent proficiency in reading and math by the 2013-14 school year. Produce state, school district, and school report cards. Disseminate this and other information to parents and the public.

For schools that receive Title I, Part A funding and fail to make AYP for two or more consecutive years, develop and implement improvement plans and provide students with the option of transferring to schools that make AYP. For schools that receive Title I, Part A funding and fail to make AYP for three or more consecutive years, offer supplemental services outside the school day to low-performing students. For schools that receive Title I, Part A funding and fail to make AYP for four or more consecutive years, take corrective actions, such as curriculum or staff changes.

For schools that receive Title I, Part A funding and fail to make AYP for five or more consecutive years, plan and later implement school restructuring. Ensure that all teachers of core academic subjects are "highly qualified" by the end of the 2005-06 school year. Ensure that all paraprofessionals working in Title I, Part A programs meet NCLB qualifications by January 2006. Set annual measurable objectives concerning the provision of "high-quality" professional development for teachers. Ensure that school districts and schools meet these objectives. Implement activities to involve parents in programs funded by Title I, Part A.

NOTE: For school districts that repeatedly fail to make AYP, NCLB imposes an analogous set of sanctions to those outlined in this table for failing schools.

SOURCES: No Child Left Behind Act, Title I, Part A; U. S. Department of Education, Office of Elementary and Secondary Education, No Child Left Behind: A Desktop Reference 2002( Washington, D. C., 2002), 9-21; Education Commission of the States, No Child Left Behind: The Challenge and Opportunities of ESEA 2001 (Denver, CO: Education Commission of the States, March 2002), 1-24; Education Commission of the States, No Child Left Behind: State Requirements Under NCLB (Denver, CO: Education Commission of the States, January 2003); and Education Policy Reform Research Institute, "No Child Left Behind Act of 2001: Title I-Related Provisions," http://www.eprri.org/titleI.html, accessed July 9, 2003.

States receiving NCLB funding are required to implement new accountability measures.

NCLB differs from previous versions of the Elementary and Secondary Education Act because it requires states to establish a single accountability system for all schools, not just schools receiving Title I, Part A funds. As described in Table 1.2, states are required to evaluate each school's progress toward the goal of having all children proficient in math and reading by the 2013-14 school year. However, only Title I schools are subject to the sanctions prescribed in NCLB for low-performing schools.

Under Title I, Part A, schools receive federal funding to help ensure that children, particularly the disadvantaged, have the opportunity to receive a quality education and reach proficiency . Schools use these funds to provide such things as additional instruction, teachers, and professional development. For schools in which at least 40 percent of students come from low income families, Title I, Part A funds can be used for school-wide educational enhancements. However, for schools with less than 40 percent of students from low income families, the educational enhancements must be targeted toward students who are at risk of f ailing to meet the state's academic standards. 12

The amount of Title I, Part A funding that the federal government allocates to school districts depends on several factors, including the number of low income children in each district and the level of per-pupil educational spending in each state. 13 School districts that receive Title I, Part A funding face some restrictions on how they allocate these funds to their schools. For example, districts must rank order all their schools by the percentage of students from low income f amilies. Schools with the highest percentage of low income students receive their funding first. After districts have funded all of their schools with more than 75 percent low income students, the districts can then concentrate their funding on schools with certain grade spans— for example, funding elementary schools first. Districts are required to work their way down their rank-ordered lists until the objectives. In our view, this requirement is a subset of the Title I, Part A requirement ensuring that all limited-English students become proficient. Second, section 9532 of the NCLB Act requires states to allow students who attend persistently dangerous schools or have been the victim of a violent criminal offense while at school to transfer to a safe school. From a cost perspective, it is unclear how this requirement will interact with the Title I, Part A requirement that school districts allow students who attend schools that have repeatedly failed to make "adequate yearly progress" to transfer to a higher-performing school. It is possible that many of the unsafe schools are also low-performing schools.

BACKGROUND 7 States must establish a single "accountability system" for all schools, not just for schools receiving federal funds.

11 There are two requirements of particular interest in portions of NCLB other than Title I, Part A. First, Title III requires state educational agencies to develop annual measurable achievement objectives for limited-English students and to hold school districts accountable for meeting these funding runs out. 14 In the 2002-03 school year, only 42 percent of Minnesota's 2,329 schools received Title I, Part A funding, and the majority of these schools were elementary schools. 15

 

12 No Child Left Behind Act, §§ 1114( a)( 1) and 1115( a) and (b). For purposes of the 40 percent threshold, low income families are primarily those whose children are eligible to receive free and reduced-price meals at school.

13 Ibid., §§ 1124, 1124A, 1125, and 1125A. For purposes of the federal allocation of Title I, Part A funds to school districts, low income families are primarily those with incomes below the federal poverty level. For purposes of the district allocation of Title I, Part A funds to schools, low income families are those with children who are eligible for free and reduced-price meals.

 

NCLB REVENUES Besides imposing significant education accountability requirements on Minnesota, the federal government also provides the state with considerable f inancial assistance under NCLB. In state fiscal year 2004, Minnesota received $231 million through formulas prescribed in NCLB, as shown in Table 1.3. (The state and school districts also receive some discretionary/ non-formula grants under NCLB, but these grants account for a small fraction of the overall NCLB funding. 16 ) From this $231 million allocation, the federal government made available nearly $118 million for school districts'Title I, Part A programs and $114 million for 22 other NCLB programs. Table 1.3 describes the ten largest programs. Table 1.3 also shows that Congress recently decreased the funding that Minnesota is projected to receive for state fiscal year 2005. We discuss this decline in more detail later in this chapter.

While Minnesota receives over $200 million annually in NCLB funding, NCLB funding represents a relatively small proportion of school districts' operating budgets.

Statewide, Minnesota's NCLB funding for state fiscal year 2004 accounted for less than 4 percent of school districts'operating budgets, and the Title I, Part A portion accounted for less than 2 percent. 17 In fact, the state provides more money for the education of disadvantaged students than the federal government provides. Specifically, for state fiscal year 2004, the Minnesota Legislature appropriated $354 million for basic skills instruction, which is substantially more than the $231 million provided by the federal government under NCLB. 18

Minnesota received $231 million in NCLB "formula funding" for the current school year.

14 U. S. Department of Education, Office of Elementary and Secondary Education, Non-Regulatory Guidance: Local Educational Agency Identification and Selection of School Attendance Areas and Schools and Allocation of Title I Funds to Those Areas and Schools (Washington, D. C., undated), http://www.ed.gov/programs/titleiparta/wdag.doc, accessed December 29, 2003.

15 Based on the school listing and Title I status reported in Minnesota Department of Education, Adequate Yearly Progress 2003: AYP Status, Consequence and Goals (A ugust 14, 2003) , http://children.state.mn.us/content/031412.xls, accessed August 27, 2003.

16 According to the Minnesota Department of Education, the department received only two discretionary/non-formula grants under NCLB for state fiscal year 2004—$ 8.0 million from the Public Charter School program and $2.3 million from the Voluntary Public School Choice program.

These two grants are only 4 percent of the state's formula grant allocation. Of the nine school districts that we visited, seven provided us with a listing of all the NCLB funding that they received; none listed a discretionary/non-formula grant. (We did not receive this listing from the Minneapolis or St. Paul school districts.)

17 The data for district operating expenditures are from Minnesota Department of Education, "2002 District Total Expenditures," http://cfl.state.mn.us/content/035532.xls, accessed November 28, 2003.

18 Minnesota Department of Education, "Estimated Revenues, FY 2000 to FY 2005: Laws 2003 First Special Session Chapter 9," http://education.state.mn.us/content/010811.xls, accessed November 13, 2003. The basic skills funding includes compensatory, assurance of mastery, limited English proficiency, and extended time revenue.

BACKGROUND 9 Table 1.3: Major NCLB Programs and Funding Minnesota's Funding FY 2005 NCLB Title FY 2004 (Inflation Adjusted, and Part Program Name Purpose (in Millions) in Millions) a

Title I, Part A Grants to School Districts for Basic Programs Ensure that all children, particularly the disadvantaged, have the opportunity to obtain a high quality education and reach proficiency. $ 117. 7 $104. 4

Title I, Part B, Subpart 1 Reading First Help ensure that every child can read at or above grade level through the implementation of instructional programs, assessments, and professional development.

9.6 8. 2

Title II, Part A Improving Teacher Quality Increase student achievement by elevating teacher and principal quality through recruitment, hiring, and retention strategies. 38. 9 37. 5

Title II, Part D Educational Technology Improve student academic achievement through the use of technology, and assist every student to become technologically literate. 6.1 5. 0

Title III Language Instruction for Limited-English Students Assist school districts in teaching English to limited-English students and in helping these students meet the same academic standards required of all students. 5.3 6. 0

Title IV, Part A, Subpart 1 Safe and Drug-Free Schools and Communities Prevent violence in and around schools; prevent illegal use of alcohol, drugs, and tobacco; and foster safe and drug-free learning environments. 5.9 5. 9

Title IV, Part B 21st Century Community Learning Centers Provide services, during non-school hours or periods, to students and their families for academic enrichment, including tutorial and other services. 5.9 9. 1

Title V, Part A Innovative Programs Assist local education reform efforts that are consistent with and support statewide reform efforts. 6.6 4. 9

Title VI, Part A, Subpart 1 State Assessments Help states develop the assessments required under NCLB. 6.9 6. 9 Title VIII Impact Aid Provide financial assistance to school districts that contain federal property, which is exempt from local property taxes. 12. 0 13. 2

Other Titles and Parts Other NCLB programs that provide formula funding

Carry out other NCLB activities. 16. 5 15. 1

Total NCLB Formula Funding $ 231.2 $216.0 NOTE: Congress appropriated these funds for federal fiscal years 2003 and 2004, but the funds were made available in Minnesota for state fiscal years 2004 and 2005. The 2005 figures are preliminary estimates by the U. S. Department of Education and are subject to change. a These funds have been adjusted for inflation to reflect prices in state fiscal year 2004.

SOURCE: Compiled by the Office of the Legislative Auditor from information contained in (1) U. S. Department of Education, Office of Elementary and Secondary Education, No Child Left Behind: A Desktop Reference 2002( Washington, D. C., 2002); and (2) U. S. Department of Education, "Fiscal Year 2001-2005 State Tables for the U. S. Department of Education," http://www.ed.gov/about/overview/budget/statetables/index.html?src=rt, accessed February 16, 2004.

Federal funding under NCLB plays a greater role in some districts than others. For example, the Pine Point school district in Becker County currently receives $1,093 in Title I, Part A funding per K-12 student, which accounts for roughly 6 percent of the district's operating budget. 19 At the other extreme, the Minnetonka and Wayzata school districts in Hennepin County do not receive any Title I, Part A funding. The variation occurs because NCLB bases each district's allocation on its poverty level.

In support of NCLB's ambitious goals, the federal government increased its Elementary and Secondary Education Act (ESEA) funding under NCLB. However, we found that:

Minnesota's federal funding increase under NCLB has been smaller than that of other states.

Nationwide, the federal government has increased formula allocations for ESEA programs by 49 percent from a pre-NCLB base of $14.8 billion in state fiscal year 2002 to $22.1 billion in state fiscal year 2005. 20 In contrast, during the same period, Minnesota's overall NCLB formula allocation has increased 24 percent, from $174 million to $216 million. 21 With respect to Title I, Part A funding, the federal government has increased the national appropriation 34 percent between state fiscal years 2002 and 2005, while Minnesota's allocation has increased 3 percent, from $102 million to $104 million. (NCLB funds appropriated by Congress for a federal fiscal year are made available to states for the following state fiscal year. For example, the NCLB funds just appropriated by Congress for federal fiscal year 2004 will be made available to Minnesota for state fiscal year 2005, which begins July 1, 2004.) When we asked officials from the Minnesota Department of Education to explain why Minnesota's allocation did not rise as f ast as that of other states, the department reported that the distribution of funds for most education programs is tied to federal poverty measures. Consequently, when Minnesota experienced economic growth and declining poverty in the late 1990s that outpaced the national averages, the state started to receive a smaller share of federal education funds. 22

10 NO CHILD LEFT BEHIND To implement NCLB, the federal gover nment is pr oviding states with increased funding for elementary and secondary education.

19 While the Title I, Part A funding applies to the 2003-04 school year, the enrollment figures are from October 1, 2002, and the operating budget figures are from the 2001-02 school year. Minnesota Department of Education, unpublished table "Minnesota Department of Education: Title

I, 03-04 Entitlements/ Concentration Grants," received November 7, 2003; Minnesota Department of Education, "School and District Fall Population Files," http://cfl.state.mn.us/datactr/fallpops/index.htm, accessed September 29, 2003; and Minnesota Department of Education, "2002 District Total Expenditures," http://clf.state.mn.us/content/035532.xls, accessed November 28, 2003.

20 We adjusted the funding levels for fiscal years 2002 and 2005 for inflation to reflect prices in fiscal year 2004. 21 U. S. Department of Education, "Fiscal Year 2001-2005 State Tables for the U. S. Department of Education," http://www.ed.gov/about/overview/budget/statetables/index. html?src=rt, accessed February 16, 2004.

22 When computing funding allocations, the federal government uses poverty data that are a few years old. Therefore, funding levels lag economic trends.

It is particularly noteworthy that: Minnesota's NCLB funding is projected to decline between state fiscal years 2004 and 2005, while NCLB funding is increasing nationwide.

Table 1.3 shows that Minnesota's NCLB funding is projected to decline from $231 million in state fiscal year 2004 to $216 million in state fiscal year 2005— a 7 percent reduction. This decline primarily reflects a projected reduction in Title I, Part A funds. In contrast, formula grants under NCLB are projected to increase by about 2 percent nationwide between state fiscal years 2004 and 2005, after adjusting for inflation. (The figures for 2005 are based on preliminary estimates by the U. S. Department of Education and are subject to change.)

Besides giving states more ESEA-related funding under NCLB, the federal government has also granted states greater flexibility in the use of these funds. NCLB authorizes states to transfer up to 50 percent of their non-administrative funds from five ESEA programs (Improving Teacher Quality, Educational Technology, Safe and Drug-Free Schools and Communities, 21st Century Community Learning Centers, and Innov ative Programs) to the Title I, Part A program. Alternatively, states can transfer funds among these five programs. School districts have a similar transfer authority, but it extends to only four of the f ive programs. (Districts cannot transfer funds from the 21st Century Community Learning Center program.) 23 As shown earlier in Table 1.3, Minnesota's allocation for these five programs is about $63 million annually. While the Minnesota Department of Education has not taken advantage of this transfer authority, some districts have.

BACKGROUND 11

23 No Child Left Behind Act, §6123.

NCLB's Impact on Minnesota's Education Accountability System SUMMARY Minnesota was implementing a statewide educational accountability system prior to passage of the No Child Left Behind (NCLB) Act. But, as a result of NCLB, Minnesota is implementing additional testing, more measures of student subgroup performance, new sanctions for underperformance, and more ambitious goals. Officials at the Minnesota Department of Education strongly support the act as a necessary means to improving student achievement. Meanwhile, most local education officials view the act as unrealistic, costly, and punitive, although many of them support the general goals of the act. The pervasive level of skepticism among local officials could be a significant obstacle to the continued implementation of the NCLB Act in Minnesota.

The No Child Left Behind (NCLB) Act's accountability provisions are significantly different than those in previous federal education law, and their implementation is one of the main challenges facing Minnesota schools. This chapter addresses the following questions: To what extent are the provisions of the federal NCLB Act consistent with the components of Minnesota's pre-NCLB educational accountability system? To what extent do Minnesota education officials support the goals and approaches outlined in NCLB's accountability provisions?

CONSISTENCY OF NCLB WITH EXISTING MINNESOTA POLICY The Minnesota Constitution says that it is the Legislature's duty to "establish a general and uniform system of public schools" and "secure a thorough and eff icient system of public schools throughout the state." 1 But, for most of Minnesota's history, school districts had considerable autonomy regarding academic standards, curriculum, assessment practices, and performance measurement. A 1996 report said that Minnesota was one of ten states "without a regular statewide system to report on the status, needs and performance of its students or the resources, conditions and practices of its schools." 2 Until the 1996-97 school year, the state's only requirements for high school graduation were completion in grades 9 through 12 of four English credits, three social studies credits, one math credit, and one science credit. 3

1 Minn. Const., art. XIII, sec. 1.

Nevertheless, it is important to note that: In the decade preceding passage of the NCLB Act, the Minnesota Legislature and Minnesota Department of Education took steps toward the establishment of a uniform, statewide educational accountability system.

The Legislature declared its commitment to a "rigorous, results-oriented graduation rule" in 1992, 4 and it directed the State Board of Education to develop a rule that focused on minimum competencies as well as rigorous standards. To determine whether students met minimum competencies, the board adopted basic standards tests in reading, math, and writing. Students must pass these tests to graduate from high school. 5 Students take the reading and math basic standards tests in the 8th grade, and those who do not pass the initial tests have multiple opportunities in subsequent years to retake them. 6 Minnesota started using the reading and math basic standards tests for students entering ninth grade in the 1996-97 school year.

In addition, the Legislature directed the State Board of Education to adopt a "Profile of Learning" based on "high academic standards." 7 The Profile identified content standards for students in grades K-8 and 9-12. Initially, students at public high schools were required to complete 24 high school content standards before graduating; in 2000, the Legislature authorized each school site to determine which content standards were required. 8the 2003 Legislature adopted new content standards in reading and math, and it repealed the Profile of Learning. 9

Minnesota law did not require statewide assessments of students a decade ago, but several (in addition to the basic standards tests described above) have been required in recent years:

Years before NCLB was enacted, the Minnesota Legislature mandated development of rigorous academic standards and several statewide student assessments.

2 University of Minnesota, College of Education and Human Development, Minnesota Educational Accountability Reporting System: Feasibility and Design Study, v . 1 (Minneapolis, December 1996), i.

3 Lisa Larson and Kerry Kinney Fine, State High School Graduation and Colleg e Pr eparation Requir ements Compar ed( St. Paul: House of Representatives Research Department, October 1998), 1.

4Laws of Minnesota (1992), ch. 499, art. 8, sec. 32. 5Minn. Stat. (2002), §120B. 30, subd. 1. The 1998 Legislature abolished the State Board of Education, and the department assumed many of its duties.

6The writing test is first administered to students in 10 th grade. 7Laws of Minnesota (1Sp1995), ch. 3, art. 7, sec. 1; Laws of Minnesota (1996), ch. 412, art. 7, sec. 1.

8Laws of Minnesota (2000), ch. 500, sec. 3. 9Laws of Minnesota (2003), ch. 129, art. 1. The Legislature required the Commissioner of Education to submit proposed academic standards in science and social studies to the Legislature by February 1, 2004 (sec. 3).

The 1997 Minnesota Legislature required the assessment of all 3rd and 5th grade students annually. 10 The Minnesota Department of Education developed the Minnesota Comprehensive Assessments (MCAs) in reading and math for this purpose, and they were first administered statewide to 3rd and 5th grade students in 1998. 11

The 1997 Legislature required statewide assessment of "post-8th grade students." 12 In response, the department developed a reading MCA for 10th grade and a math MCA for 11th grade. These tests are being administered for accountability purposes in 2004 for the first time.

The 2001 Legislature required the annual assessment of 7th grade students, and these MCAs are being administered for accountability purposes in 2004 for the first time. 13 There were no statewide criteria for assessing the performance of schools and school districts during the 1990s. 14 A 1998 state law said that schools failing to meet state performance criteria for two of three consecutive years would have to work with district and state officials to develop a plan to improve student achievement. 15 However, the Minnesota Department of Education did not adopt criteria until 2001. The criteria said that each Title I school was expected to achieve average MCA scores of 1420 in reading and math. 16

Prior to NCLB, the Legislature also adopted requirements for the public reporting of information on school performance. In 1996, the Legislature required the establishment of a "coordinated and comprehensive system of educational accountability and public reporting that promotes higher academic achievement." 17 The Legislature required the Commissioner of Education to report on aggregate student performance "at the school district, regional, or statewide level." 18 At the time this law passed, Minnesota lacked standards for collecting and analyzing student achievement data, and there were insuff icient data to assess state and local changes in performance. 19 In addition, the Legislature created an independent Office of Educational Accountability to help ensure that Minnesota's publicly reported measures of education performance are comprehensive, valid, and reliable. 20

NCLB'S IMPACT ON MINNESOTA'S EDUCATION ACCOUNTABILITY SYSTEM 15 But, until 2001, Minnesota did not have statewide criteria for assessing the performance of individual schools.

10 Laws of Minnesota (1997), ch. 138, sec. 1. Minnesota chose to implement tests in both 3rd and 5th grades, although a test in only one of these grades would have been sufficient to meet the requirements of the 1994 Elementary and Secondary Education Act reauthorization. State law required these assessments to be "highly correlated with the state's graduation standards." 11 The Minnesota Department of Education is the primary state agency overseeing Minnesota's K-12 school system. This agency was called the Minnesota Department of Children, Families and Learning from 1995 to 2003, but throughout this chapter we use the terms "Minnesota Department of Education" and "Commissioner of Education" to refer to the state's main education agency and its top official.

12 Laws of Minnesota (1997), ch. 138, sec. 1. 13 Laws of Minnesota (1Sp2001), ch. 6, art. 2, sec. 4. 14 State law did set standards for judging the performance of individual students on the basic standards tests. (To pass the tests, students are now required to correctly answer 75 percent of the reading and math questions, plus receive 3 of a possible 6 points on the writing test.)

15 Laws of Minnesota (1998), ch. 398, art. 9. sec. 1. 16 This target score was to increase to 1500 by the 2009-10 school year. 17 Laws of Minnesota (1996), ch. 412, art. 7, sec. 2. 18 Ibid. 19 University of Minnesota, Minnesota Educational Accountability Reporting System , 39.

In 2001, the year before NCLB was signed into law, the Legislature enacted several provisions that broadened Minnesota's educational accountability system and enhanced its visibility. For example, state law required the department to measure the adequate yearly progress of all schools, not just Title I schools. In addition to measures of school performance, the Legislature required the department to implement a system for measuring the progress of individual students, "based on highly reliable statewide or district assessments." 21 Also, the Legislature required the department to make school and school district performance data av ailable on a web site. 22

Some of the changes to Minnesota's educational accountability system occurred in response to (or in anticipation of) federal requirements. For example, the 1994 Improving America's Schools Act (the federal law that preceded NCLB) required that each state implement yearly assessments of reading and math at some point during each of the following grade spans: 3 through 5, 6 through 9, and 10 through 12. 23 This act also required states to define "adequate yearly progress" for schools receiving federal Title I funding. 24 While federal requirements played a role in Minnesota's actions, many of the changes also reflected a growing, independent interest in educational accountability by Minnesota's legislative and executive branches.

Table 2.1 summarizes how key accountability requirements of NCLB compare with the practices Minnesota had in place at the time of the law's enactment. NCLB reinforces many elements of Minnesota's emerging accountability system, but:

NCLB establishes more rigorous requirements than Minnesota had adopted previously.

First, NCLB's expectations for student achievement are more ambitious than those in previous federal or state laws. NCLB requires states to develop plans to ensure that all students in all public schools are "proficient" in reading and math by the 2013-14 school year. In contrast, previous federal education law only focused on improving the proficiency of children served by T itle I programs. In addition, before NCLB, the Minnesota Department of Education determined whether schools met state expectations by comparing their averag eMCA proficiency scores with a state-designated threshold score. In contrast, NCLB sets an expectation that each studentwill meet or exceed the state's proficiency threshold by 2013-14, and it requires schools to make "adequate yearly progress" toward this goal. Finally, it is worth noting that NCLB's proficiency expectations will escalate over time. Presently, about two-thirds of the students in a Minnesota school must be proficient for the school to meet NCLB's expectations; this will increase to 100 percent by 2013-14. 25

 

20 Laws of Minnesota (1998), ch. 398, art. 5, sec. 10. 21 Laws of Minnesota (1Sp2001), ch. 6, art. 2, sec. 5. 22 Ibid. 23 Improving America's Schools Act of 1994, Pub. L. No. 103-382. The U. S. Department of Education granted Minnesota a waiver— through January 31, 2004— regarding the time frame for implementing the grades 7, 10, and 11 assessments.

24 Improving America's School Act, §1111( b)( 2).

NCLB'S IMPACT ON MINNESOTA'S EDUCATION ACCOUNTABILITY SYSTEM 17 Table 2.1: Comparison of Key NCLB Accountability Requirements with Minnesota's Pre-NCLB Requirements Comparison with Minnesota's NCLB Requirement Accountability System as of January 2002

Statewide, grade-specific content standards in reading, math, and science. Minnesota had already implemented content standards (Profile of Learning), but it did not have the grade-specific benchmarks required by NCLB.

Reading and math assessments in grades 3, 4, 5, 6, 7, and 8, and once in high school. Minnesota had already implemented reading and math MCAs in grades 3 and 5, and pre-NCLB plans called for MCAs in grades 7 (reading and math), 10 (reading), and 11 (math). Minnesota had no specific plans for assessments in grades 4, 6, and 8.

Science assessments administered once in each of three grade spans (3-5, 6-9, and 10-12). Minnesota had no statewide science assessments, and none were planned.

Assessments of English proficiency in reading, writing, listening, and speaking. Minnesota already had an English proficiency assessment in reading and writing, but it did not have an assessment in listening and speaking. Determinations of "adequate y early progress" (AYP) for each school and school district— based on (1) overall performance and the performance of student subgroups, (2) measures of proficiency, test participation, attendance, and graduation.

Minnesota already required determinations of AYP for schools— but based solely on academic proficiency, and not based on a goal of 100 percent proficiency by the 2013-14 school year. There were no AYP determinations for school districts, and the performance of student subgroups was not considered in AYP determinations.

"Report cards" on school and district performance. State law required a "continuous improvement" web site with data on each school and school district; there was no requirement for "report cards" on school or district performance.

Sanctions for low-performing schools (school choice, supplemental education services, corrective actions, and restructuring).

Minnesota required low-performing schools to develop improvement plans (which NCLB also required), but it had no specific provisions for the sanctions specified by NCLB.

"Highly qualified" teachers in core academic subjects by the 2005-06 school year (see Table 4.6). Minnesota teachers were required to meet state Board of Teaching requirements for licensure, and the state's teacher standards were not subject to federal review.

Title I paraprofessionals meet NCLB-specified qualifications by January 2006 (see Table 4.6). Minnesota previously had less stringent requirements for paraprofessionals (requiring, at most, a high school diploma).

SOURCES: Office of the Legislative Auditor analysis of No Child Left Behind Act and Minnesota statutes, plus interviews with Minnesota Department of Education staff.

NCLB imposes new accountability requirements on Minnesota.

 

Second, although the amount of statewide testing has increased significantly in Minnesota during the past decade, NCLB will require additional testing. NCLB requires that students be assessed against challenging academic content standards in reading and math in grades 3, 4, 5, 6, 7, and 8, plus once in grades 10 through 12. At the time NCLB passed, Minnesota had implemented (or had plans to implement) reading and math assessments in grades 3, 5, and 7, plus a reading assessment in grade 10 and a math assessment in grade 11. Consequently, NCLB will require Minnesota to implement new assessments in grades 4, 6, and 8. 26 (Minnesota's 8th grade basic standards tests are intended to assess minimum competencies rather than progress toward high standards— thus, these tests do not meet NCLB requirements.) In addition, NCLB requires states to implement science assessments at least once during each of the following grade spans: 3-5, 6-9, and 10-12; Minnesota has had no previous, statewide science assessments. Finally, although Minnesota previously assessed English proficiency in reading and writing, NCLB required Minnesota to add statewide assessments of English proficiency in listening and speaking.

Third, NCLB requires more measures of performance than Minnesota's education accountability system previously had. For example, state and federal law previously did not require the state to assess the progress of student subgroups. In contrast, NCLB requires separate determinations of progress for (1) economically disadvantaged students; (2) students from major racial and ethnic groups (including separate determinations for white, black, American Indian, Hispanic, and Asian subgroups); (3) students with disabilities; and (4) students with limited English proficiency . In addition, previous determinations of student progress were based solely on "proficiency ," as measured by reading and math assessments. Under Minnesota's state NCLB plan, however, assessment of progress is now based not only on proficiency , but also on test participation rates (all schools), graduation rates (high schools only), and attendance rates (elementary and middle schools only). Prior to NCLB, Minnesota schools had a small number of ways to fail to make "adequate yearly progress" (AYP). 27 But, because NCLB mandates accountability for student subgroups and requires new measures of performance, there are as many as 37 separate hurdles that each Minnesota school or school district must clear to make AYP, as shown in Table 2.2. Although most schools will not be subject to all 37 performance targets, schools that fail to meet any of these hurdles will not achieve AYP. 28

18NO CHILD LEFT BEHIND Individual schools will be accountable for their overall academic performance and for the performance of certain student subgroups.

25 As described in Chapter 3, Minnesota uses index points to determine whether schools have made "adequate yearly progress." Table 3.3 shows annual changes in proficiency targets, with all targets increasing to 100 index points by 2013-14.

26 Some people contend that, even without NCLB, Minnesota would have implemented assessments in grades 4, 6, and 8— for purposes of complying with state-required measures of individual students' academic progress over time ( Minn. Stat . (2002), §120B. 35, subd. 1 and 3).

27 There were four ways that elementary schools could fail to make AYP— low proficiency on 3rd grade reading, 3rd grade math, 5th grade reading, or 5th grade math.

28 If a subgroup's number of tested students is fewer than a state-designated minimum, then the schools is not held accountable for the performance of that subgroup.

Fourth, NCLB primarily focuses on whether students meet a statewide pr oficiency standard at a given point in time, rather than monitoring the gr owth of individual students over time. Prior to NCLB, the Minnesota Legislature began to explore the concept of "value-added" assessments— that is, methods of evaluating the academic growth of individual students over time. For example, the 2001 Legislature required the department to (1) develop measures of individual student progress, and (2) recommend ways to integrate such measures with the federally-required AYP determinations. 29 But the NCLB Act makes no specific provisions for value-added performance measures in its definition of AYP. Rather, determinations of AYP are based on an absolute measure of performance (the proportion of students who are proficient). 30

Fifth, NCLB specifies stronger consequences for schools "needing improvement." Before NCLB, schools that failed to make AYP for two or more consecutive years were required by federal and state laws to prepare improvement

NCLB'S IMPACT ON MINNESOTA'S EDUCATION ACCOUNTABILITY SYSTEM 19 Table 2.2: Components of "Adequate Yearly Progress" To make "adequate yearly progress" (AYP) under NCLB, students in each school and school district must meet or exceed standards in each of the following applicable categories (marked "X") NCLB Subgroup American Limited-Special Low Criteria for AYP All White Black Indian Asian Hispanic English Education Income Determination Students Students Students Students Students Students Students Students Students a

Reading proficiency XXXXXXXXX Reading participation XXXXXXXXX Math proficiency XXXXXXXXX Math participation XXXXXXXXX Attendance or graduation rate b X

NOTE: For each of the 36 categories related to test proficiency or participation, adequate yearly progress is computed for the school or school district on the basis of test data aggregated across those grades for which tests are given. For measures of proficiency, AYP determinations are not made for subgroups with fewer than 20 students— or 40 students, in the case of special education. For measures of participation, AYP determinations are not made for subgroups with fewer than 40 students.

a Low income students are defined as those from families eligible for free or reduced-price meals. b Elementary and middle schools are held accountable for their attendance rates, while high schools are held accountable for their graduation rates.

SOURCE: Minnesota Department of Education, Minnesota Consolidated State Application Accountability Workbook( Roseville, MN: January 31, 2003).

29 Laws of Minnesota (1Sp2001), ch. 6, art. 2, sec. 5. 30 NCLB has a "safe harbor" provision for schools that do not meet the absolute standard. Specifically, schools can make AYP if they reduce their proportion of non-proficient students by 10 percent from one year to the next (and if they make progress on the requirements for attendance or plans. 31 There were no additional sanctions for persistent failure to make AYP. In contrast, NCLB specifies a series of increasingly serious consequences that apply to underperforming schools or school districts, as shown in Table 2.3. For example, school districts with underperforming schools may have to give parents the option of sending their children to other schools or tutoring services outside the school day. If schools still do not make AYP, NCLB requires implementation of "corrective action" or "restructuring."

graduation, whichever is applicable). In addition, the NCLB Act allows states to use performance measures (such as value-added measures) besides the measures specified in the act— however, these additional measures cannot reduce the number of schools categorized as needing improvement for having failed to make AYP for at least two consecutive years.

 

Sixth, NCLB sets standards for some school staff that exceed previous r equirements. NCLB requires that all teachers of core academic subjects be "highly qualified" by the end of the 2005-06 school year, and it requires that all paraprofessionals working in Title I, Part A programs meet NCLB qualifications by January 2006. As we discuss in Chapter 4, the NCLB-related teacher requirements will likely have limited impact on Minnesota's teachers because teachers who are teaching in their field of licensure will be deemed "highly qualified," according to Minnesota Department of Education staff. 32 In contrast, previous versions of the federal Elementary and Secondary Education Act prescribed minimal requirements for paraprofessionals, so the new NCLB provisions could have considerable impact on school districts'hiring practices for paraprofessionals.

 

Table 2.3: NCLB Requirements for Schools Failing to Make Adequate Yearly Progress Number of Years That the School Has Failed to Make AYP Requirement/ Sanction 1 2 3 4 5 6 Improvement plan X X X XX School choice X X X XX Supplemental services XXXX Corrective action X Restructuring plan X Implement restructuring X

IMPROVEMENT PLAN: Must develop (or revise) a school improvement plan. SCHOOL CHOICE: Must offer school choice options, if possible, to parents of all children in the school failing to make AYP. (Districts are not required to provide school choice if there are no other schools in the district or if all the other schools have failed to make AYP for at least two years.)

SUPPLEMENTAL SERVICES: Must offer supplemental educational services (such as tutoring) outside the school day to eligible children. CORRECTIVE ACTION: The school district must take at least one of the following actions: (1) replace staff who are relevant to the school's low performance, (2) implement a new curriculum, (3) significantly decrease management authority at the school level, (4) appoint an outside expert to advise the school, (5) extend the school's academic year or lengthen its school day, or (6) change the internal organizational structure of the school.

RESTRUCTURING: In the fifth year of failing to make AYP, the school district must prepare a restructuring plan and arrange to implement it. NCLB outlines various restructuring options, including: (1) reopen the school as a charter school, (2) replace staff who are relevant to the school's low performance, (3) contract with another entity (such as a private management company) to operate the school, (4) turn the operation of the school over to the state department of education, or (5) enter into other major restructuring arrangements. If the school fails to make AYP for a sixth year, the district must implement the plan.

SOURCE: No Child Left Behind Act, §1116.

Low-performing schools will face increasingly serious consequences.

31 The exception was schools that were "making progress," according to Minnesota's previous AYP definition. These schools' average achievement scores were below the statewide target score, but their scores showed significant growth from one year to the next.

 

PERCEPTIONS OF EDUCATION OFFICIALS Minnesota Department of Education We interviewed various department officials regarding NCLB, including top department administrators and staff who work on issues related to curriculum, assessment, services for disadvantaged students, and licensure and training of school staff. Department staff said that it has been challenging to implement the law in a relatively short time frame— for example, developing definitions of "adequate yearly progress" and "highly qualified" teachers that comply with NCLB. In addition, the department has faced these challenges at a time when its staff ing levels have been reduced due to state budget shortfalls. Nevertheless, we found that:

Minnesota Department of Education officials strongly support the goals and methods of the NCLB Act.

As noted in Chapter 1, department officials have expressed particular support for NCLB's requirements for performance reporting by student subgroup. They noted that Minnesota's strong overall performance on standardized assessments has masked lagging performance by some subgroups.

In addition, department officials support the ambitious goals of NCLB. As the commissioner stated last year:

Yes, NCLB sets challenging achievement goals that Minnesota and other states will struggle with over the next 12 years. But if we are going to start setting public policy goals in education based on the assumption that we expect some kids to fail, I think we have a responsibility to tell parents and the public which kids we are planning on leaving out of the picture. I'm not prepared to do that, and I don' t believe the vast majority of educators are either. 33

NCLB'S IMPACT ON MINNESOTA'S EDUCATION ACCOUNTABILITY SYSTEM 21 Officials at the Minnesota Department of Education believe that the NCLB law is fundamentally sound.

32 As we discuss in Chapter 4, however, the impact of NCLB requirements on special education, English as a Second Language, and alternative learning center teachers is still being assessed by the department.

33 Commissioner Cheri Pierson Yecke, "Closing the Achievement Gap: Why Minnesota's Accountability Plan Must Address Achievement Disparities Among Our Students," Presentation at Citizens League Forum, Minneapolis, May 22, 2003,

http://education.state.mn.us/stellent/groups/public/documents/translatedcontent/pub_ 041625.jsp, accessed February 2, 2004.

Department staff told us that some schools, by virtue of not making AYP, have been forced to look more closely at their curricula, teaching strategies, staff ing, and other educational approaches. They said that, in underperforming schools, NCLB strengthened the conviction of staff to improve, as demonstrated by schools that made AYP in 2003 after not making it previously. Department officials said that the NCLB law may need "fine tuning" but is fundamentally sound.

Local Education Officials In November 2003, we sent surveys regarding NCLB to all school district superintendents and charter school directors in Minnesota. This section focuses on the responses of superintendents, although our web site presents separate summaries of the superintendent and charter school director responses. 34 We received responses from more than 90 percent of those surveyed. 35 We found that:

A majority of Minnesota school district superintendents agree with some of the central components of the NCLB Act.

Like the federal education law that preceded it, NCLB requires public reporting on students'academic achievement, and it requires underperforming schools to identify ways to raise student achievement. Our survey indicated that:

99 percent of superintendents favor measuring the academic performance of their students.

85 percent of superintendents favor publicly reporting on the academic performance of their students, in aggregate.

94 percent of superintendents favor developing plans to improve the performance of student subgroups that are under-achieving.

Many school officials credit the NCLB Act for having worthy goals and for focusing attention on the low achievement levels of some student subgroups. For example, we heard the following comments:

The most positive aspect is the fact that, while we have always examined what we are doing, NCLB has forced us to dig deeper and scrutinize what we do to find our strengths and weaknesses in the core areas of math, reading, and language arts.

[Having data that is broken down by subgroups] has awakened me and my district to the fact that we have some students who are not performing as well as others. That awareness will allow us to make appropriate decisions to assist remediation.

22 NO CHILD LEFT BEHIND Many local school officials credit NCLB for having worthy goals but are concerned about some aspects of the law.

34 See the following web site: http://www.auditor.leg.state.mn.us/Ped/2004/pe0404.htm. 35 We received responses from 326 of the state's 342 school districts (95 percent), and we received responses from 79 of 92 charter schools (86 percent).

NCLB has caused educators to re-examine the way decisions are made and placed data at the threshold of every decision. The act has, as no other, clarified the needed vigilance for setting high academic standards for all students, and forces educators to truly understand assessment and analysis.

While many school officials told us that NCLB's goal of ensuring success for all children is admirable,

Most Minnesota superintendents have significant concerns about NCLB. They regard it as unrealistic, costly, and punitive.

Table 2.4 shows that school officials have significant concerns about applying uniform standards of academic proficiency to all subgroups of students. On the one hand, 72 percent of superintendents said that they fav or holding all racial/ ethnic subgroups to the same standards. Similarly, 73 percent of superintendents think that students from lower income families (that is, those eligible for free or reduced-price lunches) should be held to the same standards as other students. But, contrary to the requirements of the NCLB Act, only 5 percent of superintendents said that special education students should be held to the same academic standards as other students, and only 17 percent of superintendents said that limited-English students should be held to the same standards as others.

Our survey gave school officials the opportunity to express comments— positive or negative— about NCLB. The most common comment we heard was concern about the requirement for 100 percent of students to achieve proficiency , and the following is a sampling of these comments:

[The] 100 percent achievement standard is absurd, especially for special ed students. have we forgotten there is a bell curve of abilities? Keep the accountability component, but get real with expectations. Where is parent and student accountability in all this? Is there any awareness at all of the amount of dysfunction, mental illness, poverty, etc. that affects student performance?

NCLB'S IMPACT ON MINNESOTA'S EDUCATION ACCOUNTABILITY SYSTEM 23

Table 2.4: Superintendents' Perceptions About Using Uniform Standards to Measure Students'Academic Proficiency Survey question: It is appropriate for schools and school districts to hold Percentage Who Responded: to the state's uniform Neither Agree No standard of academic proficiency. Agree Disagree Nor Disagree Response Total

All racial/ethnic student subgroups 72% 13% 14% 1% 100% Free and reduced-price lunch students 73 12 14 1 100 Special education students 5 79 15 1 100 Limited-English students 17 62 20 1 100

SOURCE: Office of the Legislative Auditor survey of school district superintendents, November-December 2003 ( N= 326).

Most superintendents do not fav or holding special education and limited-English students to the same academic standards as other students, contrary to what NCLB requires.

The requirements of NCLB have been extremely challenging for our district because we serve a high population of… English language learners. Research suggests that it will take these children 7 to 11 years to become academically proficient in English.

Students in special education and limited-English programs are now the most vulnerable students in our schools. Just because they can't meet unrealistic federally determined standards, they now take the brunt of criticism for a school failing [to make] AYP. [These students] are working hard and want to succeed but have either innate diff iculties or haven't learned the language enough to pass a test.

There is no doubt that the goals of NCLB are laudable. There is no doubt that schools need to be accountable for creating the conditions for student success. Where the law is fatally flawed is in the premise that simply ratcheting up expectations will magically lead to students achieving at grade level.

Overall, although NCLB says that all students shall be proficient by 2013-14, only 17 percent of superintendents said that it is "likely" or "very likely" that their districts could accomplish this. This may be one reason why just 33 percent of superintendents said that it is appropriate for national policy to have a goal for all children to be academically proficient by 2013-14. In Chapter 3, we present simulations which suggest that it will indeed become increasingly diff icult for Minnesota school districts to comply with NCLB's proficiency requirements.

In addition, we found that a large majority of superintendents did not think that schools should face NCLB-prescribed consequences for persistent failure to make "adequate yearly progress." Under NCLB, schools that fail to make AYP for two consecutive years must offer parents in these schools the option to transfer to schools that have not failed to make AYP for two years (unless there are no such options within the school district). If schools continue failing to make AYP in subsequent consecutive years, their school districts must offer supplemental education services or consider "corrective actions" (see Table 2.3 earlier). But, as shown in Table 2.5, most superintendents oppose such consequences. For example, even though most superintendents believe that all racial/ ethnic subgroups should be measured against uniform proficiency standards, 74 percent of superintendents said that schools should not face NCLB-prescribed consequences for persistent failure by at least one racial or ethnic subgroup to make AYP.

24 NO CHILD LEFT BEHIND Relatively few superintendents said that their school districts are likely to meet NCLB's goals for student achievement.

As indicated in the comments below, some superintendents objected to sanctions because they thought that NCLB's measure of "adequate yearly progress" is inadequate or misleading. Some others thought that sanctions were not the best strategy for fostering school improvement:

Corrective legislation is needed to prevent a revolt on the part of our professionals and our parents. Our public will not stand for labels of failing schools when only a limited number of the students are not performing to proficiency . The law needs to better distinguish those areas where we are having challenges, keep goals high but not unreasonable to achieve for those subgroups, invest more resources into those areas, and leave alone those schools or subgroups that are performing well.

NCLB criteria for "adequate yearly progress" has misidentified a large number of schools [in our district] that are making strong gains across our [district's] multiple-measure accountability system. Schools that are making strong longitudinal gains should not be labeled as "failing."

If research drives this law, then those who promulgated it should know that punishment is the least likely way to get improvement. Yet the only form of motivation for teachers and schools [in NCLB] is the threat of loss of revenue, prestige, and the school itself.

[The] current AYP point system does not differentiate between extremely low performing schools with many low performing subgroups and schools with just one low performing subgroup.

NCLB'S IMPACT ON MINNESOTA'S EDUCATION ACCOUNTABILITY SYSTEM 25 Table 2.5: Superintendents' Perceptions About NCLB-Prescribed Consequences for Schools Failing to Make Adequate Yearly Progress Survey question: Schools should face consequences such as mandatory school choice, supplemental services, corrective actions, or restructuring Percentage Who Responded: if there is persistent failure to make Neither Agree No AYP (as presently defined) by: Agree Disagree Nor Disagree Response Total

At least one racial/ ethnic student subgroup 13% 74% 12% 1% 100% Free and reduced-price lunch students 22 60 16 1 100 Special education students 3 88 8 1 100 Limited-English students 5 85 9 1 100

SOURCE: Office of the Legislative Auditor survey of school district superintendents, November-December 2003 ( N= 326).

Most superintendents disagree with the sanctions required by NCLB for low student performance.

It seems that NCLB punishes schools not making AYP rather than providing assistance and support. To me this is like a teacher telling the students who are not passing that they will receive less attention from him/ her while the students receiving passing grades will now receive more help and attention.

Another broad area of concern that emerged in our survey of school officials is the perception that NCLB is an unfunded federal mandate. Less than 3 percent of Minnesota superintendents said that the new federal revenues received by their districts under the NCLB Act will be sufficient to cover the cost of new spending required by the act. We discuss NCLB fiscal impacts in detail in Chapter 4. However, below is a sampling of school officials' general comments regarding NCLB-related fiscal concerns:

I applaud the concept of universal proficiency . However, this will not be accomplished "on the cheap." Our school district is now reducing spending to find money to allocate for services to special populations. With flat or decreasing state aid, the likelihood is that we will continue to rob Peter to pay Paul… We are asking public schools to accomplish the impossible: raise all students to levels of proficiency , but meet all the state and federal mandates with the same resources.

The biggest challenge with NCLB is the need to reallocate existing resources (staff and operating) to meet requirements. This means that we don' t provide some of the other programs that have been in place. For example, exploratory curriculum, specialists at the elementary level, and vocational/ fine arts offerings at the secondary level will likely be reduced as we focus on NCLB needs.

Finding, hiring, retaining highly qualified teachers will be diff icult, if not impossible, in many districts. The same will hold true for paraprofessionals. These concerns could impact our budgets significantly. Most likely we would have to cut other positions and increase class sizes. In the end, would there be a net gain in what students learn?

Many school officials also expressed concerns about the student assessments mandated by NCLB, as shown in Table 2.6. About half of the superintendents said that it was necessary, in their opinion, to test students annually to have an effective accountability system, but many of the remainder expressed concern that annual testing resulted in a loss of too much instructional time. In addition, Table 2.6 shows that superintendents did not offer a particularly strong endorsement of the tests that Minnesota uses to comply with NCLB, the Minnesota Comprehensive Assessments (MCAs). NCLB requires that states' assessments be adequate for purposes of both (1) accountability (that is, measuring aggregate student achievement against state standards), and (2) diagnosing the needs of individual students. 36 However, only 36 percent of superintendents said that the MCAs provide a sound basis for ev aluating the academic performance of schools and school districts, and only 35 percent said that the MCAs help teachers understand the specific academic needs of individual students. Many Minnesota school districts presently administer assessments in addition to the MCAs, partly because they believe that these assessments provide richer, more timely information for the benefit of teachers and administrators.

26 NO CHILD LEFT BEHIND Local officials generally view NCLB as an unfunded federal mandate.

Finally, many school district officials expressed concerns about the overall educational impact of NCLB. Only 7 percent of superintendents said that the educational benefits of NCLB will outweigh any adverse impacts the act will have on their respective districts. Sixty-eight percent said that the benefits of NCLB will not outweigh its disadvantages, and the rest of the superintendents were undecided. Some of the concerns about NCLB's educational impacts included the following:

I take extreme exception to the concerted effort to take the art of teaching and turn it into a science… Teaching is the art of reaching as many human beings as possible by employing every means at your disposal to motivate your students. NCLB is a cookie cutter approach that will turn off more students than it can ever hope to help.

I believe that NCLB will lead people to segregate their [minority students] if schools with high minority populations are not meeting AYP.

NCLB'S IMPACT ON MINNESOTA'S EDUCATION ACCOUNTABILITY SYSTEM 27 Table 2.6: Superintendents' Perceptions About NCLB-Prescribed Assessments Percentage Who Responded: Neither Agree Survey questions: Agree Disagree Nor Disagree Total Annual reading and math assessments, which are required by NCLB for grades 3-8, are a necessary component of an effective accountability system. 49% 39% 12% 100%

The Minnesota Comprehensive Assessments (MCAs) provide a sound basis for evaluating the academic performance of school districts and schools. 36 43 21 100

The MCAs help teachers understand the specific academic needs of individual students. 35 50 15 100 SOURCE: Office of the Legislative Auditor survey of school district superintendents, November-December 2003 ( N= 326).

Many superintendents question whether Minnesota's statewide achievement tests ar e useful for ev aluating school performance or identifying the needs of individual students.

36 No Child Left Behind Act, §1111( b)( 3)( C)( vii) and (xii).

There is growing concern for those [schools] which house [specialized services for students with disabilities]. We are beginning to see a reluctance to have these programs within buildings, due to the fear that they could cause the entire building to be labeled as a failing school.

The NCLB Act, with the added subjects and grade levels being tested, will require schools to "teach to the test" at the expense of many other needed instructional topics.

Overall, our survey indicates that many school district superintendents and charter school directors have concerns about the fiscal and non-fiscal impacts of the NCLB Act. In our view, the skepticism of local officials is so pervasive that it could be very challenging for the Minnesota Department of Education to build the confidence of local officials who are responsible for implementing many aspects of NCLB.

Compliance with " Adequate Yearly Progress" (AYP) Requirements SUMMARY Few Minnesota schools (8 percent) did not make "adequate yearly progress" (AYP) in the 2002-03 school year, but a much larger proportion of schools will likely fail to make AYP in future years. Last year's evaluations of school progress were based only on student assessments in grades 3 and 5, and small numbers of students in tested grades excused student subgroups in many schools from NCLB's accountability provisions. In future years, AYP determinations will include assessments at more grade levels, and schools' proficiency levels will be measured against higher benchmarks. Based on a range of assumptions regarding student achievement levels, we estimated that between 80 and 100 percent of Minnesota elementary schools will fail to make AYP for proficiency by 2014. In addition, we estimated that between 35 and 76 percent of elementary schools receiving federal Title I funds will face "restructuring," as required by NCLB. Thus, Minnesota faces significant disruption in its education system unless there is change in the federal NCLB law, adoption by the state of a less stringent definition of "proficiency," or dramatic improvement in student achievement levels. In addition, as measures of student progress are used to hold educators accountable, the Legislature and Minnesota Department of Education should ensure that these measures are valid, reliable, appropriate, and properly reported.

As discussed in the previous chapter, the No Child Left Behind (NCLB) Act significantly expands the accountability provisions of the Elementary and Secondary Education Act (ESEA) from its previous version. Measures of AYP are the cornerstone of this accountability system. The manner in which AYP is implemented will significantly affect the number of Minnesota schools that will be labeled as "failing" and, consequently, the costs that the state and school districts will face in complying with the act.

In this chapter, we address the following questions: How does Minnesota define "adequate yearly progress" (AYP) for the purpose of meeting NCLB requirements?

To what extent did Minnesota schools and school districts make AYP in the 2002-03 school year?

How will the number of schools failing to make AYP likely change in the next several years? How did the Minnesota Department of Education use AYP data for the school report cards that it developed in 2003? What issues should policy makers consider as the state continues its efforts to hold school districts, schools, and other education service providers accountable for the progress of their students?

DETERMINING AYP IN MINNESOTA NCLB requires states to create a single, statewide accountability system that determines whether all public schools and school districts are making "adequate yearly progress" (AYP) toward achieving 100 percent proficiency by the 2013-14 school year. 1 In Chapter 2, we briefly described how AYP is determined. In this chapter, we will describe the process in more detail.

As shown in the box at the right, the Minnesota Department of Education holds schools and school districts accountable using several measures. First, based on standardized assessments in reading and math, the department determines the proficiency of students in nine subgroups. (One of the nine subgroups includes all the students in the school.) In addition, NCLB requires that at least 95 percent of students in each subgroup take the reading and math assessments to ensure that all students in a school or district are included in the proficiency measurements. NCLB also requires states to determine AYP based on the graduation rates of high schools and at least one other academic indicator for elementary and middle schools. 2 Minnesota has chosen attendance as its other indicator. The state-adopted target rates are 80 percent for graduation and 90 percent for attendance. If schools do not meet these targets in a given year, or improve the previous year's rate, they fail to achieve AYP. Table 2.2 in Chapter 2 lists all the AYP criteria and each of the subgroups for which schools and districts are accountable. In total, there are 37 possible ways for an individual school or district to fail to make AYP.

Determining AYP for proficiency is a four-step process. While the process we describe below inv olves schools, the same process is carried out for school districts. First, states determine if students are proficient by administering statewide assessments in reading and math. 3 As Table 3.1 shows, Minnesota has divided its assessment scores into five levels of achievement. Students scoring at level IIB or above are deemed proficient, while students scoring at level IIA are deemed partially proficient.

30 NO CHILD LEFT BEHIND Measures Used to Determine Adequate Yearly Progress Target Proficiency rates 100 percent (by 2013-14)* Participation rates 95 percent Attendance rates 90 percent Graduation rates 80 percent \

*As shown later in Table 3.3, the target rates increase steadily over the next decade, up to 100 percent.

Measures of student achievement are a cornerstone of NCLB's accountability requirements.

1 No Child Left Behind Act, §1111( b)( 2)( A) and (F). 2 Ibid., §1111( b)( 2)( C)( vii). States may add indicators to the AYP determination beyond those prescribed by NCLB, but the law says that any additional indicators may not reduce the number of schools that would otherwise be subject to sanctions under the act ( Ibid., §1111( b)( 2)( D)).

Second, states determine proficiency rates for each school. Under NCLB, proficient students (achievement level IIB or above) receive one index point, while partially proficient students (achievement level IIA) receive one-half of a point. As shown in Table 3.2, proficiency rates are calculated by aggregating the index points and then dividing by the total number of tested students. A separate proficiency rate is calculated for each of a school's subgroups— except for subgroups with fewer than 20 tested students. (For the special education subgroup, the threshold is 40 students.)

Third, states compare the computed proficiency rates with the state's proficiency targets. Under NCLB, these targets rise from a baseline level up to 100 percent for the 2013-14 school year. As shown in Table 3.3, Minnesota's proficiency targets remain at the baseline level for two years and then increase in equal increments up to 100 percent by 2014. To establish a target rate for a subgroup within a school, states compute a weighted average of the grade-specific targets based on the number of students in each grade for that subgroup. If the proficiency rate for any of a school's subgroups is below its respective proficiency target, the entire school fails to achieve AYP. 4

COMPLIANCE WITH "ADEQUATE YEARLY PROGRESS" (AYP) REQUIREMENTS 31 Table 3.1: Achievement Levels on the Minnesota Comprehensive Assessments (MCAs) Index Points Achievement Level Type of Achievement Earned I "Below basic" level. Student has significant gaps in the knowledge and skills necessary for grade-level work. 0.0

IIA "Basic" level. Student has partial knowledge and skills required for grade-level work. 0.5 IIB "Proficient" level. Student is working successfully on grade-level material. 1.0 III "Advanced" level. Student is working on material above grade level. 1.0 IV Beyond "advanced" level. Student is achieving well beyond grade-level performance. 1.0

NOTE: A Level IIB, or "proficient," score on the 3rd and 5th grade reading and math assessments is a scale score from 1420 to 1499. The score required for the other levels of achievement varies for the four assessments. SOURCE: Minnesota Department of Education, Consolidated State Application Accountability Workbook( Roseville, MN, January 31, 2003), 8.

NCLB requires states to determine student "proficiency" in reading and math.

3 States are required by NCLB to implement science assessments, but these assessments will not be used to make AYP determinations. 34 C. F .R. §200.20 (2003).

Fourth, if a subgroup has a proficiency rate below its respective target, the school can still make AYP by achieving "safe harbor" for the subgroup. 5 The safe harbor process is explained in Table 3.4.

In 2003, with respect to proficiency , 95 Minnesota elementary schools were not held accountable for any of the subgroups, even the "all students" subgroup. These schools had fewer than 20 tested students in the 3rd and 5th grades.

32 NO CHILD LEFT BEHIND Table 3.2: Calculation of a Minnesota School's Proficiency Rate The table below shows the calculation of a proficiency rate in reading for a hypothetical school. The shaded area shows achievement levels that are considered "proficient" or better.

Lower Higher Achievement Levels

Level I Level IIA Level IIB Level III Level IV Total Number of 3rd Grade Students 20 20 50 10 0 100 Number of 5th Grade Students 30 10 30 30 10 110

Total 3rd and 5th Grade Students 50 30 80 40 10 210 Proficiency Points Per Student 0 0.5 1 1 1 Total Proficiency Points a 015 80 4010145

Proficiency rate = 69 percent a "Total proficiency points" is computed by multiplying the total number of students scoring at a level by the corresponding proficiency points per student.

SOURCE: Office of the Legislative Auditor analysis of information in Minnesota Department of Education, Consolidated State Application Accountability Workbook( Roseville, MN, January 31, 2003), 8; and Minnesota Department of Education, Sample Calculations, unpublished spreadsheets.

145 proficiency points 210 total students

Table 3.3: Minnesota's Annual Proficiency Targets 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 Grade 3 Reading 62. 8 62. 8 66. 5 70. 2 74. 0 77. 7 81. 4 85. 1 88. 8 92. 6 96. 3 100 Grade 5 Reading 69. 9 69. 9 72. 9 75. 9 78. 9 81. 9 85. 0 88. 0 91. 0 94. 0 97. 0 100 Grade 3 Math 66. 2 66. 2 69. 6 73. 0 76. 3 79. 7 83. 1 86. 5 89. 9 93. 2 96. 6 100 Grade 5 Math 65. 4 65. 4 68. 9 72. 3 75. 8 79. 2 82. 7 86. 2 89. 6 93. 1 96. 5 100

NOTE: The proficiency target represents the proficiency rate that a school or school district (and each of its subgroups) must attain to achieve AYP in a given year.

SOURCE: Minnesota Department of Education, Consolidated State Application Accountability Workbook( Roseville, MN, January 31, 2003), 26.

NCLB requires 100 percent of students to meet their state's academic standards by the 2013-14 school y ear.

4Minnesota uses a confidence interval to increase the statistical validity and reliability of AYP determinations. A subgroup's proficiency rate must exceed the lower bound of the confidence interval, rather than the actual proficiency target. The state applies a confidence interval ranging from 95 to 99 percent, with higher confidence intervals used for schools accountable for more subgroups.

5No Child Left Behind Act, §1111( b)( 2)( I)( i).

However, officials with the Minnesota Department of Education told us that they are creating an alternative AYP determination process for these schools. Once this process is established, all elementary schools will be held accountable for proficiency .

AYP DETERMINATIONS FOR THE 2002-03 SCHOOL YEAR

Minnesota schools were first subject to NCLB's AYP determinations in the 2002-03 school year. In that year, the Minnesota Department of Education determined AYP for elementary schools based on (1) academic proficiency measured by the Minnesota Comprehensive Assessments (MCAs) in grades 3 and 5, (2) participation in the MCAs, and (3) attendance rates. The MCAs in grades 7, 10, and 11 were still in development and were not used to determine AYP. As a result, AYP determinations for middle schools were based only on attendance rates, and AYP determinations for high schools were based only on graduation rates. As districts administer the additional assessments required by NCLB, the state will include these assessments in the determination of AYP.

We examined the results of the first AYP determinations under NCLB and found that:

Eight percent of Minnesota schools failed to make adequate yearly progress for the 2002-2003 school year.

As shown in Table 3.5, the vast majority of Minnesota schools made AYP for the 2002-03 school year. The low percentage of failing schools partly reflects the fact that middle and high schools were not subject to a proficiency measurement in that year. Schools most frequently failed to make AYP because they failed to meet proficiency requirements. Less often, schools failed to make AYP because of low participation, attendance, or graduation rates. The percentage of failing schools is likely to increase in the 2003-04 school year because AYP determination will include measures of proficiency and participation for middle schools (based on the reading and math MCAs for the 7 th grade) and for high schools (based on the reading MCA for the 10 th grade and the math MCA for the 11 th grade).

COMPLIANCE WITH "ADEQUATE YEARLY PROGRESS" (AYP) REQUIREMENTS 33 Table 3.4: NCLB's "Safe Harbor" Provision A school that fails to achieve the proficiency target (for a subgroup or all students) on the state assessment in reading or math will meet safe harbor and be treated as making "adequate yearly progress" if: In the failing subgroup, the percentage of students failing to achieve the proficiency target is reduced by 10 percent compared with the previous year's percentage, and

The failing subgroup makes progress on the attendance or graduation indicator, whichever is applicable.

SOURCE: No Child Left Behind Act, §1111( b)( 2)( I)( i).

In 2002-03, statewide student assessments in the 3rd and 5th grades were a central component of Minnesota's accountability measures.

The relatively few subgroups for which elementary schools were accountable also contributed to the low AYP failure rate. Schools are accountable for a subgroup's proficiency only if the number of students in the subgroup exceeds a state-adopted minimum. We found that:

In 2002-03, a majority of individual schools were not accountable for the proficiency of most NCLB-designated subgroups— because these subgroup populations did not exceed the minimum threshold for accountability.

Table 3.6 shows for each subgroup the percentage of elementary schools that were accountable for the proficiency of that subgroup in the 2002-03 school year. In a majority of elementary schools, the number of white students and free and reduced-price lunch students in the tested grades was suff iciently large to hold the school accountable. In contrast, less than 15 percent of elementary schools were accountable for the proficiency of each of the other subgroups, ranging from 14 percent of schools accountable for their black subgroup to 2 percent of schools accountable for their special education subgroup.

The small percentage of elementary schools accountable for their special education subgroup was the result of the Minnesota Department of Education's policy regarding the minimum number of students required for accountability. The department requires a minimum of 40 students for the special education subgroup, while it requires a minimum of 20 for all other subgroups. If the department had required a minimum of only 20 students for the special education subgroup, the number of schools subject to an AYP determination would have increased from 21 schools (2 percent of schools) to 309 schools (30 percent of schools) in the 2002-03 school year.

34 NO CHILD LEFT BEHIND Table 3.5: Schools Failing to Make Adequate Yearly Progress, by Various Measures, 2002-03 School Year

Type of Schools Number of Schools Percentage of Schools AYP Measure Subject to AYP Measure Subject to AYP Measure Failing to Make AYP Proficiency Elementary schools 920 7.0% Participation Elementary schools 847 2.7 Attendance Elementary and middle schools 1,528 4.0

Graduation High schools 296 0.7 One or more All types of the above measures 1,836 7.8

NOTE: Percentage of schools failing is based on the number of schools accountable for that measure. For proficiency and participation, schools are treated as failing if they failed for any subgroup. SOURCE: Office of the Legislative Auditor analysis of Minnesota Department of Education data.

Eight percent of Minnesota schools did not meet NCLB's performance targets in 2002-03.

We also looked at the performance of subgroups with suff icient numbers of students to hold schools accountable for proficiency . As shown in Table 3.7, we found that:

Student subgroups differed considerably in the extent to which they made AYP on proficiency measures in 2002-03.

Differences in subgroup performance reflected the "achievement gap" for minority and low income students discussed in Chapter 1. All 844 schools with 20 or more white 3rd and 5th graders made AYP for proficiency for the white subgroup. In addition, all the schools made AYP for the white subgroup by meeting the proficiency target rather than by using the safe harbor provision. In contrast, at least 10 percent of accountable schools failed to make AYP in proficiency for the following subgroups: American Indian students (math), Hispanic students (reading), limited-English students (math and reading), and black students (math and reading). In addition, many schools made AYP in proficiency for these subgroups only because the subgroup met NCLB's safe harbor provision. For these subgroups, up to 37 percent of schools made AYP only through the safe harbor provision.

COMPLIANCE WITH "ADEQUATE YEARLY PROGRESS" (AYP) REQUIREMENTS 35 Table 3.6: Schools Held Accountable on Proficiency Measures, by Subgroup, 2002-03 School Year Number of Percentage of Schools Accountable Schools Accountable for the Proficiency for the Proficiency Subgroup Subject of the Subgroup of the Subgroup

All Students Math 920 a 90. 6% All Students Reading 920 a 90. 6 White Math 844 83. 2 White Reading 844 83. 2 Low Income b Math 651 64. 1 Low Income b Reading 653 64. 3 Black Math 140 13. 8 Black Reading 140 13. 8 Limited-English Math 136 13. 4 Limited-English Reading 137 13. 5 Asian Math 96 9.5 Asian Reading 96 9.5 Hispanic Math 58 5.7 Hispanic Reading 58 5.7 American Indian Math 21 2.1 American Indian Reading 21 2.1 Special Education Math 21 2.1 Special Education Reading 21 2.1

NOTE: AYP determinations for proficiency were based on the 3rd and 5th grade assessments in reading and math in the 2002-03 school year. Subgroups were accountable for an AYP determination in proficiency only when they included the minimum number of required students (40 for special education, and 20 for other subgroups). The percentage of schools accountable for the proficiency of a subgroup is the number of schools with the minimum number of students required divided by 1,015, the total number of schools with at least one tested student in the 3rd or 5th grade.

a Only 920 schools were accountable for the "all students" group because 95 schools had fewer than 20 tested students. The Minnesota Department of Education has not yet determined how it will measure AYP for these schools this year.

b Low income students are defined as those from families eligible for free or reduced-price meals. SOURCE: Office of the Legislative Auditor analysis of Minnesota Department of Education data.

In 2002-03, most schools were not held accountable for the performance of special education, limited-English, and minority students because too few students were tested.

 

Minnesota's percentage of schools failing to make AYP in 2002-03 (8 percent) was lower than the percentage in most other states, according to analyses reported in some education publications. For example, other states such as Alaska, Delaware, and Florida reported over half of their schools failing. 6 However,

Due to inter-state differences in proficiency standards, testing practices, and "adequate yearly progress" calculations, there is no meaningful way to use AYP data to make multi-state comparisons of education performance.

36NO CHILD LEFT BEHIND Table 3.7: Schools Failing to Make Adequate Yearly Progress in Proficiency, by Subgroup, 2002-03 School Year Percentage of Accountable Number of Schools Schools With Proficiency

Accountable for Rates Below the State Target the Proficiency And Not Making And Making Subgroup Subject of the Subgroup "Safe Harbor" "Safe Harbor" Total All Students Math 920 a 1.4% 1.6% 3.0% All Students Reading 920 a 1.0 3. 3 4.2

White Math 844 0.0 0. 0 0.0 White Reading 844 0.0 0. 0 0.0 Low Income b Math 651 3.5 4. 6 8.1 Low Income b Reading 653 2.8 7. 5 10. 3 Black Math 140 13. 6 13. 6 27. 1 Black Reading 140 10. 7 17. 1 27. 9 Limited-English Math 136 11. 8 17. 6 29. 4 Limited-English Reading 137 15. 3 37. 2 52. 6 Asian Math 96 1.0 4. 2 5.2 Asian Reading 96 2.1 13. 5 15. 6 Hispanic Math 58 8.6 22. 4 31. 0 Hispanic Reading 58 15. 5 25. 9 41. 4 American Indian Math 21 19. 0 0.0 19. 0 American Indian Reading 21 9.5 9. 5 19. 0 Special Education Math 21 0.0 19. 0 19. 0 Special Education Reading 21 0.0 33. 3 33. 3

NOTE: AYP determinations for proficiency were based on the 3rd and 5th grade assessments in reading and math in the 2002-03 school year. Subgroups were accountable for an AYP determination in proficiency only when they included the minimum number of required students (40 for special education, and 20 for other subgroups).

a A total of 1,015 schools had at least one tested student in the 3rd or 5th grade, but only 920 schools were accountable for the "all students" group because 95 schools had fewer than 20 tested students. The Minnesota Department of Education has not yet determined how it will measure AYP for these schools this year. b Low income students are defined as those from families eligible for free or reduced-price meals.

SOURCE: Office of the Legislative Auditor analysis of Minnesota Department of Education data.

6" A Look at School AYP Failure by State," eSchool News, October 1, 2003, http://www.eschoolnews.com/news/showStory.cfm?ArticleID= 4677& ref= wo, accessed November 14, 2003; "Adequate Yearly Progress: A State Snapshot," Education Week, September 3, 2003, http://www.edweek.org/ew/ewstory.cfm?slug= 01ayp_ s1.h23, accessed December 10, 2003.

First, states have adopted various definitions of "proficiency ," as allowed by NCLB. 7 Since AYP is based on the percentage of students scoring at the "proficient" level on state assessments, states with more rigorous performance standards may have a higher percentage of schools failing to make AYP.

Second, the number of NCLB-required tests that a state administers can also influence the percentage of schools failing to make AYP in a state. States are at v arying stages of developing the tests required by NCLB. In the 2002-03 school year, Minnesota's AYP determinations were based on only 4 of the 17 tests required by NCLB— specifically, the reading and math assessments in grades 3 and 5. In contrast, as of March 2003, 5 states reported they had all of the 17 tests required by NCLB in place, and 4 more reported they needed only 1 to 3 additional tests. 8 States administering a greater number of tests may have a higher percentage of schools failing to make AYP because (1) schools at more levels (elementary, middle, and high schools) are subject to an AYP determination for proficiency , and (2) schools are likely to be accountable for the performance of a greater number of subgroups as the number of students tested increases. 9

Third, variations in the way states calculate AYP also limit cross-state comparisons of AYP performance. For example, schools are not accountable for a subgroup's performance if the number of students in the subgroup is below the minimum set by a state, and NCLB gives states latitude to determine these thresholds. For proficiency determinations, the minimum number of students required in a subgroup ranges from 5 to 50 students among states. States also differ in the statistical "confidence interval" they apply in calculating AYP for schools or subgroups. 10

AYP DETERMINATIONS FOR FUTURE YEARS

The impact of NCLB on Minnesota's education system will depend largely on the ability of schools to meet the law's ambitious goals. As schools face the challenge of meeting steadily increasing proficiency targets, NCLB proponents hope that the law will lead to significant improvements in school performance. Meanwhile, skeptics question whether such improvements are realistic.

COMPLIANCE WITH "ADEQUATE YEARLY PROGRESS" (AYP) REQUIREMENTS 37 States vary widely in the number of NCLB-required tests they have in place.

7Ford Fessenden, "How to Measure Student Proficiency?" New York Times, December 31, 2003, sec. B, p. 8; and Northwest Evaluation Association, "The State of State Standards: Research Investigating Proficiency Levels in Fourteen States,"( 2003), http://www.young-roehr.com/nwea/

NWEA_ National_ Report. pdf, accessed January 9, 2004. 8U. S. General Accounting Office, T itle I: Characteristics of Tests Will Influence Expenses; Information Sharing May Help States Realize Efficiencies, (Washington, D. C., May 2003), 13, 33. The GAO surveyed the 50 states and Puerto Rico and the District of Columbia regarding the number of tests they would need to develop or augment with additional questions to comply with NCLB's assessment requirements.

9Asassessments are added in more grades, the number of students in subgroups (summed across grades) is more likely to exceed the minimum required for AYP determination.

10 Council of Chief State School Officers, Statewide Educational Accountability Under NCLB , (Washington, D. C., July 2003), 20-23. A confidence interval is a range of values that is likely to contain the actual value. Use of a confidence interval is intended to increase the validity and reliability of AYP determinations.

This section presents our findings from a simulation analysis that used Minnesota assessment data and various assumptions to help estimate NCLB's future impacts. In addition, we review national test data and research literature to help consider the likelihood of large-scale improvements in student achievement.

Simulation Methods We contracted with the Office of Educational Accountability (OEA) at the University of Minnesota to estimate the number of Minnesota schools that will potentially fail to make AYP over the next decade. 11 Based on discussions with staff from OEA and the Minnesota Department of Education, we provided OEA with the key assumptions that were used in this analysis.

We limited our analysis to elementary schools because most Title I programs are at elementary schools. 12 NCLB requires public reporting on student performance at all public schools, but only the schools that receive federal Title I, Part A funding are subject to NCLB sanctions for not making AYP. AYP determinations for elementary schools are based on students'academic proficiency levels, test participation rates, and school attendance, but our simulation analysis focused solely on proficiency . We assumed that all schools would meet the attendance and test participation levels prescribed by NCLB, although a small number of schools did not meet these standards during the 2002-03 school year. 13

Minnesota elementary schools presently administer the Minnesota Comprehensive Assessments (MCAs) in the 3rd and 5th grades. As required by NCLB, schools will begin administering MCAs for AYP purposes in the 4th and 6th grades in 2006. We used actual 2003 MCA scores from all Minnesota public schools as a starting point for simulating future levels of student performance. Our analysis used the 3rd grade assessment scores as a proxy for the 4th grade scores, and we used the 5th grade scores as a proxy for the 6th grade scores. Specifically, we assumed that the percentages of a school's 4th grade students scoring in each MCA achievement level in future years would mirror the percentage of that school's 3rd grade students who will score in these achievement levels. Likewise, we assumed that the distribution of scores for 6th grade students in future years would mirror the distribution of scores among 5th grade students. 14

For each school (and each subgroup in that school), we determined proficiency rates separately for reading and math. We then compared each school's reading We contracted with the Office of Educational Accountability (OEA) to estimate the number of Minnesota elementary schools that might fail to make AYP in the future.

11 The 1997 Legislature created OEA to provide Minnesota's executive and legislative branches with independent information regarding school accountability ( Minn. Stat. (2002), §120B. 31, subd. 3).

12 We defined elementary schools as schools with at least one grade in grades three through five. Another reason for focusing on elementary schools is that these schools have administered statewide tests for more years than middle and high schools.

13 We did not include participation and attendance in our simulation because the NCLB standards for these measures remain the same over time, in contrast to the rising standards for proficiency. To the extent that some schools will fail to achieve the participation and attendance standards in the and math proficiency rates with its reading and math proficiency targets. 15 We estimated whether schools would make AYP, fail to make AYP, make "safe harbor," or have too few students to make an AYP determination.

future, our analyses understate the percentage of schools that will fail to make AYP. 14 The MCAs are based on content standards that were repealed by the Minnesota Legislature in 2003. The MCAs will be revised to reflect new, grade-specific standards and benchmarks, and it is unclear what impact these revisions might have on the distribution of test scores.

 

The simulation allowed us to examine how some important changes required by NCLB will affect the ability of schools to make AYP in the future. First, as required by NCLB, the proficiency targets used to evaluate school performance will increase steadily from 2005 through 2014. (NCLB expects all students to be proficient by 2014.) Second, when schools begin administering MCAs at the 4th and 6th grades, elementary schools'AYP determinations will be based on test scores from a larger number of students. Thus, more schools (and more subgroups within those schools) will exceed the minimum number of students needed to make AYP determinations (20 students, or 40 in the case of the special education subgroup).

We made our estimates using three different assumptions regarding future assessment scores, as shown in Table 3.8. 16 These ranged from a "no improvement" scenario— assuming no growth over time in overall student achievement levels— to a "high improvement" scenario— based on the relatively large average annual improvements in proficiency that occurred statewide over the 2000-03 period. OEA staff expressed skepticism about the "high improvement" assumption, noting that (1) an unusually large increase in a single year (from 2002 to 2003) caused the relatively large three-year average increase, and (2) previous research suggests that it would be unusual to sustain large improvements in achievement over a long period of time. Thus, our third scenario assumed a more modest increase in achievement (" modest improvement"), consistent with Minnesota's statewide experience from 2000 to 2002. In the simulations where we assumed that future test scores would increase, we assumed that students new to Minnesota (from other countries or states) would experience score increases identical to those of other Minnesota students— arguably, an optimistic assumption.

COMPLIANCE WITH "ADEQUATE YEARLY PROGRESS" (AYP) REQUIREMENTS 39 Table 3.8: Scenarios Used to Simulate Schools' Future AYP Status "No improvement" scenario: Assumed that student proficiency rates will remain the same in future years as in 2003. "Moderate improvement" scenario: Assumed that the proficiency rate of each student subgroup will increase annually by the average statewide 2000-02 rate of increase among Minnesota students (0.57 points per year). "High improvement" scenario: Assumed that the proficiency rate of each student sub-group will increase annually by the average statewide 2000-03 rate of increase among Minnesota students (2.54 points per year).

SOURCE: Office of the Legislative Auditor.

OEA estimated schools' ability to comply with future NCLB expectations, based on three different assumptions about students' achievement levels.

15 We assumed that the state's 4th and 6th grade targets (which have not yet been set) were the same as its 3rd and 5th grade targets, respectively.

16 The analysis used an approach known as "sampling with replacement"— keeping the number of students in each school and grade constant each year, but generating from the 2003 data for that school and grade a new sample of tested students for each subsequent year. Within a school and grade, this approach can result in random year-to-year fluctuations in the number of students tested, the number of students in each subgroup, and a school's proficiency rate.

 

In our analysis, we assumed that the demographic characteristics of students within schools would remain the same in coming years, although there have been statewide increases in recent years in the percentages of limited-English and special education students, as well as students eligible for free and reduced-price meals. We also assumed that students categorized as having "limited English proficiency" in 2003 would remain in this subgroup in the subsequent years' analyses— even though their achievement levels may, in fact, rise to a point where they would be considered proficient in English. We also assumed that each school's percentage of students taking "alternate assessments" (due to disabilities) would remain about the same from year to year, and we assumed that the students taking the alternate assessments would reach proficiency at the same rate as the students taking regular assessments. 17 Finally, our data did not incorporate the results of 2003 school district AYP appeals, so it slightly overstates the number of schools that failed to make AYP in 2003 for proficiency reasons.

Simulation Analysis According to the simulations we received from the Office of Educational Accountability, Minnesota's schools will have a very diff icult time meeting the state's increasing proficiency targets. 18 As shown in Figure 3.1,

According to the simulations, between 80 and 100 percent of Minnesota's elementary schools will fail to make adequate yearly progress by 2014. Assuming no improvement in the state's academic proficiency of students over time, the percentage of failing schools would rise from 7.7 percent in 2003 to 99.9 percent in 2014. There would be a similar pattern under the "modest improvement" scenario, with the AYP failure rate increasing from 7.7 percent in 2003 to 98.9 percent in 2014. However, during the intervening years, the percentage of schools failing to make AYP under the "modest improvement" scenario would be somewhat lower than under the "no improvement" scenario. Fewer schools would fail to make AYP under the "high improvement" scenario, but even this scenario shows the school failure rate increasing to 82.3 percent in 2014.

Several patterns in Figure 3.1 merit further explanation. First, all three scenarios show that more schools would fail to make AYP in 2004 than in 2003 even though the state's proficiency targets do not increase during this period. The higher f ailure rates occur because fewer schools would achieve "safe harbor" in 2004 than they did in 2003. There was a large increase in Minnesota test scores in 2003, and this resulted in a relatively large number of schools achieving safe harbor in that year. Even under the "high improvement" scenario, an increase in test scores of this size is not assumed to occur again.

40 NO CHILD LEFT BEHIND Minnesota schools will have an increasingly difficult time complying with NCLB's student achievement targets.

17 The proficiency of severely-disabled students is assessed using "alternate assessments," completed by each student's individualized education plan team. These assessments are based on a checklist of items, not the state's academic standards. With our "sampling with replacement" method of analysis, there may be random fluctuations from year to year in a school's percentage of students taking the alternative assessments.

18 Ernest C. Davenport, Jr., Mark Davison, and Yi-Chen Wu, Adequate Yearly Progress Simulation: Final Report (Minneapolis: Office of Educational Accountability, University of Minnesota, January 26, 2004).

Second, under all three scenarios, the AYP failure rate increases substantially between 2005 and 2006. Starting in 2006, Minnesota will determine AYP for elementary schools based on test scores in grades 3, 4, 5, and 6, rather than just grades 3 and 5. Assuming that proficiency rates do not vary substantially between grades within a school, testing at more grade levels should not affect a school's overall proficiency rate. Nevertheless, the addition of these assessments will increase the number of subgroups for which some schools will be held accountable. As discussed above, a school is held accountable for a subgroup's performance if the school has 20 or more tested students in the subgroup. 19 Historically, the academic performance of the subgroups for which more schools will become accountable has been lower than that of the overall student population.

Third, under the "high improvement" scenario, the failure rate increases sharply between 2013 and 2014. In this case, a lot of schools would be able to keep their proficiency rates above the state's proficiency targets until they are subject to the most stringent target— that is, the expectation for all students to be proficient in 2014.

COMPLIANCE WITH "ADEQUATE YEARLY PROGRESS" (AYP) REQUIREMENTS 41 Figure 3.1: Percentage of Elementary Schools Failing to Make AYP in Proficiency

NOTE: The percentage applies to schools that have 20 or more tested students. SOURCE: Office of the Legislative Auditor analysis of the Office of Educational Accountability's simulation of school performance.

0 10 20 30 40 50 60 70 80 90 100

2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 Year

No Improvement Scenario Modest Improvement Scenario

High Improvement Scenario

It is likely that most schools will not meet NCLB's goals by 2014.

19 The threshold is 40 students for the special education subgroup.

 

We also compared the AYP failure rates of different types of schools. As shown in Table 3.9, we found that:

Under all three scenarios, 100 percent of elementary schools in the Minneapolis and St. Paul school districts would fail to make AYP in 2014. Under the "high improvement" scenario, outstate elementary schools from small districts would, on average, have lower AYP failure rates than other Minnesota elementary schools.

42 NO CHILD LEFT BEHIND

Table 3.9: AYP Status of Minnesota Elementary Schools, 2008 and 2014 No Improvement Scenario Modest Improvement Scenario High Improvement Scenario Percentage Percentage of Percentage Percentage Percentage Percentage of of Schools Schools Failing of Schools of Schools of Schools Schools Failing Failing AYP AYP for Five Failing AYP Failing AYP for Failing AYP AYP for Five in Proficiency or More Years in Proficiency Five or More Years in Proficiency or More Years 2008

State 59. 5% 12. 9% 53. 0% 11. 8% 34. 1% 7.8% Type of District a Minneapolis/ St. Paul 95. 8 53. 8 94. 1 50. 4 81. 5 31. 1 Twin Cities Suburbs 63. 4 7.5 56. 6 6. 1 33. 3 3.9

Outstate Districts (> 2,000 enrollment) 59. 6 9.8 52. 3 8. 8 30. 6 6.2 Outstate Districts (< 2,000 enrollment) 40. 8 2.5 32. 8 2. 5 17. 2 1.9

Title I Status of School Title I 60. 8 15. 0 53. 6 13. 9 35. 5 9. 4 Not Title I 55. 0 5. 7 50. 7 4.7 29. 4 2. 4

2014 State 99. 9 73. 6 98. 9 63. 2 82. 3 33. 2

Location of District a Minneapolis/ St. Paul 100. 0 97. 5 100. 0 96. 7 100. 0 86. 7

Twin Cities Suburbs 100. 0 77. 3 99. 6 65. 5 90. 6 32. 4 Outstate Districts (> 2,000 enrollment) 100. 0 72. 7 99. 5 63. 4 82. 0 29. 4 Outstate Districts (< 2,000 enrollment) 99. 7 61. 3 97. 8 47. 0 67. 9 13. 7 Title I Status of School Title I 99. 9 75. 7 98. 9 65. 2 81. 5 35. 3

Not Title I 100. 0 66. 5 99. 1 56. 1 84. 9 25. 9

NOTE: This table excludes 63 schools in 2008 and 61 schools in 2014 that are expected to have fewer than 20 tested students for both reading and math. In contrast, in its analysis, the Office of Educational Accountability included these schools in its failure rate calculations. The AYP estimates in this table are based on proficiency only. a The percentages broken out by district type are based only on schools in the state's independent and special school districts.

SOURCE: Office of the Legislative Auditor analysis of the Office of Educational Accountability's simulation of school performance.

Under the "no improvement" and "modest improvement" scenarios, nearly all Minnesota schools would fail to make AYP by 2014. With "high improvement" in test scores, most schools would still fail to make AYP— but the failure rate would differ by the location of the district. Under this scenario, outstate schools from small districts would have the lowest failure rate (68 percent). 20 In general, outstate schools from small districts are held accountable for the racial/ ethnic minority, limited-English, and special education subgroups less often than the other types of schools, as shown in Table 3.10. Consequently, outstate schools from small districts have fewer ways to fail AYP than other schools. Furthermore, it will take only one non-proficient student for a school to fail to make AYP in 2014. Although it will be challenging for all schools to make AYP in 2014, large schools will have a more diff icult time achieving universal proficiency than small schools— simply because having more students will present more opportunities for a school to have one non-proficient student.

Through the simulations, we also found that: Under the three scenarios, between 35 and 76 percent of Minnesota's Title I elementary schools will start a restructuring process by the 2014-15 school year. NCLB requires that Title I schools plan for "restructuring" after five consecutive years of failing to make AYP. If schools fail to make AYP for a sixth year, they have to implement the restructuring plan. Restructuring could inv olve such things as reconstituting the school as a charter school or contracting for the school's management. Table 3.9 shows that, assuming no improvement in student proficiency , 76 percent of Title I schools would fail to make AYP for at least f ive consecutive years by 2014. Even under the "high improvement" scenario, 35 percent of Title I schools would need to start the restructuring process.

COMPLIANCE WITH "ADEQUATE YEARLY PROGRESS" (AYP) REQUIREMENTS 43 Table 3.10: Percentage of Elementary Schools Accountable for the Proficiency of Each Subgroup by District Type, 2014 Minneapolis/ Twin Cities St. Paul Suburban Outstate Districts Outstate Districts Subgroup Districts Districts (> 2,000 Enrollment) (< 2,000 Enrollment) All Students 92% 94% 98% 94% White 64 94 97 93

Black 85 32 8 0 Asian 55 28 7 1 Hispanic 36 13 14 7 American Indian 8 0 8 7 Low Income a 91 77 89 88 Limited-English 72 28 18 4 Special Education 27 42 34 15

NOTE: This table applies to reading proficiency. The percentages are based only on schools in the state's independent and special school districts. a Low income students are defined as those from families eligible for free or reduced-price meals. SOURCE: Office of the Legislative Auditor analysis of the Office of Educational Accountability's simulation of school performance.

Many schools will probably face NCLB's strictest consequences within the next decade.

20 In fact, the only school that achieves AYP in 2014 under the no improvement scenario is a K-3 outstate school from a small district. Last year, this school had just over 20 3rd -grade students and was not accountable for any subgroup other than white students.

 

Under the "modest" improvement scenario, schools would generally fail to achieve AYP in 2014 because of the performance of all their subgroups, not just one or two of them. As Table 3.11 shows, almost all schools held accountable for a subgroup in 2014 would fail to clear the AYP proficiency hurdle for that subgroup. The white and Asian subgroups are the only ones with a failure rate below 90 percent. In contrast, as we showed in Table 3.7, most schools that were held accountable for a subgroup in 2003 cleared the AYP proficiency hurdle for that subgroup. Some subgroups, such as limited-English students, had a difficult time meeting the proficiency targets in 2003 but were able to make AYP through the "safe harbor" provision.

Table 3.11: Proficiency Failure by Subgroup, 2014 -Modest Improvement Scenario Number of Schools Percentage of Accountable Schools With Accountable for Proficiency Rates Below the State Target the Proficiency And Not Making And Making Subgroup Subject of the Subgroup "Safe Harbor" "Safe Harbor" Total

All Students Math 944 a 96.9% 0.0% 96.9% All Students Reading 946 a 96.5 0.0 96.5 White Math 882 85.6 0.0 85.6 White Reading 886 84.4 0.0 84.4 Low Income b Math 842 95.6 0.0 95.6 Low Income b Reading 844 95.3 0.0 95.3 Special Education Math 266 99.6 0.0 99.6 Special Education Reading 279 99.3 0.0 99.3 Limited-English Math 239 99.2 0.0 99.2 Limited-English Reading 234 99.6 0.0 99.6 Black Math 231 99.1 0.0 99.1 Black Reading 232 98.7 0.0 98.7 Asian Math 176 84.7 0.0 84.7 Asian Reading 173 86.7 0.0 86.7 Hispanic Math 128 99.2 0.0 99.2 Hispanic Reading 137 97.8 0.0 97.8 American Indian Math 49 98.0 0.0 98.0 American Indian Reading 51 94.1 0.0 94.1

a There were 1,009 elementary schools in the simulation model. Thus, 65 schools were not held accountable in math, and 63 schools were not held accountable in reading because they had fewer than 20 tested students overall.

b Low income students are defined as those from families eligible for free or reduced-price meals. SOURCE: Office of the Legislative Auditor analysis of the Office of Educational Accountability's simulation of school performance.

By 2014, it will be difficult for student subgr oups to meet NCLB's performance targets.

Large-Scale Educational Impr ovements To help us determine whether it is realistic to achieve NCLB's goal of having all children proficient by 2013-14, we reviewed trends in the National Assessment of Educational Progress (NAEP), an assessment that is given to a nationally representative sample of students. In addition, we examined education literature regarding education reforms and factors that affect achievement levels.

Nationally, and in Minnesota, there have been improvements in recent years in students'mathematics test scores, while reading scores have been more stable. For example, the national average NAEP math score for 4th grade students grew from 224 to 235 between 1996 and 2003, and the percentage of 4th -graders with math scores at or above the "basic" level increased from 63 to 77 percent during this period. Meanwhile, the national average NAEP reading scores for 4th grade students grew from 215 to 218 between 1998 and 2003, and the percentage of 4th -graders performing at or above the "basic" level increased from 60 to 63 percent. 21

A recent analysis of NAEP achievement data concluded that: . . . Significant gains are occurring in math scores across most states, with sizable gains in some states. The source of these gains cannot be traced to resource changes, and the most likely explanation would suggest that ongoing structural reform within public education might be responsible. This reform suggests that well-designed standards linked to assessments and some forms of accountability may change the incentives and productivity within public schools and even introduce competition among public schools. Thus, these results certainly challenge the tradi-tional view of public education as "unreformable." 22

In addition, evaluation studies have documented positive impacts from certain education-related reforms and interventions. For example, interventions known as "comprehensive school reforms" have demonstrated significant, sizable impacts on student achievement in some early studies, although a recent summary noted that "there are clear limitations on the overall quantity and quality of studies supporting [these] achievement effects." 23 Other analyses have examined the characteristics of selected schools or school districts that have succeeded in increasing student achievement. 24

COMPLIANCE WITH "ADEQUATE YEARLY PROGRESS" (AYP) REQUIREMENTS 45 Nationally, test scores have been improving in math, but reading scores have remained relatively unchanged.

21 National Center for Education Statistics, "National Assessment of Educational Progress: The Nation's Report Card," http://nces.ed.gov/nationsreportcard/, accessed December 9, 2003. Data on 1996 reading achievement were not available.

22 David Grissmer, Ann Flanagan, Jennifer Kawata, and Stephanie Williamson, Improving Student Achievement: What State NAEP Test Scores Tell Us (Santa Monica, CA: RAND, 2000), 103-4.

23 Geoffrey D. Borman, Gina M. Hewes, Laura T. Overman, and Shelly Brown, Comprehensive School Reform and Student Achievement: A Meta-Analysis (Baltimore: Center for Research on the Education of Students Placed At Risk, November 2002). Comprehensive school reforms (CSRs) take a variety of forms. While all CSRs focus on implementing programs supported by scientific research, CSRs use various models and curricula— some of which have been studied more rigorously than others.

 

Such studies suggest that school changes can contribute to significant improvements in student achievement, even among disadvantaged students. Nevertheless,

It is far from certain that Minnesota schools could produce the large-scale improvements that would be needed to meet NCLB's goals by 2013-14. The recent increases in NAEP scores have been encouraging but not dramatic. Researchers Robert Linn, Eva Baker, and Damian Betebenner noted that the average yearly increase in "proficient" 4th grade students in states that administered the NAEP tests over the past decade was "relatively modest"— usually less than 1 percentage point. Given these small statewide improvements, they concluded that it would be diff icult for all individual schools to consistently meet NCLB's targets in both math and reading. 25 Likewise, education researchers have noted that, on a large scale, improvements in student test scores have usually occurred in small increments rather than in large leaps:

. . . For large groups of students, it is an attainable goal to have small but sustained yearly decreases in the percentages of stu-dents in the lowest performance levels and corresponding in-creases in the higher levels . . . . Substantial annual growth on broad measures of achievement for large groups of stu-dents— particularly growth that is sustained over time— can oc-cur, but it is clearly unusual. 26

In addition, the federal Title I program was developed decades ago to help disadvantaged students improve their achievement levels, yet the results of these services have often been modest. As a 2000 summary of Title I studies concluded,

46 NO CHILD LEFT BEHIND Previous research shows that it is unusual for students to sustain large improvements in achievement over a long period of time.

24 Richard Tappan, Characteristics of Highly Impr o ved Sc hools: A Case Study of Selected Sc hoolsinEconomicallyDisadvantaged Districts (Dover, NH: National Center for the Improvement of Educational Assessment, February 2003), http://www.nciea.org/publications/HighlyImpSchools_ Tappan03. pdf, accessed January 2, 2004. This analysis reviewed seven schools with sustained, significant achievement gains. Also, see Jason Snipes, Fred Doolittle, and Corinne Herlihy, Foundations for Success: Case Studies of How Urban School Systems Improve Student Achievement( New York: Manpower Demonstration Research Corporation, September 2002). This analysis examined four large urban school districts.

25 Robert L. Linn, Eva L. Baker, and Damian W. Betebenner, "Accountability Systems: Implications of Requirements of the No Child Left Behind Act of 2001," Educational Researcher 31, n. 6 (August/ September 2002), 3-16. The NAEP definition of "proficient" is likely more stringent than the definition of "proficient" that Minnesota has adopted for NCLB purposes. But these researchers concluded that NCLB's proficiency goals would be challenging to meet even with standards less stringent than the NAEP definition of proficiency. Elsewhere, Linn says that NCLB's student achievement goals are "quite unrealistic, so much so, that they are apt to do more to demoralize educators than to inspire them" (Linn, Accountability: Responsibility and Reasonable Expectations (Los Angeles: Center for the Study of Evaluation, July 2003), 21).

26 Richard D. Schwarz, Wendy M. Yen, and William D. Schafer, "The Challenge and Attainability of Goals for Adequate Yearly Progress," Educational Measurement: Issues and Pr actice , 20, n. 3 (2001), 32.

The evidence from Title I evaluations indicates that the program has not fulfilled its original expectation: to close the achieve -ment gap between at-risk students and their more advantaged peers. The results do suggest, though, that without the program, children served over the last 30 years would have fallen further behind academically. 27

Furthermore, even in cases where significant increases in student achievement have occurred, it is not always clear what specifically contributed to the increases. In general, few education studies have used the most rigorous, conclusive research approaches (specifically, random assignment of participants to the program under study or to a "control" group). Consequently, researchers Thomas Cook and Monique Payne recently concluded: "Much has been spent to ev aluate educational innovations, but not much has been learned about what works and can be used to improve schools and student performance." 28the U. S. Department of Education established a "What Works Clearinghouse" in 2002 to help school officials identify effective approaches, but no publications have yet been produced for this clearinghouse. Thus, even in cases where schools have increased student achievement significantly, it is still a challenge to (1) identify the factors that made these interventions successful, and (2) replicate these interventions successfully in a variety of school environments.

There is debate not only about which school interventions are most effective, but also about whether additional resources would be required to significantly increase student achievement levels. Researcher Eric Hanushek has observed that, despite significant growth in educational expenditures in the U. S. and other countries in recent years, there is little evidence to suggest that this has resulted in significant changes in educational outcomes. 29 Hanushek says there are instances where small classes or additional resources have had an impact, but there is "no good description of when and where these situations occur." 30 Meanwhile, RAND's education researchers contend that additional resources can have substantial impact on student achievement— particularly when targeted to disadvantaged students in the early grades. 31 Over the years, courts and legislatures in many states have considered what constitutes an "adequate" level of education expenditures, using varying approaches to do so. 32 But even if a consensus had emerged from these cases, it would still be unclear whether funding levels deemed "adequate" in past years would be suff icient to achieve the new standards set by NCLB— notably, assuring that all students achieve proficiency by 2013-14.

COMPLIANCE WITH "ADEQUATE YEARLY PROGRESS" (AYP) REQUIREMENTS 47 There is continuing debate about the role that education funding plays in student performance.

27 Geoffrey D. Borman, "Title I: The Evolving Research Base," Journal of Education for Students Placed at Risk, 5 (2000), 27-45.

28thomas D. Cook and Monique R. Payne, "Objecting to the Objections to Using Random Assignment in Educational Research,"Evidence Matters: Randomized T rials in Educational Researc h, ed. Frederick Mosteller and Robert Boruch (Washington, D. C.: Brookings Institution Press, 2002), 150.

29 Eric A. Hanushek, The F ailure of Input-Based Schooling P olicies (Cambridge, MA: National Bureau of Economic Research, July 2002), http://edpro.stanford.edu/eah/papers/input.pdf, accessed January 9, 2004.

30 Ibid., 30. 31 Grissmer and others, Improving Student Achievement: WhatStateNAEP Test Scores T ellUs , xviii-xxxi and 100-103.

32 The methods used in various states to determine "adequate" funding levels have included (1) judgments by panels of experts, (2) statistical techniques that identify key cost drivers, and (3) evaluations that determine the level of expenditures by "successful" schools.

 

Perhaps NCLB— because it is more far-reaching than previous education laws— will create powerful new incentives for schools to improve performance. Still, achieving a goal of 100 percent proficiency by 2013-14 would be an unprecedented educational accomplishment, and many educators are very skeptical. They question how it would be possible to achieve proficiency in a short period of time among non-English-speaking children who immigrate to the U. S. in the years immediately preceding 2013-14. Also, as noted in Chapter 2, they question whether students with cognitive disabilities can ever become proficient, and they question whether effective school interventions can always overcome non-school influences, such as unsupportive home environments.

If NCLB does not result in unprecedented increases in student achievement levels, there would be only two ways for most Minnesota schools to comply with NCLB requirements. The first would be through a change in the federal NCLB law— for example, if Congress modified the requirement for all students to achieve proficiency by 2014. Second, Minnesota could adopt a definition of "proficiency" that is easier for schools to meet. However, Minnesota would probably have to adopt a substantially less rigorous definition of proficiency to significantly alter the overall level of school compliance with NCLB requirements. We offer no recommendations for changes in the "proficiency" definition, and our simulations assumed no such changes.

OTHER AYP AND ACCOUNTABILITY ISSUES

NCLB aims to strengthen educational accountability. It requires public reporting on student academic progress— in part, to assess the performance of school districts, schools, and supplemental services providers. The following sections discuss several additional issues about this still-developing accountability system, highlighting issues that may need the attention of the Minnesota Department of Education or the Legislature.

Use of AYP Measures on State Report Cards The 2003 Legislature passed legislation requiring the Minnesota Department of Education to develop report cards on the performance of each Minnesota public school. 33 The law specified that the department must determine performance levels using objective criteria— including academic performance, school safety, and staff characteristics. (The federal NCLB law also required state and local agencies to prepare report cards, but the school report cards unveiled by the

48 NO CHILD LEFT BEHIND The Minnesota Department of Education published report cards on school performance for the first time in 2003.

33 Laws of Minnesota (2003), ch. 129, art. 1, sec. 9.

Minnesota Department of Education in August 2003 were not developed to comply with NCLB. 34 )

When the school report cards were first posted on the department's web site, the Governor called them "comprehensive" and encouraged the public to "check the Internet to see how their child's school is performing." 35 Each school's report card showed separate ratings for its academic performance in math and reading. The ratings ranged from one to five stars, with one being the lowest. However,

The Minnesota Department of Education's use of AYP determinations to rate schools' academic performance on state report cards in 2003 was sometimes misleading. As explained on the report cards, a school's AYP status was the only factor used to identify the lowest performing schools in reading and math. Schools that did not make AYP in the two most recent years received one star, while schools that did not make AYP in just the most recent year received two stars. 36 But, if any subgroup of students failed to make AYP in eithermath or reading, the school automatically received a low rating on its report card in bothmath and reading.

For example, Franklin Elementary School in Rochester failed to make AYP in 2002-03. The school's reading proficiency and math proficiency scores exceeded the state's targeted level in 2002-03— overall, and for each of the subgroups that had a suff icient number of students to warrant an AYP determination. However, the state determined that only 59 of the school's 63 free or reduced-price lunch students (93.7 percent) took the reading assessment— so the school did not meet the 95 percent test participation requirement of NCLB. Thus, although the school met AYP on the state's measures of math proficiency and math participation, the school's report card showed a two-star rating for math (as well as reading)— due entirely to the school's failure to meet the requirement for student participation on the reading test.

After the AYP determination was made (and after the Minnesota Department of Education's prescribed 30-day period for making appeals had passed), the district discovered that the number of students who took the Franklin Elementary reading test was actually one more than previously thought. 37 If this student had been counted as taking the test, the school would have met the NCLB-required 95 percent test participation rate. But, while acknowledging that the AYP determination was based on erroneous data, the Minnesota Department of Education decided not to change the earlier determination that the school had f ailed to make AYP— because the appeal regarding this student was not submitted within the 30-day appeal period. 38

COMPLIANCE WITH "ADEQUATE YEARLY PROGRESS" (AYP) REQUIREMENTS 49 On the department's report cards, schools received low ratings in both math and reading even if students performed poorly in only one subject.

34 The U. S. Department of Education worked with several non-profit organizations, including Standard and Poor's Evaluation Services, to develop a single web site where states can choose to display the report card information required by NCLB. The Minnesota Department of Education has chosen to comply with NCLB's report card requirements by participating in this project. Minnesota's data (state, school district, and school levels) were posted on the web site in January 2004. 35 Minnesota Department of Education, "Governor Unveils Accountability on a Stick," August 21, 2003, http://children.state.mn.us/stellent/groups/public/documents/translatedcontent/pub_ 034011.jsp, accessed December 30, 2003.

36 For schools that made AYP, the department considers various factors when determining whether they will receive three, four, or five stars.

37 This student was mistakenly categorized as "refused" (that is, refusing to take the test), even though the student had, in fact, taken the test and scored at a proficient level. Although the district submitted a timely appeal to the department regarding five free and reduced-price lunch students, its appeal did not include the above student.

 

In our view, it is misleading for a school report card to label schools as low-performing in math only because they failed to make AYP in reading. Likewise, it is misleading to label schools as low-performing in reading only because they failed to make AYP in math. In 2002-03, there were 50 Minnesota schools that failed to make AYP in one subject but not the other— and in each case, the department's report cards for these schools declared that the schools were deficient in both math and reading. Fortunately, the department recently decided to change its method of determining school ratings. For the 2003-04 report cards, the department plans to base math ratings only on math scores and reading ratings only on reading scores.

However, we question whether AYP determinations should remain unchanged indefinitely if errors are discovered after the end of the appeals process. Minnesota schools were first ev aluated in 2003 using the AYP definitions and processes prescribed by NCLB, and school districts are still figuring out the best way to review their AYP data and identify possible mistakes. AYP determinations can have many important implications— on parents'perceptions of schools (and their enrollment decisions), on the morale of school staff, and on the NCLB sanctions to which schools are subject. Although we understand the department's desire to have a timely, orderly appeals process, we think it is especially important for AYP-related sanctions to be based on accurate information, when possible.

RECOMMENDATION In cases where the Minnesota Department of Education determines that an AYP determination was made in error, it should ensure that the error does not adversely affect the school's or school district's sanction status in subsequent years.

New Measures of Student Performance In Chapter 2, we noted that many school district officials have concerns about the NCLB definition of AYP. In particular, they noted that, while the definition is based on the proportion of students deemed proficient in a given year, it does not take into account academic progress made by individual students over time. They expressed concern that some students— because of disabilities or limited English skills— may have diff iculty achieving proficiency in the same time frame as other students, even if these students are making steady progress. Although the federal NCLB law allows states to develop additional measures of student performance for determining AYP, the law says that these measures can only be used to identify additional schools needing improvement; these measures cannot change

50 NO CHILD LEFT BEHIND Under NCLB, measures of achievement gr owth by individual students over time are not considered when assessing school performance.

38 Cheri Pierson Yecke, Commissioner, Minnesota Department of Education, letter to Jerry Williams, Superintendent, Rochester School District, October 21, 2003.

AYP determinations for schools that would otherwise be subject to NCLB sanctions. 39

Minnesota law requires the development and implementation of "value-added" measures of student achievement— that is, measures that assess the progress of individual students over time. 40 Specifically, the law requires the Commissioner of Education to develop and publicly report value-added measures of achievement growth by the 2006-07 school year. In light of the concerns expressed by school district officials regarding existing AYP measures, we asked the commissioner about options for incorporating value-added measures into the AYP definition. She said that it makes sense to track student progress over time and that the department hopes to incorporate value-added measures into the state's process for making AYP determinations.

It is unclear exactly how the department intends to incorporate value-added measures of student achievement into the AYP determination process, given the constraints of federal law. The federal government has rejected proposals from some other states to incorporate value-added measures into the AYP, and federal officials gave us no indication that they would accept such a proposal from Minnesota. We think the department should provide the Legislature with a specific plan for how a value-added measure could be incorporated into the AYP process. State law required the previous education commissioner to prepare a report on integration of value-added measures with AYP measures (by January 2002), 41 but the department recommended deferring decisions about integration of value-added measures until later.

RECOMMENDATION The Minnesota Department of Education should provide the 2005 Legislature with a plan that outlines how value-added measures of student achievement could be incorporated into the annual AYP determination process.

In our view, the department's plan should address (1) how the use of a v alue-added measure in the AYP process would comply with federal law, and (2) what criteria would be used to assess "adequate" growth, using a value-added measure. In addition, some test experts suggested to us that there may be unintended inconsistencies in the diff iculty levels of assessments administered to Minnesota students in various grades. That is, perhaps a score of 1500 on a 3rd-grade MCA has a somewhat different meaning than a score of 1500 on a 5th-grade MCA. 42 If necessary, the department should also propose any steps that would be needed to ensure that growth can be measured in valid, reliable ways. Finally, the department should clarify whether incorporating value-added measures into the the AYP process would be administratively feasible. For example, if the department decided to incorporate value-added measures through the AYP appeals process rather than by changing the AYP definition, it might be challenging for the department to act in a timely way on appeals filed by potentially hundreds of schools.

COMPLIANCE WITH "ADEQUATE YEARLY PROGRESS" (AYP) REQUIREMENTS 51 The Minnesota Department of Education should clarify its plans for incorporating measures of individual student growth into its assessments of school progress.

39 No Child Left Behind Act, §1111( b)( 2)( D)( ii). 40Minn. Stat. (2003 Supplement), §120B. 30, subd. 1a. 41 Laws of Minnesota (1Sp2001), ch. 6, art. 2, sec. 5. 42 This could occur because the tests for various grade levels are reviewed by separate testing committees. Also, unlike some states, Minnesota has not "vertically linked" its tests— that is, made a systematic effort to adjust the scoring levels to provide for consistency across grades.

 

Students w ith Limited English Skills In recent years, many Minnesota schools have experienced increases in their number of students with limited English skills. The Minnesota Department of Education reported that 6 percent of students in Minnesota public schools had "limited English proficiency" (LEP) in 2003, up from 4 percent in 2000. 43 Some individual schools and school districts have high concentrations of LEP students. For example, LEP students comprised 25 and 34 percent of the 2003 students in Minnesota's two largest school districts (Minneapolis and St. Paul, respectively), and LEP students in some districts speak dozens of languages.

Under NCLB, students who are learning the English language are tested for math and reading proficiency , and schools are held accountable for their performance. School officials have expressed concerns that it is unreasonable to expect students lacking a basic understanding of English to be academically proficient. In 2003, students who had passed Minnesota's Test of Emerging Academic English were, for the most part, not counted in the LEP subgroup for purposes of AYP calculations. Thus, the LEP subgroup consisted of students who, by definition, were not proficient in English, and many schools'LEP subgroups did not make AYP.

In November 2003, the Minnesota Department of Education received approv al from the federal government to change Minnesota's LEP definition. For purposes of AYP determinations, the department expanded the LEP subgroup to include students who have passed the state's English language proficiency assessments but have not yet demonstrated proficiency in reading on the MCAs for three years. Once students consistently pass the reading MCAs, they will no longer be classified as LEP. Thus, the LEP subgroup will now consist of students who are making the transition to full use of the English language, in addition to those students who lack English proficiency altogether.

This change will improve the ability of schools'LEP subgroups to make AYP in 2003-04. But, starting in 2005, Minnesota will compute AYP using increasingly strict standards— until, in 2014, all students are expected to be proficient. Thus,

To the extent that schools' LEP subgroups continue to have students who, by definition, lack proficiency in the English language, it is unlikely that these subgroups will meet the NCLB goal of having all students achieving academic proficiency.

52 NO CHILD LEFT BEHIND NCLB appears to have unrealistic expectations for limited-English students.

43 Minnesota Department of Education, "Minnesota School & District Information/Analysis, State of Minnesota Data Analyis," http://education.state.mn.us/html/intro_ mde-analysis.htm, accessed February 4, 2004.

In summary, NCLB appears to have unrealistic expectations for some students, and it will be increasingly difficult for Minnesota schools to comply with the law as the next decade progresses. This is further reason for the department to clarify how it intends to use measures of individual student achievement growth in making AYP determinations— as we recommended in the previous section. While it may be unrealistic to expect all recent immigrants to be academically proficient, it would be more realistic to expect them to make substantial growth toward the goal of proficiency .

Validity and Reliability of the Accountability System

Minnesota's statewide educational accountability system is relatively young. In Chapter 2, we noted that Minnesota was in the process of implementing a statewide accountability system at the time that NCLB became law. The federal law's enactment in early 2002 altered the nature of Minnesota's emerging accountability system— requiring more tests, more ambitious performance goals, and stricter sanctions than Minnesota was implementing previously.

While implementing federal accountability requirements, Minnesota officials have taken some important steps to help ensure that these provisions are applied fairly and accurately. For example, Minnesota has adopted practices that are intended to ensure that AYP determinations reflect student performance, rather than measurement error. 44 Also, the 2003 Legislature created an assessment advisory committee to review all statewide assessments prior to final implementation and make recommendations to the Legislature and education commissioner. 45

Still, as we simulated future AYP determinations and talked with education experts, we concluded that some important questions remain regarding the validity and reliability of Minnesota's accountability system. For instance:

Validity of AYP determinations. To the extent possible, AYP determinations should accurately reflect school quality and performance. But, if our simulations of future AYP levels are based on reasonable assumptions, there could be very high percentages of Minnesota schools that fail to make AYP in future years. This would suggest either that (1) many schools are not performing well, or (2) the AYP determinations are inaccurate or based on overly stringent expectations. Regarding the latter possibility, perhaps NCLB's expectations (aiming toward 100 percent proficiency by 2013-14) are simply unrealistic for all subgroups to achieve. Or, perhaps the definition of AYP flags too many schools due to random fluctuations in performance levels among their student subgroups. 46

COMPLIANCE WITH "ADEQUATE YEARLY PROGRESS" (AYP) REQUIREMENTS 53 Questions remain about the validity and reliability of Minnesota's education accountability system.

44According to Minnesota's NCLB plan, AYP proficiency determinations are not made for subgroups with fewer than 20 students (or 40 students, in the case of proficiency determinations for special education students). In addition, determinations regarding test participation are not made for subgroups with fewer than 40 students. Also, Minnesota uses "confidence intervals" ranging from 0.95 to 0.99, depending on the number of subgroups for which a school is being held accountable. These confidence intervals are intended to reduce the likelihood that a school's performance will be labeled inadequate if, in fact, it is adequate.

45Laws of Minnesota (2003), ch. 129, art. 1, sec. 11.

Consistency of proficiency levels among grades. When students at various grade levels take the Minnesota Comprehensive Assessments (MCAs), a score of 1420 indicates "proficiency." However, as noted earlier, it is unclear whether the MCA scores reflect consistent difficulty levels across various grades. Thus, there may be questions about (1) the practice of aggregating test scores from multiple grade-levels for purposes of making AYP determinations, or (2) whether it is possible to draw valid conclusions about individual students' progress over time.

Adequacy of the MCAs. These tests are supposed to align with Minnesota's academic standards, and state law says they should serve diagnostic and accountability purposes. 47 Although Minnesota adopted new statewide math and reading standards in 2003, the Minnesota Department of Education does not expect to revise the MCAs for a year or two to reflect the new standards. More important, however, school district superintendents that we surveyed expressed limited confidence in the MCAs for diagnostic or accountability purposes, as we noted in Chapter 2. 48

The NCLB Act requires states to use "valid and reliable" assessments and AYP measures. 49 The Minnesota Department of Education reported to the federal government that it planned to solicit proposals by September 2003 for an ev aluation of the validity of Minnesota's accountability system. 50 (As of early 2004, the department has not solicited proposals.) In addition, Minnesota law established an independent Office of Educational Accountability to assess whether the state's accountability system has valid, reliable measures. 51

RECOMMENDATION The Minnesota Department of Education and the Office of Educational Accountability should report to the 2005 Legislature on any unresolved issues regarding the validity and reliability of Minnesota's education accountability system.

46It would be interesting to track whether schools with persistent failure to make AYP have repeated underperformance by the same subgroups of students, rather than underperformance by differing subgroups in different years. In addition, it would be interesting to determine whether the number of subgroups for which individual schools are held accountable affects their likelihood of making AYP.

47Minn. Stat. (2003 Supplement), §120B. 30, subd. 1( b) and subd. 1a. 48In the only external critique of the MCAs, a 2001 evaluation conducted for the Minnesota Department of Education concluded that "the tests contain a number of strong elements," but "in both English and mathematics, the tests are not as challenging as they could be." See Achieve, Inc.,

Measuring Up: A Report on Education Assessments for Minnesota (Washington, D. C., April 2001), 3, 16.

49No Child Left Behind Act, §§ 1111( b)( 2)( C) and 1111( b)( 3)( C)( iii). 50Minnesota Department of Education, Minnesota Consolidated State Application Accountability W orkbook( Roseville, MN, January 31, 2003), 38.

51 Minn. Stat. (2002), §120B. 31, subd. 3.

Standards for Monitoring the Performance of Supplemental Services Providers

If a school fails to make AYP for three consecutive years, its school district must offer "supplemental educational services," such as tutoring, to low income families with children at the school. Public and private organizations may apply to offer supplemental services, and the Minnesota Department of Education decides which of these agencies will be approved to provide services. As authorized by the 2003 Legislature, the department promulgated temporary rules last year that outline the criteria the department uses to approve supplemental services providers. 52 Once approved by the state, a service provider remains on the state's list of approved providers for three years and it may then re-apply for state approval. 53

NCLB requires state education agencies to "develop, implement, and publicly report on standards and techniques for monitoring the quality and effectiveness of the services offered by approved [supplemental services] providers . . . and for withdrawing approval from providers that fail, for two consecutive years, to contribute to increasing the academic proficiency of students [they serve]." 54 In other words, the Minnesota Department of Education must not only select which providers will be initially authorized to offer supplemental services, but it must also monitor the providers'subsequent performance. The department is still determining the details of how it will monitor supplemental services providers. Department staff told us that they expect to ev aluate the providers based partly on the academic performance of students enrolled in supplemental services. Staff also said that they will rely on parents and school districts to play a significant role in monitoring providers.

We think there are important questions about the standards that the department will use to judge supplemental services providers. For example, if the department tracks students'standardized test results or other measures of individual student performance to assess the effectiveness of supplemental services providers, it is unclear how the department will disentangle the impact of the providers from the impact of regular school instruction. In addition, it would be useful to know whether the department anticipates using an absolute measure to assess the effectiveness of providers (such as the percentage of students achieving "proficiency") or a measure of individual student growth over time toward the goal of proficiency .

Department staff told us they hope to determine methods of overseeing supplemental services providers in early 2004. We think that legislators should review the approaches that the department proposes. Just as the Legislature has discussed measures of "adequate yearly progress" for schools and school districts, we think the Legislature should examine the standards by which the department plans to hold supplemental services providers accountable. We offer no opinion about whether, or to what extent, the department's methods of ongoing provider monitoring should be addressed in state rules, but we think the department needs to articulate monitoring policies that are more specific than those in existing rules. 55

COMPLIANCE WITH "ADEQUATE YEARLY PROGRESS" (AYP) REQUIREMENTS 55 The Minnesota Department of Education should clarify its standards for monitoring the effectiveness of tutoring services.

52 The Department of Education does not have general rule-making authority; it must have legislative authorization to promulgate rules. The department is now seeking legislative approv al to issue permanent rules on supplemental services.

53 Minn. Rules (2003), ch. 3512.5400. The rules also outline circumstances in which the state may remove providers from the list of approved providers.

54No Child Left Behind Act, §1116( e)( 4)( D).

 

RECOMMENDATION The 2004 Legislature should require the Minnesota Department of Education to submit a plan to the House and Senate education committees that outlines how it will monitor the quality and effectiveness of supplemental educational services providers.

56NO CHILD LEFT BEHIND

55For example, the 2003 temporary state rules discuss circumstances in which providers may be removed from the list of state-approved providers, but they do not specify how the state would comply with the NCLB provision for withdrawing approv al from providers that fail to increase student proficiency for two consecutive years.

Fiscal Impacts SUMMARY The No Child Left Behind (NCLB) Act will likely have a significant fiscal impact on Minnesota's education system in the next decade— although the exact magnitude of many of these costs cannot be accurately predicted. The state and school districts will likely spend a total of roughly $19 million annually to develop and administer reading and math assessments in three additional grades, plus three new science assessments. In addition, most Minnesota school districts will probably have to set aside funds to pay for sanctions and services for low-performing schools, and some districts will incur higher costs to attract or retain staff who meet NCLB's more stringent qualification requirements.

If Minnesota were to "opt out" of NCLB, it is unclear whether the cost savings would be sufficient to offset the annual loss of federal education funds. In fact, less than 20 percent of school district superintendents said they would favor opting out, despite their serious reservations about NCLB. Nevertheless, it is quite plausible that the cost of implementing the new requirements of NCLB (above what the state was doing prior to NLCB) could exceed the increase in funding that Minnesota has received under the act.

Policy makers in several states, including Minnesota, are questioning whether their states should participate in NCLB. In fact, five states (Indiana, North Dakota, Ohio, Utah, and Vermont) passed measures requiring studies to figure out how much it will cost to comply with NCLB's testing and reporting requirements. 1 Similarly, the Minnesota Legislative Audit Commission directed our Office to carry out this study. Therefore, in this chapter, we address the following questions:

How much will it cost the Minnesota Department of Education, school districts, and schools to carry out the NCLB Act?

Which categories of NCLB-related costs will likely be the most significant, and what factors will affect the magnitude of these costs? How do the new costs that Minnesota is incurring under NCLB compare with Minnesota's increased revenues under the act?

Some policy makers are concerned about the costs that NCLB will impose on Minnesota.

1 Pamela M. Prah, "Utah Considers Opting Out of No Child Left Behind," Stateline. org, December 30, 2003, http://www.stateline.org/stateline/?pa= story& sa= showStoryInfo& id= 341320, accessed January 16, 2004.

How have school districts paid for any increased costs resulting from NCLB? What would be the likely financial consequences if Minnesota "opted out" of participation in the NCLB Act?

BACKGROUND One of the methods we used to evaluate the cost of implementing NCLB's new requirements was to collect cost estimates from the Minnesota Department of Education and a sample of nine school districts. (The districts were Bloomington, Detroit Lakes, Mahnomen, Minneapolis, Osseo, Rochester, Shakopee, St. Paul, and Willmar.) To facilitate the consistent collection of cost data, we gave instructions to the department and districts regarding how to estimate costs related to Title I, Part A of NCLB. Some of the key instructions included the following:

Cost categories: We identified over 200 activities needed to carry out Title I, Part A and grouped them into 26 categories of state activities and 25 categories of local activities. (Listings of these activities and cost categories are provided on our web site— http://www.auditor.leg.state.mn.us/Ped/2004/pe0404.htm.) We then asked the Minnesota Department of Education to estimate its costs for each category of state activities, and we asked the nine school districts to estimate their costs for each category of local activities. These estimates included (1) the staff time and associated costs needed to carry out the NCLB-related activities, and (2) the cost of significant non-staff items, such as supplies and contracts with vendors.

"Total costs" and "NCLB costs": For each cost category, we asked for estimates of (1) the total cost of carrying out the NCLB-related activities, even if they would have been carried out without NCLB, and (2) the portion of the total costs directly attributable to NCLB. We defined these NCLB-attributed costs as the resources needed to carry out any activity that is newly required under the NCLB Act and that would not have been carried out by the state, school districts, or schools on their own. Specifically, we did not want the Department of Education or school districts to attribute costs to NCLB for activities required in (1) the previous version of the Elementary and Secondary Education Act (ESEA), (2) a federal law other than the ESEA, or (3) state law. In addition, we wanted to exclude the cost of NCLB-required activities that the department, school districts, or schools would have carried out on their own, even without a federal or state requirement. For example, NCLB requires annual assessment of 3rd graders in reading. While the median school district estimated that it annually spends $17 per 3rd -grade student to administer this assessment, districts did not attribute these costs to NCLB because they were administering this assessment prior to NCLB in order to comply with the previous version of the ESEA.

58NO CHILD LEFT BEHIND ACT We obtained and ev aluated NCLB-related cost estimates from the Minnesota Department of Education and nine school districts.

Time period: For the NCLB activities with more definitive costs, such as developing and administering assessments, we asked for estimates that spanned state fiscal years 2002 through 2008. In contrast, for the activities with more speculative costs, such as supplemental services provided by Title I schools, we only asked for estimates that spanned fiscal years 2002 through 2005. We requested estimates in today's dollars, without adjustments for price inflation and cost-of-living changes.

Financing of costs: We wanted state and district officials to include costs in their estimates regardless of how they have paid (or will pay) for them— whether through additional federal, state, or local funding or through reallocation of existing resources. For example, implementing additional testing will require teachers to spend additional time proctoring tests rather than teaching; we instructed districts to count these teacher costs in their estimates. Likewise, the state and districts may accomplish other NCLB tasks by reassigning existing staff rather than by spending more money, and state and district officials included these staff costs in their estimates.

We considered the department and district cost estimates to be informative but not definitive because:

Identifying and estimating NCLB-related costs is very challenging and can be quite subjective.

First, existing information systems provide a limited basis for attributing state or local expenditures to NCLB. The Minnesota Department of Education has not had an activity-based time-reporting system to track staff time associated with NCLB. In addition, the department's statewide system for reporting school districts' expenditures (UFARS) is limited in its tracking of specific NCLB expenditures. Consequently, in many cases, state and district officials provided NCLB cost estimates based on their best recollection of staff time dev oted to specific activities, and these estimates should be considered tentative.

Second, many NCLB-required activities have not yet been undertaken. Thus, there is uncertainty about exactly how these activities will be implemented and what costs will be incurred. For example, to estimate the cost of providing supplemental services to students at schools that have failed to achieve adequate yearly progress (AYP) for at least three consecutive years, districts would have to consider (1) how many schools would fall into this category each year and (2) how many parents would choose to enroll their children in these services. Presently, however, only two districts have had experience offering supplemental services.

Third, in many cases, state and district officials had a diff icult time deciding which costs to attribute to NCLB. For example, the St. Paul school district reported:

FISCAL IMPACTS 59 Reliable cost estimates are not y et possible.

Superintendent Patricia Harvey brought an agenda to [St. Paul Public Schools] five years ago when she arrived in the district… That agenda has some of the same characteristics as NCLB. From that perspective, it was not an easy task to differentiate between the cost of our own agenda and that of NCLB. We found that it took an incredible amount of time and effort on the part of staff to analyze expenditures and we made some very subjective decisions in arriving at our costs. 2

Finally, our sample of nine school districts was neither random nor representative of all districts. The sample contained districts of differing sizes from various parts of Minnesota, and officials from these districts offered a range of perspectives regarding NCLB implementation. Still, it is worth noting that these districts tended to be somewhat larger than the average Minnesota district and have above-average proportions of low income, limited-English, and minority students.

OVERVIEW OF NCLB COSTS We used various methods to explore NCLB's fiscal impacts, but this report does not present a dollar estimate of NCLB's overall cost— either for the present time or for some future date. The primary reason is that the cost of implementing the law depends largely on events that cannot be accurately predicted. Nevertheless, we think that the law will likely have a substantial fiscal impact— based on (1) our analysis of state and local cost estimates, (2) interviews with stakeholders, including state and district officials, (3) our superintendents'survey , and (4) the simulation results presented in Chapter 3.

Although we do not offer a "bottom line" estimate of NCLB's costs, we think that it is possible to identify the expenditure areas that NCLB will most significantly affect. In our view,

NCLB's main fiscal impacts will be in the areas of (1) development and administration of student assessments, (2) sanctions and services for low-performing schools, and (3) compliance with requirements concerning teacher and paraprofessional qualifications. School districts also face potentially large curriculum alignment costs, but it is debatable how much of these costs should be attributed to NCLB.

We discuss these categories in subsequent sections of this chapter. However, Table 4.1 provides an overview of our judgments regarding NCLB's likely statewide fiscal impacts. In this table, we generally classified NCLB-attributed costs as "small" if they were expected to total $1 million or less annually on a statewide basis.

As part of our assessment of NCLB costs, we also asked superintendents to identify which category of NCLB activities would likely impose the greatest costs on their school districts. The superintendents'most common response was sanctions and services for low-performing schools, as shown in Table 4.2. It is worth noting that all of the categories in this table reflect costs that have yet to be fully realized. First, while the state will develop and pilot test questions over the next few years, the reading and math assessments for grades 4, 6, and 8 will not be fully operational for AYP determinations until state fiscal year 2006, and the science assessments will not be operational until 2008. Second, the costs associated with sanctions and services— school choice, supplemental services, corrective action, and restructuring— will only increase over the next several years as more schools fail to achieve AYP for multiple years. Finally, current teachers and Title I paraprofessionals do not need to meet NCLB's qualification requirements until 2006.

60 NO CHILD LEFT BEHIND ACT Future NCLB costs will depend on events that cannot be pr edicted.

2 Lois Rockney , Executive Director of Business and Financial Aff airs, St. Paul Public Schools, letter to John Patterson, Office of the Legislative Auditor, December 23, 2003.

FISCAL IMPACTS 61 Table 4.1: Fiscal Impact of NCLB-Initiated Activities Cost Category Fiscal Impact Attributable to NCLB General ESEA administration State— Small impact. Local— Small impact.

At both the state and local level, there have been some initial planning and implementation activities required by NCLB, but many of the general administrative and financial duties mandated by NCLB are similar to previous requirements. Standards development and curriculum alignment State— Small impact. The state's cost for developing grade-specific standards in reading, math, and science has been small. Local— Potentially significant impact, but attribution to NCLB is debatable. While aligning school districts' curriculum with state content standards can be expensive, it is debatable how much of these costs should be attributed to NCLB. NCLB would have required some curriculum alignment by districts, and the high stakes nature of NCLB may have created an urgency for districts to devote more resources to these efforts. But others contend that due to the state's independent adoption of new content standards in 2003, districts would have had to carry out curriculum alignment without NCLB. Student assessment State— Significant future impact. In order to comply with NCLB, the state has to develop and administer (1) reading and math assessments for grades 4, 6, and 8, (2) three science assessments, and (3) English proficiency assessments in listening and speaking. Federal revenues for test development will offset many, or all, of the state's costs. NCLB appears to have been the major impetus for the development of the reading and math assessments in grades 4, 6, and 8; in contrast, the other reading and math assessments required by NCLB were already in place or were in the planning stages prior to NCLB. a Local— Significant future impact. Once school districts begin to administer the reading and math assessments for grades 4, 6, and 8 in state fiscal year 2006, and the science assessments in state fiscal year 2008, the districts will face significant new costs. b AYP determination State— Small impact. Federal law prior to NCLB required states to calculate AYP for Title I schools; however, NCLB now requires some additional state effort (e. g., AYP calculations for all schools, rather than just Title I schools; and separate AYP calculations for a variety of subgroups). Local— Small impact. As a result of the high stakes nature of NCLB's sanctions, school districts have spent additional time verifying demographic and assessment data used to determine schools' AYP status. Also, as school districts administer assessments at more grade levels, more verification will be required. Nevertheless, these costs should be relatively small. Reporting (report cards and notices) State— Small impact. The state developed templates to help school districts provide notices regarding school sanctions. In addition, there will be some staff costs to prepare NCLB-specific school report cards each year, starting in 2004. Local— Small impact. Some NCLB notification requirements (notifying parents of assessment results, providing notices regarding Title I parent meetings) were in federal law before NCLB. But NCLB contained new requirements for parent notices regarding school sanctions and teacher qualifications. Sanctions and supplemental services State— Small but growing impact. The state has incurred some additional costs to administer supplemental services. It may incur additional costs as more schools and school districts fail to make AYP, thus requiring more assistance from the state and more time to oversee service providers. Local— Potentially significant impact in the future. Relatively few schools and school districts have been categorized as "needing improvement," but this number will grow— resulting in implementation of more improvement plans, school choice-related transportation, supplemental services, corrective actions, and school restructuring. NCLB requires affected districts to set aside the equivalent of up to 20 percent of their Title I, Part A funds for school choice and supplemental services. Teacher and paraprofessional requirements State— Small impact so far; future costs are unclear. The state has spent staff time determining its qualification standards for teachers and paraprofessionals. Future costs will depend on how the state monitors compliance with its standards and how many districts comply. Local— Variable among districts, but potentially significant impact. To comply with NCLB's stricter requirements regarding staff qualifications, districts could incur costs for salary increases, training, or hiring additional staff. But the state's standards were still in flux at the time districts provided estimates, and many districts did not know what actions they would take to comply with the requirements. Professional development State— Small impact so far; future costs are unclear. The state has not yet defined or set annual objectives for implementing "high quality" professional development, and it is unclear how the state will monitor professional development activities. Local— Small impact so far; future costs are unclear. Until standards and objectives are set by the state, it is difficult to estimate additional costs that might be incurred by school districts. Parental involvement State— No impact. Local— Small impact. Some districts have intensified their parent-related activities in response to NCLB, but the overall cost of these efforts is limited. NOTE: "Small" impacts are those estimated to cost less than $1.5 million annually on a statewide basis. a Some people contend that the state would have implemented the grade 4, 6, and 8 tests without NCLB, as reflected in the fact that these tests are required in state law. But the Legislature adopted this testing requirement after NCLB was enacted.

b However, there may be some offsetting cost savings if districts decide to discontinue some of their non-NCLB standardized tests. SOURCE: Office of the Legislative Auditor.

 

In our assessment of NCLB costs, we did not try to estimate the cost of ensuring that 100 percent of students are proficient by the 2013-14 school year, which is the ultimate goal of NCLB. The costs associated with such efforts are highly uncertain, and any estimate would be extremely speculative. As we discussed in Chapter 3, there is considerable debate about the role that educational funding plays in increasing achievement. But even with significantly higher spending levels, it might be diff icult for all children to become proficient— due to cognitive impairments, English language difficulties, or other factors.

In the following sections, we discuss the significant cost categories— the development and administration of student assessments, sanctions and services for low performing schools, teacher and paraprofessional qualifications, and curriculum alignment— in more detail.

62 NO CHILD LEFT BEHIND ACT Table 4.2: Superintendent's Opinions Regarding Which NCLB Requirements Will be the Most Costly Percentage of Superintendents who Identified this Requirement NCLB Requirement as the Most Costly Implementing sanctions and additional services for low-performing schools 33%

Implementing additional grade-level tests 26 Complying with new requirements for paraprofessional qualifications 26 Complying with new requirements for teacher qualifications 11 Did not respond to question 4

NOTE: The survey question asked, "In your judgment, which one of the following requirements of the NCLB Act will be the most costly for your district to implement?" The question did not list curriculum alignment as an option for respondents. ( N= 326) SOURCE: Office of the Legislative Auditor, Survey of School District Superintendents (November-December 2003).

School districts have not yet fully implemented the most expensive components of NCLB.

COSTS RELATED TO ASSESSMENT DEVELOPMENT AND ADMINISTRATION As discussed earlier, NCLB requires states to implement a wide range of annual student assessments, including:

Reading and math assessments for each grade level between 3rd and 8th and once during the high school grades;

Separate science assessments for grade spans 3-5, 6-9, and 10-12; and Assessments of English proficiency in reading, writing, listening, and speaking for students who speak a foreign language at home and have limited English proficiency. 3 Minnesota's assessments of reading, math, and science are the Minnesota Comprehensive Assessments (MCAs). The state has two assessments of English proficiency . The Test of Emerging Academic English (TEAE) assesses proficiency in reading and writing, while the Minnesota Student Oral Language Observational Matrix (MNSOLOM) assesses proficiency in listening and speaking.

According to Minnesota Department of Education cost estimates, the department will spend $18 million to administer all the NCLB-required assessments (including those developed prior to NCLB) in state fiscal year 2008, when they will be fully operational for accountability purposes. These costs largely inv olve department contracts with the vendors who develop, distribute, and score the assessments. In addition, based on cost information that districts provided, we estimated that districts and schools will spend roughly $21 million annually administering these assessments, which inv olves packing and unpacking the materials, maintaining assessment security and integrity, and proctoring classrooms while students take the assessments. 4

In our view, however, only the costs of developing and administering the reading and math assessments in grades 4, 6, and 8, the science assessments, and the MNSOLOM should be attributed to NCLB. Unlike the other assessments, the state was under no requirement to implement these assessments prior to NCLB. We estimated that:

The Minnesota Department of Education, school districts, and schools will spend roughly $19 million to implement these new assessments in state fiscal year 2008. The Minnesota Department of Education estimated that it will spend $8 million annually to carry out its part of these assessments, and we estimated that school districts and schools will spend approximately $11 million annually for their part.

FISCAL IMPACTS 63 NCLB requires an extensive amount of student testing.

3 No Child Left Behind Act, §1111( b)( 3) and (7). 4Our estimate is based on the median per pupil estimate that we received from the districts for each of the assessments multiplied by the number of students taking each of the assessments statewide.

As stated, the costs of the other assessments should not be attributed to NCLB. Prior to NCLB, the state was already administering the reading and math assessments for grades 3 and 5, and it had plans to develop and administer the reading and math assessments for grades 7, 10, and 11 to comply with pre-NCLB federal law. Table 4.3 shows Minnesota's schedule for implementing the reading, math, and science assessments. Prior to NCLB, the state had also developed the Test of Emerging Academic English (which assesses English proficiency in reading and writing) to identify students eligible to receive additional state funding for English language instruction. 5

There is some debate about whether the state would have developed and administered the reading and math assessments for grades 4, 6, and 8 if NCLB had never been enacted. In 2001 (prior to NCLB), the Legislature required the state to develop a system for measuring the growth in individual students' educational progress over time based on state or district assessments. 6 To the extent that Minnesota needs scores from state assessments at each grade level to measure this progress, there is an argument that the reading and math assessments in grades 4, 6, and 8 would have been developed without NCLB and thus should not be attributed to NCLB. On the other hand, Minnesota law did not require annual assessments until 2003, well after the passage of NCLB. 7 The Minnesota Department of Education did not attribute the costs of the 4th , 6th , and 8th grade assessments to NCLB in the cost estimates it prepared for us; in contrast, the school districts did attribute the costs of these assessments to NCLB.

64 NO CHILD LEFT BEHIND ACT Table 4.3: Minnesota's Schedule for Initiating Assessments for Performance Reporting Grade Reading Math Science 3 XX 4 2005-06 2005-06 2007-08 5 XX 6 2005-06 2005-06 7 2003-04 2003-04 2007-08 8 2005-06 2005-06 9 —— 10 2003-04 — 11 —2003-04 2007-08 12 ——

NOTE: Where a school year is shown, this is the first year that this test will be used for purposes of performance reporting. "X" designates a presently-administered assessment that predates NCLB requirements. "—" designates subjects/ grades in which no test is required. SOURCE: Minnesota Department of Education.

Minnesota was implementing several statewide tests prior to the passage of NCLB.

5Laws of Minnesota (1Sp2001), ch. 6, art. 3, sec. 4. 6Laws of Minnesota (1Sp2001), ch. 6, art. 2, sec. 5. 7Laws of Minnesota (2003), ch. 129, art. 1, sec. 7.

We based our estimates of statewide school district assessment costs on the median cost estimate that we received from school districts. (The median cost estimate for administering the Minnesota Comprehensive Assessment was $17 per student taking each subject assessment.) Although there are many similarities among districts in the assessment administration process, we received estimates from districts that varied from $4 to $50 per student taking the assessment. 8there are several reasons for this variation. Most notably, the districts with higher estimates assumed longer test administration times than the other districts. For, example, one of the high-end districts estimated that each assessment for each subject takes the equiv alent of one full day to administer because it is untimed and the district has many students who take a long time, such as limited-English students. In contrast, one of the low-end districts assumed that each assessment takes only three hours to administer. 9

By administering the NCLB assessments, school districts could potentially achieve some offsetting savings. According to our survey of superintendents, 90 percent of Minnesota school districts have administered standardized tests other than the NCLB-required assessments in order to assess the skills and achievement levels of individual students in their district. (The most common of these tests in Minnesota are the Iowa Test of Basic Skills, the Northwest Achievement Levels Test, and the Stanford Achievement Test.) By discontinuing these non-NCLB tests, districts would av oid the cost of administering them. In f act, in our statewide survey , 64 percent of school district superintendents reported that they have dropped or will likely drop at least some of their non-NCLB tests as a direct result of NCLB. If districts are dropping these tests because the MCAs will meet their student assessment needs, the cost savings are unmitigated. However, if the MCAs will not meet the districts'assessment needs and districts are discontinuing the other tests out of financial necessity, there will be a budgetary savings but also a loss of assessment information that schools and teachers could use to better meet the academic needs of their students. In this latter case, we would consider the cost savings quite tenuous. Many districts appear to fall in this category. Of the 210 districts that reported that they have dropped or will likely drop at least some of their non-NCLB tests, only 37 percent agreed with the statement, "The [MCAs] help teachers understand the specific academic needs of individual students." 10 We did not try to estimate the amount of unmitigated cost savings that districts may incur by discontinuing some of their non-NCLB tests, but it is a potential offsetting factor to the NCLB costs discussed earlier in this section.

FISCAL IMPACTS 65 Most Minnesota school districts administer more standardized tests than are r equired by NCLB.

8This range applies to seven of the nine districts from which we received cost information. For two of the nine districts (one with a low estimate and the other with a high estimate), we questioned the accuracy of their estimates.

9The districts also varied in the number of classroom teachers and other staff needed to proctor and administer the assessments on a per student basis.

10 Office of the Legislative Auditor, Surve y of Sc hool District Superintendents (No vember -December 2003) . Fourteen percent neither agreed nor disagreed with the statement, and 50 percent disagreed.

COSTS OF NCLB-PRESCRIBED CONSEQUENCES FOR LOW PERFORMANCE Under NCLB, public schools that receive federal Title I funding face consequences if they fail to make AYP. 11 If a school fails to make AYP for two consecutive years, parents of all children at the school must be given choices of other schools to attend. After a school has failed to make AYP for three years, low income parents must be offered "supplemental educational services" (tutoring, for example) for their children attending this school. If a school fails to make AYP for four or five years, NCLB requires implementation of "corrective actions" or "restructuring," respectively. 12 Table 4.4 provides additional details on these consequences. There were no provisions for such consequences in the federal Elementary and Secondary Education Act or state law prior to NCLB, so any costs associated with these sanctions are directly attributable to NCLB.

NCLB establishes spending requirements for school districts that are subject to its school choice or supplemental educational services provisions. Specifically, unless a lesser amount is needed to provide choice-related transportation or satisfy all requests for supplemental educational services, the district must spend the equiv alent of 20 percent of its Title I, Part A allocation on these activities. Of this 20 percent, a district must spend 5 percent for choice-related transportation and 5 percent for supplemental services, with the remaining 10 percent divided between transportation and supplemental services according to the district's wishes. 13 Districts can pay for choice-related transportation and supplemental services with their Title I funds, or they can use other allowable federal, state, local, or priv ate revenues.

So far, School districts and the Minnesota Department of Education have borne limited costs related to school choice, supplemental educational services, and other NCLB sanctions— mainly because the NCLB Act is still at a very early stage of implementation.

During the 2003-04 school year, only 5 of Minnesota's 342 school districts (1.5 percent) and 2 of its 92 charter schools (2.2 percent) were required to implement school choice or supplemental services. The 25 Minnesota schools that were required by NCLB to offer school choice or supplemental services in 2003-04 served about 10,000 students, or 1.2 percent of Minnesota's total public school enrollment. No school will be considered for "corrective action" until the

Low-performing schools face sanctions under NCLB.

11 NCLB also prescribes sanctions for school districts that fail to make AYP for at least two consecutive years, but this section focuses primarily on sanctions for schools.

12 Under NCLB, schools required to offer supplemental educational services must continue to offer school choice. Schools subject to corrective action or restructuring must continue to offer both school choice and supplemental services.

13 No Child Left Behind Act, §1116( b)( 10).

2004-05 school year, at the earliest, and no school will have to begin "restructuring" planning until the 2005-06 school year, at the earliest. In the following sections, we discuss costs associated with these NCLB consequences in more detail.

School Choice Since 1988, Minnesota's "enrollment options" law has allowed pupils to attend school in a district where the pupil does not reside. 14 In addition, some districts have provided families with enrollment options within district boundaries— for instance, by allowing students to enroll in "magnet" schools 15 or other schools that have room for additional students. In some cases, families are expected to pay for transportation to schools outside of their designated "attendance areas;" in other cases, districts transport students to these schools free of charge.

FISCAL IMPACTS 67

Table 4.4: NCLB's Consequences for Repeated Failure to Make AYP Failing to make AYP for 2 or more years— school choice After a second year of failing to make AYP, a Title I school is "identified for improvement" under NCLB. Such a school must develop a school improvement plan, and the school district must notify all parents of children in the school that they are eligible to transfer their children to a higher-performing school (that is, one that has not been "identified for improvement"). Districts can determine which of their higher-performing schools will be options from which eligible parents can choose, and districts cannot subsequently deny transfers to these schools due to lack of space. In cases where there are no other schools in the district to which students could transfer, the federal government requires districts "to the extent practicable" to establish agreements with other districts to allow for inter-district choices. If funds are insufficient to provide transportation to each student requesting a transfer, the district must give priority for transportation funding to the lowest-achieving eligible students from low-income families. In Minnesota, the 2002-03 school year was the first year in which some districts were required to offer school choice under NCLB. Failing to make AYP for 3 or more years— supplemental educational services

If a Title I school fails to make AYP for three consecutive years, it must continue to offer school choice. In addition, its students from low-income families will be eligible to enroll in supplemental educational services outside the regular school day. These services must be "high quality, research-based, and specifically designed to increase the academic achievement of eligible children." Eligibility for supplemental services is notlimited to students in those grades or subgroups for which low performance resulted in the determination that the school did not make AYP. Public or private organizations apply to the Minnesota Department of Education to provide supplemental services, and the department determines which organizations meet the state's criteria. A school "identified for improvement" cannot provide supplemental services to its own students, but school districts (or individual schools) that have not been "identified for improvement" can apply to provide these services. Enrollment in supplemental services is voluntary. Once parents are notified of their child's eligibility for supplemental services, they may select from the state-approved providers serving that district. The staff of supplemental services providers are not required to meet the NCLB provisions that apply to public school teachers and paraprofessionals. School districts must pay supplemental services providers the lesser of (1) the district's Title I, Part A per-child allocation (the median per-child allocation among Minnesota districts was about $1,300 in fiscal year 2003), or (2) the actual cost of the services. If funds are insufficient to provide supplemental services to each eligible student whose parent requests the services, the district must give priority to the lowest-achieving eligible students.

Failing to make AYP for 4 years (corrective action) or 5+ years (restructuring) If a Title I school fails to make AYP for four consecutive years, NCLB requires the school district to take "corrective action," including at least one of the following: (1) replacement of staff, (2) implementation of a new curriculum, (3) reduction of school-level management authority, (4) appointment of an outside expert to advise the school, (5) extending the school year or school day, and/ or (6) restructuring the school. If an entire school district fails to make AYP for four consecutive years, the Minnesota Department of Education would be required to take similar actions or to reduce district funding for programs or administrative purposes. At the earliest, some Minnesota schools could be subject to corrective actions in the 2004-05 school year. If a school fails to make AYP for five consecutive years, the district must plan for implementation of alternative governance arrangements, such as reopening the school as a charter school, contracting for the school's management, or turning school operations over to the Minnesota Department of Education. Such plans would be implemented if the school fails to make AYP for a sixth year.

SOURCES: No Child Left Behind, §1116; U. S. Department of Education, Public School Choice: Draft Non-Regulatory Guidance (Washington, D. C., December 4, 2002); U. S. Department of Education, Supplemental Educational Services: Non-Regulatory Guidance, Draft— Final Guidance (Washington, D. C., August 22, 2003); U. S. Department of Education, LEA and School Improvement (Washington, D. C., January 7, 2004).

14 Minn. Stat. (2002), §124D. 03.

 

Because some districts offered enrollment options with subsidized transportation prior to NCLB, it is difficult to estimate the fiscal impact of NCLB's school choice requirements. 16 However, officials from four of the five Minnesota school districts that have been required by NCLB to offer school choice told us that these requirements have had minimal impact on their transportation costs so far:

Minneapolis: For the 2003-04 school year, 13 schools were subject to NCLB's school choice provisions. District officials said that only eight families requested school transfers for the 2003-04 school year in response to letters informing parents of their NCLB-related school choice options. 17 Staff said that these families have probably been served with existing transportation options, so the main NCLB-related costs to the district have been for mailings to parents and consultations with families considering transfers.

St. Paul: Officials estimated that about 250 families transferred for the 2003-04 school year from the seven schools required by NCLB to offer school choice. However, they said that most of these transfers were unrelated to NCLB, and they said that the level of transfers from these schools did not increase noticeably due to NCLB. Officials said that the few transfers that might be attributed to NCLB have not imposed significant new transportation costs on the district.

Mahnomen: For the 2003-04 school year, 14 students transferred from an underperforming elementary school to the district's other elementary school. Thirteen of these students are being served by previously existing bus routes, and the district modified a bus route to serve the other student.

Red Lake: For the 2003-04 school year, three children transferred from an underperforming elementary school in response to the NCLB school choice option. These students were served with existing bus routes. Meanwhile, officials in the Osseo school district attributed significant 2003-04 transportation expenses to the NCLB choice provisions. Specifically, the district is providing transportation to 21students who transferred from a school that did not make AYP for a second year. District officials estimated that the cost of adding transportation services for these students will be about $62,000 during the 2003-04 school year.

So far, Minnesota school districts have spent relatively little to transport students transferring from low-performing to higher-performing schools.

15 Magnet schools often have a unique curriculum focus, and they typically are open to students from throughout a district rather than just students from one part of the district.

16 If, in the absence of NCLB, a district would have given families school enrollment options and paid for their transportation, then it would not be reasonable to attribute these transportation costs to NCLB. Often, however, it may be diff icult to determine what options a district would have offered in the absence of NCLB's provisions. 17 For the 2003-04 school year, Minneapolis' letters were sent by the first day of school and parents had three weeks to request transfers. District staff acknowledged that some parents may have been reluctant to request transfers once their child had already started school in the fall. It is possible that some parents who chose not to transfer during the 2003-04 school year might decide to enroll their children in a different school at the beginning of a subsequent school year.

 

The Minnesota Department of Education plays only a small role role in administering NCLB's school choice provisions, and the department has borne few costs for these activities. The department's main activity has been to develop templates of letters that districts could send to parents regarding school choice, translated into several languages.

Supplemental Educational Services Supplemental educational services include tutoring and other academic enrichment services that are "high quality, research-based, and specifically designed to increase the academic achievement" of students. 18the Minnesota Department of Education determines which providers are authorized to provide these services. In November 2003, the department issued a list of 24 state-approved providers, including private agencies and school districts. 19

During the 2003-04 school year, three Minnesota school districts (Minneapolis, St. Paul, and Red Lake) were required to offer NCLB supplemental services. 20 Each of these districts had at least one school that had failed to make AYP for a third consecutive year. As of early February 2004, the Red Lake district had not yet entered into a contract with a supplemental services provider. 21 The other two districts sent letters to eligible parents, informing them of supplemental services options. 22 In some cases, the districts made other efforts to publicize supplemental services— such as parent information meetings in the affected schools. Once parents signed up for supplemental services, districts were required by NCLB to specify achievement goals for each child, based on consultation with the parents and service provider. 23 Some district staff said that implementing these tasks for even a limited number of schools was diff icult and time-consuming.

The number of students who enrolled in supplemental services in the 2003-04 school year was about 1,420 in Minneapolis and 90 in St. Paul. The total supplemental services enrollment in these districts represented less than 0.2 percent of all students in Minnesota public schools. In the Minneapolis and St. Paul districts, a large majority of the enrollees signed up for supplemental services offered by the respective school districts.

FISCAL IMPACTS 69 Currently, only three school districts in Minnesota are r equired to pr ovide supplemental services for students at low-performing schools.

18 No Child Left Behind Act, §1116( e)( 12)( C). 19 Once approved by the state, a supplemental services provider remains on the list of approved providers for three years. School districts may be approved as providers, but only if the district as a whole has not failed to make AYP for two consecutive years. Minnesota's list of state-approved providers included the Minneapolis and St. Paul school districts, 16 other Minnesota providers, and 6 providers based in other states.

20 The number of affected schools were ten in Minneapolis, three in St. Paul, and one in Red Lake. In addition, one Minnesota charter school was required to offer supplemental services.

21 The priv ate provider that the district initially intended to contract with went out of business. 22 In schools where supplemental services must be offered, students eligible for free and reduced-price lunches are eligible to receive supplemental services.

23 No Child Left Behind Act, §1116( e)( 3)( A).

For fiscal years 2004 and 2005, the Minnesota Department of Education estimated that it will incur about $52,000 in staff ing costs per year to fulfill its responsibilities to administer NCLB-required supplemental services. Although the department has not yet started to systematically monitor the performance of supplemental services providers, it does not anticipate that such monitoring will increase the department's annual costs above the $52,000 cited above. However, in our view, if a large percentage of districts are required by NCLB to implement supplemental services in future years, the department may need to consider dev oting additional resources to ensuring the quality and effectiveness of supplemental services, as required by NCLB. 24

Future District Costs As discussed earlier, we asked Minnesota school superintendents in a statewide survey to identify which requirement of NCLB will likely be the most costly for their districts to implement. The most common response— by 33 percent of superintendents— was "implementing sanctions and additional services for low-performing schools." 25 Thus, although the fiscal impact of these consequences has been very limited so far, school officials expect a larger impact in the years to come. In addition, our simulations in Chapter 3 suggest that:

A much larger number of schools will likely fail to make AYP in coming years, leading to significant growth in the cost of NCLB-related sanctions and services for low-performing schools. Table 4.5 shows the percentage of schools that our simulations indicate would be subject to NCLB sanctions under various scenarios. With modest improvement in Minnesota's student test scores, our simulations indicate that 42 percent of Minnesota's elementary schools with Title I services would be required to offer school choice for the 2008-09 school year. Furthermore, most of these schools would also be required to offer supplemental services, often in conjunction with corrective actions or restructuring. In 2014, following full implementation of NCLB's requirements, our simulations indicate that parents in 93 percent of Title I elementary schools would be offered school choice— again, assuming modest improvement in student test scores over time.

Even if Minnesota were to sustain a much larger increase in test scores over time, it is still likely that a large percentage of schools would face NCLB sanctions. As Table 4.5 shows, our simulations indicate that parents in 56 percent of Title I schools would be offered school choice in 2014 under the "high improvement" scenario.

At Title I funding levels for fiscal year 2005, Minnesota school districts would be required to spend as much as $20 million annually for NCLB-required school choice and supplemental services. This upper estimate assumes that (1) all Minnesota school districts would have at least one Title I school subject to NCLB sanctions, and (2) each of these districts would spend 20 percent of its Title I, Part A allocation on school choice and/ or supplemental services. It is worth reiterating that NCLB-required expenditures for school choice and supplemental services would not necessarily lead to increases in districts'overall levels of education expenditures. Some districts might decide to pay for these costs through reallocations of existing funds (Title I or other), while other districts might seek new revenues (for example, through levy increases). 26

70 NO CHILD LEFT BEHIND ACT As more schools fail to make "adequate yearly pr ogress," more schools will face NCLB sanctions.

24 Ibid., §1116( e)( 4)( D). 25 Office of the Legislative Auditor, Surve y of Sc hool District Superintendents (November-December 2003).

 

To some extent, school districts'future level of spending for school choice and supplemental services will depend on parents'level of interest in these options. On the one hand, perhaps parents will be reluctant to transfer their children from low-performing schools— due to (1) the attachment of parents or children to a school, (2) concern about the disruption that transfers might cause, (3) the lack of a guarantee in subsequent years that children who transfer to another school will receive district-provided transportation to that school, 27 or (4) differences in the specialized services av ailable at the schools to which the children could transfer. 28 On the other hand, it is likely that some parents will choose not to keep their children in a school identified as underperforming. In fact, officials in one Minnesota school district expressed concerns to us that a "mass exodus" might occur if a junior high school was required to offer school choice— because, they said, students and parents feel less allegiance to junior high schools than to elementary schools.

FISCAL IMPACTS 71 Statewide, NCLB may require Minnesota school districts to annually spend as much as $20 million to transfer students out of low-performing schools and to provide supplemental services.

Table 4.5: Estimates of the Percentage of Minnesota Title I Schools Offering School Choice and Supplemental Services, 2008 and 2014 Percentage of Schools Required to Provide School Choice and Supplemental Services No Modest High Improvement Improvement Improvement Scenario Scenario Scenario 2008 School Choice 47% 42% 28% School Choice and Supplemental Services 36 32 21 2014 School Choice 99 93 56 School Choice and

Supplemental Services 94 86 47

NOTE: The percentages exclude 22 Title I schools in 2008 and 21 Title I schools in 2014 that would have fewer than 20 tested students. SOURCE: Office of the Legislative Auditor analysis of Office of Educational Accountability's simulation of school performance. ( N= 735 for 2008 and N= 736 for 2014)

26 Districts that decide to use Title I revenues to pay for NCLB-required costs will bear an "opportunity cost" even if their overall district expenditures do not increase. That is, by using some Title I revenues to pay the cost of complying with NCLB sanctions, these districts will have fewer revenues av ailable to serve disadvantaged students in their regular Title I programs. 27 Assuming that a student transfers from a school failing to make AYP to a school making AYP, there is no guarantee that the original school will continue to fail to make AYP or that the destination school will continue to make AYP.

28 For example, students who transfer to a higher-performing school may find that Title I services are not available at that school.

 

Similarly, various factors may affect the extent to which parents enroll eligible children in supplemental educational services. For instance, enrollments may depend on the convenience of the service providers' tutoring times and locations, the types of information parents receive on the tutoring services, the reputation of the service providers, the interest of parents in the academic performance of their children, and the alignment of tutoring services with regular school curricula. Perhaps parents will be attracted to services that they perceive as individualized or innov ative. Also, no-cost, afterschool tutoring programs may appeal to some parents as a form of child care. On the other hand, many school districts in other states have experienced limited levels of enrollment in supplemental services, according to a sample of districts whose experience we reviewed. For example, in the current school year, New York City enrolled 50,000 students in supplemental services out of more than 212,000 eligible students; 29 Chicago enrolled 15,000 out of 133,000 eligible students; and Los Angeles enrolled 18,500 out of 186,000 eligible students. 30

Finally, the costs of NCLB-related sanctions for schools failing to make AYP for four or more years could be large, depending on the types of sanctions pursued. NCLB requires "corrective actions" for schools that have failed to make AYP for four years, and it requires schools that have failed to make AYP for five years to plan for implementation of "restructuring" one year later. The law outlines v arious options in these categories, but it gives state and local education agencies discretion regarding which specific actions to pursue. The Commissioner of Education has established a committee to further explore these options (including legislative changes that may be necessary), and the department will present recommendations to the 2005 Legislature.

Some of the options for corrective action or restructuring could be expensive, such as replacing a school's curriculum, replacing its staff, or extending the school year. Others might have relatively low costs, such as having school district officials assume responsibility for some decisions that were previously made by school officials. Again, the costs of implementing corrective actions and restructuring will depend considerably on the number of districts statewide that f ail to make AYP for at least four consecutive years. According to our simulations (described in Chapter 3), the percentage of Minnesota's Title I schools that would require corrective action or restructuring by 2014 ranges from 41 percent (assuming "high improvement" in student test scores) to 88 percent (assuming "no improvement" in student test scores). Thus, even our most optimistic estimate suggests that a substantial number of schools will be subject to NCLB's strictest sanctions. While it is not possible to precisely estimate the fiscal impact of these sanctions, the large-scale implementation of corrective actions and restructuring in Minnesota schools would, by its very nature, result in a high level of disruption in Minnesota's school system.

72 NO CHILD LEFT BEHIND ACT NCLB requires school districts to "restructure" persistently failing schools, which could be costly and disruptive.

29 New York's preliminary estimate was that 212,000 students were eligible, although district officials told us that this number will likely increase. In 2002-03, New York enrolled 30,000 out of 243,000 eligible students.

30 We collected information from two state education agencies (regarding all of their districts), plus we reviewed the experience of more than 20 individual districts in other states through phone contacts and reviews of published documents. The largest supplemental services enrollment we encountered was in Nashville, which enrolled 29 percent of eligible students according to a published account (Julian E. Barnes, "Off to a Slow Start," U .S. News & World Report , November 25, 2002, http://www.homeroom.com/press/article.asp?display= 36, accessed November 17, 2003.

 

COSTS RELATED TO TEACHER AND PARAPROFESSIONAL Q UALIFICATIONS

As outlined in Table 4.6, NCLB establishes minimum qualifications for all teachers of core academic subjects and for paraprofessionals working in a Title I program. 31 Each state receiving Title I, Part A funds must develop a plan to ensure that all teachers of core subjects are "highly qualified" by the end of the 2005-06 school year. 32 Districts failing to achieve the plan's objectives after two years must develop an improvement plan. After three years of failing to meet the objectives (and failing to make AYP), the act imposes sanctions on districts, such as limitations on the use of Title II, Part A funds (which districts use to improve teacher quality). 33 In addition, districts must ensure that all paraprofessionals working in a Title I program meet NCLB qualifications by January 2006. 34 To reach the goal of having all teachers and all Title I paraprofessionals meeting NCLB qualifications by 2006, the law also requires new teachers and paraprofessionals working in Title I programs to meet NCLB qualifications at the time of hiring. 35

The cost to districts of complying with these requirements depends, in large part, upon how many teachers and paraprofessionals already meet NCLB qualifications. At this time,

Most teachers in Minnesota are "highly qualified," as defined by NCLB. Meanwhile, it is unclear what proportion of Minnesota's paraprofessionals meet NCLB's qualification requirements. While the exact proportion of teachers who already meet the definition of "highly qualified" is unknown, it is quite likely that most teachers currently meet it. In January 2004, the Minnesota Department of Education, in collaboration with the Board of Teaching, released guidelines for determining which teachers in the state are "highly qualified." 36 All teachers of core subjects teaching in their field of licensure will meet the "highly qualified" definition because the state's licensure system already requires teachers to demonstrate subject matter competence through an academic major or a subject matter test. 37 In the 2002-03 school year, only about four percent of Minnesota teachers were not "highly qualified" because they were teaching a core subject outside their field of licensure. 38 At this time, however, the department has not assessed the impact of the guidelines on special education, English as a Second Language (ESL), and alternative learning center teachers. 39

FISCAL IMPACTS 73 NCLB establishes minimum qualifications for instructional staff.

31 NCLB defines the core academic subjects as English, reading or language arts, mathematics, science, foreign languages, civics and government, economics, arts, history, and geography . No Child Left Behind Act, §9101( 11).

32 Ibid.,§ 1119( a)( 2). 33 Ibid.,§ 2141. 34 Ibid., §1119( c)-( d). 35 New teachers working in a Title I program hired after the beginning of the 2002-2003 school year must be "highly qualified." Paraprofessionals hired after January 8, 2002 must meet NCLB qualifications. Ibid.,§ 1119( a)( 1), §1119( c); 34 C. F .R.§ 200.58 (2003).

36 Minnesota Department of Education, The Minnesota Plan for verifying "Highly Qualified" T eacher s( Roseville, MN: January 2004); and Minnesota Department of Education, Highly Qualified T eacher Criteria for Unique Instructional Settings (Roseville, MN: February 2004).

74 NO CHILD LEFT BEHIND ACT Table 4.6: NCLB Teacher and Paraprofessional Qualification Requirements "Highly Qualified" Teacher A "highly qualified" teacher a : Has full state certification (not an emergency, temporary, or provisional license), Holds at least a bachelor's degree, AND Has demonstrated subject matter competence.

Demonstration of subject matter competency by elementary teachers: New teachers must pass a state test of subject knowledge and teaching skills in reading, writing, mathematics, and other areas of the basic elementary school curriculum. Current teachers b can (1) meet the same standard as new teachers OR (2) meet the state's high, objective, uniform state standard of evaluation (HOUSSE). Demonstration of subject matter competency by middle and secondary level teach-ers:

New teachers must (1) pass a state subject matter test in each subject taught OR (2) have an academic major, graduate degree, coursework equivalent to a major, or advanced certification in each subject taught.

Current teachers can (1) meet either of the two options for new teachers OR (2) meet the state's high, objective, uniform state standard of evaluation (HOUSSE).

Qualified Paraprofessional A qualified paraprofessional working in a Title I program c : Has completed two years of study at an institution of higher education, Has obtained an associate's (or higher) degree, OR Has met a rigorous standard of quality and has demonstrated through a state or local assessment knowledge of reading, writing, and mathematics and instruction of these subjects.

a Only teachers of core academic subjects must be "highly qualified." Core academic subjects include English, reading or language arts, mathematics, science, foreign languages, civics and government, economics, arts, history, and geography.

b Current teachers are those who are not new to the profession. c Current Title I paraprofessionals have until January 8, 2006 to meet these requirements, but all Title I paraprofessionals, regardless of their hiring date, must have at least a high school degree. Paraprofessionals work in a Title I program if they (1) work in a school with a schoolwide Title I program, or (2) work in a school with a targeted assistance program and their salaries are funded with Title I funds.

SOURCE: No Child Left Behind, §§ 1119, 9101( 11), and 9101( 23).

There are several ways for instructional staff to meet NCLB's r equirements.

37 All teachers are required to have an academic major in their subject area, and new rules effective in 2001 require new teachers to pass a subject matter test. Minnesota Department of Education, Consolidated State Application: September 1, 2003 Submission (Roseville, MN, September 1, 2003), 17; and Minn. Rules (2003), ch. 8710.0500, subp. 1.

38 Minnesota Department of Education, unpublished document, "General Teacher Data, 2002-03," as submitted to the Senate Education Committee on February 5, 2004.

 

In our survey of superintendents, 64 percent reported that it is "likely" or "very likely" that all of the teachers in their districts would be "highly qualified" by 2005-06. However, superintendents from rural districts expressed more doubts than other superintendents about their ability to meet this goal. Sixty-one percent of superintendents from outstate districts with fewer than 2,000 students reported that it is "likely" or "very likely" that all of their teachers would be "highly qualified" by 2005-06; in contrast, 73 percent of superintendents from all other districts reported the same. 40

Although the department routinely collects data on the qualifications of teachers, statewide data on the qualifications of paraprofessionals are not available. In the future, the department will be required to report to the U. S. Department of Education regarding statewide compliance with paraprofessional requirements. 41 Based on interviews with staff from nine school districts, districts vary in their progress in determining whether current paraprofessionals meet NCLB qualifications. Some districts collected the necessary information through the hiring process or by surveying paraprofessionals specifically for this purpose, while others have yet to collect this information.

In complying with the teacher and paraprofessional qualification requirements, districts may incur costs for (1) determining which staff already meet the requirements, (2) monitoring staff progress in meeting the requirements, (3) planning and administering assessments or evaluations of staff competency , and (4) increasing compensation. When we examined districts'estimated costs of complying with the these requirements, we found that:

Estimates of the cost of complying with NCLB staff requirements varied widely among school districts— due to (1) still-evolving state policies, (2) differences in district compliance with NCLB requirements, and (3) differing assumptions about how to get into and remain in compliance with NCLB staff requirements. The Minnesota Department of Education only recently determined which methods teachers and paraprofessionals could use to meet NCLB requirements. As shown in Table 4.6, one way current teachers may demonstrate subject matter competence is to meet a "high, objective, uniform state standard of ev aluation" (HOUSSE). NCLB permits each state to design its HOUSSE standard. The department announced the final version of its HOUSSE standard in January 2004. In addition, the department finalized the options available to districts for helping paraprofessionals meet the NCLB qualifications in the fall of 2003. 42 At the time we solicited estimates from school districts regarding the costs associated with meeting NCLB's teacher and paraprofessional requirements, districts were not fully informed about which options would be av ailable to teachers and paraprofessionals and how burdensome they might be.

FISCAL IMPACTS 75 Most Minnesota teachers meet NCLB r equirements, although the exact impact of these r equirements is still being assessed.

39 Special education, ESL, and alternative learning center teachers may teach more than one core subject. If these teachers are the "teacher of record" for a core subject, they must be licensed in that subject area to be "highly qualified." If special education or ESL teachers are not the "teacher of record" for a core subject, they must have the appropriate special education license or an ESL license to be "highly qualified."

40Office of the Legislative Auditor, Surve y of Sc hool District Superintendents (November-December 2003).

41 The department stated in its September submission to the federal government that it would report on compliance with paraprofessional requirements by January 2004. However, the department has not yet collected this data from school districts. Minnesota Department of Education, Consolidated State Application: September 1, 2003 Submission , 28.

 

For the nine districts providing cost estimates, annual costs of complying with NCLB's teacher and paraprofessional requirements ranged from over $60 per pupil to less than $1 per pupil. Districts with low and high estimates differed in both the extent to which they were already in compliance with the requirements and the assumptions they made about what costs would be incurred. One district with a low estimate had already collected qualification data for existing paraprofessionals through the hiring process, and most paraprofessionals met the NCLB qualifications. This district assumed that salary increases for paraprofessionals would be unnecessary. In contrast, one district with a high cost estimate projected it would hire an additional staff person to monitor staff progress in meeting NCLB qualifications. This district also assumed that it would need to increase the wage rate for paraprofessionals in the next contract negotiation.

While districts vary in their estimates of the costs of complying with NCLB's teacher and paraprofessional requirements,

Local school district officials are more concerned about the cost of meeting NCLB's paraprofessional requirements than they are concerned about the cost of meeting teacher requirements. In our statewide survey , 26 percent of school district superintendents reported that the paraprofessional qualification requirements would be the most costly NCLB requirement to implement, while 11 percent said the teacher requirements would be the most costly. Furthermore, when asked about the need to increase average salaries, 39 percent of superintendents said that they have increased or are likely to increase average compensation levels to attract and retain qualified paraprofessionals as a direct result of NCLB. In contrast, 19 percent reported the same for "highly qualified" teachers. 43

Superintendents may be more concerned about paraprofessional requirements than the teacher requirements because most teachers are already "highly qualified." In addition, the requirements for ne wteachers working in a Title I program have had less impact on districts than the requirements for ne w paraprofessionals. (As we discussed earlier, in order to reach the goal of having all teachers and all Title I paraprofessionals meeting NCLB qualifications by 2006, NCLB also requires ne wteachers and paraprofessionals working in Title I programs to meet NCLB qualifications at the time of hiring.) According to officials with the Minnesota Department of Education, if districts have diff iculty hiring licensed teachers in core subjects, the shortage is generally at the middle or high school level, rather than at the elementary school level. Because most schools receiving Title I funds are elementary schools, the requirement that new teachers working in Title I programs be "highly qualified" has had little impact. In contrast, districts have hired paraprofessionals for Title I schools following the enactment of NCLB and have had to ensure that they meet the new qualifications.

76NO CHILD LEFT BEHIND ACT Complying with NCLB's staff qualification r equirements may be costly for some districts.

42 Districts may choose which options they will permit paraprofessionals to use. Paraprofessionals can pass the state-approved test (ParaPro) or demonstrate the Minnesota paraprofessional competencies through a training program. Districts may develop their own local assessments, but they must be approved by the Minnesota Department of Education. 43 Office of the Legislative Auditor, Surve y of Sc hool District Superintendents (November-December 2003).

 

NCLB requires not only that districts meet annual objectives in the state plan for increasing the number of "highly qualified" teachers, but also that districts meet annual objectives for increasing the percentage of teachers "who are receiving high-quality professional development." 44 (" Professional development" is training that licensed teachers receive as part of their employment by a school district.) As of late 2003, the Minnesota Department of Education had not collected data from districts needed to establish a baseline level of professional development and the future annual objectives. Costs of compliance with this requirement are diff icult to estimate because the extent to which districts will need to increase professional development opportunities is unclear.

In addition to meeting annual objectives for professional development, school districts receiving Title I, Part A funds must spend at least 5 percent of their Title I, Part A funds on professional development activities to ensure that all teachers are highly qualified by the end of the 2005-2006 school year. 45 NCLB also requires school districts and schools failing AYP for two consecutive years to spend at least 10 percent of their Title I, Part A funds on professional development. 46 However, these requirements might not impose significant new costs on school districts and schools because the previous version of the Elementary and Secondary Education Act (ESEA) required districts to provide professional development using Title I, Part A funds and established minimum expenditure levels for professional development for schools failing to make AYP for two consecutive years. 47 Furthermore, professional development activities provided to comply with the new NCLB minimum expenditure requirements may help school districts and schools achieve the annual objectives for increasing professional development opportunities.

For fiscal years 2004-06, the Minnesota Department of Education estimated that it would incur approximately $187,000 in staff ing costs per year to implement NCLB's teacher and paraprofessional qualifications provisions. These costs include the costs of (1) developing policies to implement the provisions, such as developing the HOUSSE standard for teachers and the test/ training options for paraprofessionals, and (2) monitoring districts'compliance with the requirements. 48

FISCAL IMPACTS 77 In addition, NCLB requires school districts to meet goals for increasing the percentage of teachers r eceiving "high-quality" pr ofessional development.

44No Child Left Behind Act, §1119( a)( 2)( B). 45Ibid.,§ 1119( l). Regulations clarify that a district is not required to spend 5 percent of Title I, Part A funds if a lesser amount is suff icient to ensure that the district's teachers and paraprofessionals meet NCLB qualification requirements. 34 C. F .R.§ 200.60( a)( 2).

46No Child Left Behind Act, §§ 1116( b)( 3)( A)( iii) and 1116( c)( 7)( A)( iii). A school district may count the 10 percent of funds that schools set aside for this purpose towards its 10 percent set-aside, but it may not count towards its 10 percent set-aside the 5 percent of funds that are set aside for professional development to ensure that teachers are "highly qualified." 47Improving America's Schools Act of 1994, Pub. L. No. 103-382, §1119.

 

COSTS RELATED TO CURRICULUM ALIGNMENT

NCLB requires each state to have challenging academic standards that specify what children are expected to know in reading, math, and science. 49 In addition, NCLB requires state education agencies and local school districts to help schools identify or develop "high-quality effective curricula aligned with [the state standards]." 50 If there is inadequate alignment between local curricula and the standards, students may have diff iculty demonstrating proficiency on the state's assessments in reading, math, and science. These assessments are supposed to measure each student's academic performance against state standards. 51 Consequently, without curriculum alignment, school districts and schools risk f ailing to achieve AYP, as defined under NCLB.

The content of Minnesota's academic standards has changed considerably in the last year. At the time NCLB was enacted, Minnesota already had academic standards outlined in the Profile of Learning, and the Minnesota Department of Education originally planned to use the Profile to comply with NCLB. However, the Profile of Learning standards applied to grade spans (K-3, 4-5, 6-8, and 9-12), which is contrary to the NCLB requirement of grade-specif icstandards. 52 Consequently, the department indicated in June 2002 that it would develop grade-level benchmarks within the Profile of Learning's grade-span standards in order to comply with NCLB. 53 However, this plan changed in 2003 when the Legislature repealed the Profile of Learning and replaced it with a new set of academic standards. 54

To develop these new academic standards, the Minnesota Department of Education estimated that it spent $96,000 last year and will spend $61,000 this year. The department's standard setting process has largely inv olved (1) coordinating and supporting citizen committees, which established the standards, (2) holding public meetings concerning draft standards, and (3) soliciting reactions from national experts. The reading and math standards were established last year, while the science standards will be finalized this year. 55 The department did not attribute any of these costs to NCLB.

78NO CHILD LEFT BEHIND ACT Under NCLB, school districts ar e supposed to align their curricula with state standards.

48The department also estimated that it will spend $160,000 per year to monitor or assist districts as they try to meet NCLB's professional development requirements. However, the department did not attribute any of these costs to NCLB.

49No Child Left Behind Act, §1111( b)( 1). 50Ibid., §§ 1111( b)( 8)( D) and 1112( c)( 1)( O). 51 Ibid., §1111( b)( 3)( A). 52 U. S. Department of Education, Standards and Assessments: Non-Re gulatory Draft Guidance (Washington, D. C., March 10, 2003), 2-6.

53 Minnesota Department of Children, Families and Learning, Consolidated Plan for the Elementary and Secondary Education Act (ESEA) (Roseville, MN, June 12, 2002), 5-11.

54Laws of Minnesota (2003), ch. 129, art 1, secs. 1-5.

 

We found that: Some school districts plan to devote considerable resources to bringing their curricula into alignment with the state's new academic standards, but it is debatable about how much, if any, of these costs should be attributed to NCLB.

The annual curriculum alignment estimates that we received from nine school districts ranged from over $100 per pupil to less than $1 per pupil, depending on the amount of alignment required in each district. On the high end, one district assumed that the state's new academic standards would require a substantial curriculum overhaul for the district— including large costs for staff training and the purchase of textbooks and instructional materials. According to staff from this district, the overhaul is needed because the Profile of Learning focused on broad concepts, while the new standards will require a teaching approach that drills students on specific facts. 56 On the low end, another district told us that its existing curriculum was mostly aligned with the new standards. This district's main curriculum alignment activity has been developing instructional calendars for each grade level and subject. These documents identify which of the state's standards are covered in each instructional unit, what activities will be carried out during each unit, and when during the school year each unit should be taught.

Even if some districts will incur significant curriculum alignment costs because of the new standards, it is debatable whether such costs should be attributed to NCLB. Prior to NCLB, there were serious legislative efforts to replace the Profile of Learning with different standards. 57 If Minnesota would have adopted new standards without NCLB, then most, if not all, local curriculum alignment costs should not be attributed to NCLB because districts would have had to carry out these activities anyway. This is the position of the Minnesota Department of Education. However, it is impossible to prove what the state would have done without NCLB. In fact, some school district curriculum directors expressed the opinion that NCLB played a direct role in the adoption of the new standards. Specifically, they contended that the Minnesota Department of Education told people that the state needed to adopt the new standards to comply with NCLB. Furthermore, it is possible that the high stakes nature of NCLB made curriculum alignment more important than it would have otherwise been, thus leading districts to dev ote more resources to it. In our November 2003 survey , 91 percent of superintendents said that their district revised (or will likely revise in the next two years) classroom curricula as a dir ect r esult of NCLB. 58

FISCAL IMPACTS 79 Minnesota adopted new academic standards in r eading and math in 2003.

55To carry out its original plan of developing grade-specific benchmarks under the Profile of Learning, the Minnesota Department of Education estimated that it spent $82,000 in state fiscal year 2002 and $87,000 in state fiscal year 2003.

56In addition, according to the superintendent, the district has added advanced courses in junior high science and math so that high-performing students would not be held back by the "skill and drill" approach required for the regular students.

57Patricia G. Avery, Richard Beach, and Jodiann Coler, The Impact of Minnesota's "Pr of ile of Learning" on T eaching and Learning in English and Social Studies Classrooms( Minneapolis: Department of Curriculum and Instruction at the University of Minnesota's College of Education and Human Development, April 30, 2002). This report provides a history of the Profile of Learning up to April 2002.

 

To the extent that curriculum alignment costs are attributed to NCLB, there is an argument that they should largely be considered one-time, rather than ongoing costs. State law already requires districts to periodically review and improve their instruction and curriculum. 59 The adoption of the state's new academic standards has caused many school districts to alter their curriculum review cycles and implement new curricula earlier than they otherwise would have. However, under this argument, once the new curricula are fully in place and school districts go back to their traditional review process, the curriculum review costs of districts should generally notbe attributed to the new standards. Yet some districts may f ace additional alignment costs that are ongoing. While Minnesota law requires the Commissioner of Education to establish a system for reviewing the new academic content standards every four years, 60 some school districts have traditionally reviewed their curricula on a longer cycle, such as seven years. If the state standards are significantly changed every four years, some districts will be required to adjust their curricula more often than they otherwise would have.

OTHER COSTS Besides the major cost categories discussed above, we also examined some other potential costs— specifically, costs associated with (1) expanding school buildings at higher performing schools to accommodate school choice, and (2) NCLB-related lawsuits.

Capital Costs In our review of NCLB's potential fiscal impacts, we focused on operating costs, not capital expenditures. However, it is important to note that some school districts might have to undertake capital projects to comply with NCLB requirements.

When a school has failed to make AYP for at least two consecutive years, NCLB requires the school district to inform parents in the low-performing school about transfer options to higher-performing schools. According to the U. S. Department of Education's guidance on this issue,

The bottom line, then, is that every student enrolled in a Title I school [that has failed to make AYP for two years] who wishes to transfer to a school that is not in need of improvement must have that opportunity. Thus, if [a school district] does not have suff icient capacity in the schools it has offered under its choice plan to accommodate the demand for transfers, the [school district] must create additional capacity or provide choices of other schools. 61

80 NO CHILD LEFT BEHIND ACT The Minnesota Department of Education believes that curriculum-alignment costs should not be attributed to NCLB, but many school district officials disagree.

To accommodate student transfers, high performing schools may have to add building space.

58Office of the Legislative Auditor, Surve y of Sc hool District Superintendents (No vember -December 2003) .

59Minn. Stat . (2002), §120B. 11. 60Laws of Minnesota (2003), ch. 129, art. 1, sec. 5.

 

If a large number of schools in Minnesota fail to make AYP, as the simulations in Chapter 3 suggest, there may be many parents who are offered the option to transfer their children to a limited number of high-performing schools. The schools that receive these transferring students could find that they have to expand school facilities, perhaps at significant cost. However, it is worth noting that most of the new federal funding provided through the NCLB Act cannot be used for school construction costs.

Lawsuits Legislative leaders in various states have expressed concern about the potential for NCLB-related lawsuits against states. NCLB requires that every child be proficient by 2013-14. Recently, Officers of the National Conference of State Legislatures said that such requirements could be the basis for court judgments regarding the adequacy of education funding or the need for school finance system reforms. 62

Our simulations of school performance (Chapter 3) show that it will be very diff icult for the state, school districts, and individual schools to meet NCLB's ambitious expectations. Still, we have no basis for judging whether lawsuits are possible or what their outcomes might be. In addition, as we discuss in Chapter 3, there is considerable debate in education literature regarding whether— or how much— additional spending would be required to achieve significant improvements in student achievement.

WILL NCLB'S NEW REVENUES COVER ITS NEW COSTS?

Some Minnesota legislators have questioned whether the federal government will provide Minnesota with a suff icient increase in funding to cover the additional costs imposed by the act. In addition, many local officials expressed concerns to us that NCLB is an unfunded (or insuff iciently funded) federal mandate. For example, in our statewide survey , less than 3 percent of school district superintendents said that they think that new federal revenues for their districts will be suff icient to cover the cost of new spending required by NCLB. 63

Our analysis of this issue examined costs and funding levels in the long run, when NCLB's requirements are fully in place. We used Minnesota's NCLB funding level for state fiscal year 2005 as an estimate of future funding levels. We focused on the costs of new NCLB requirements after they are fully implemented. Furthermore, we adjusted both the funding and cost figures for inflation to reflect 2004 prices.

FISCAL IMPACTS 81 Some legislators ar e concerned that states will be sued if students fail to meet NCLB's goals.

61 U. S. Department of Education, PublicSchoolChoice: DraftNon-RegulatoryGuidance (Washington, D. C., December 4, 2002), E-7.

62 Senator Angela Monson (Oklahoma), NCSL President, and Speaker Martin Stephens (Utah), NCSL President-Elect, memorandum to state legislative presiding Officers, chairs of education committees, and legislative education staff, July 7, 2003.

63 Office of the Legislative Auditor, Surve y of Sc hool District Superintendents (November-December 2003).

 

Table 4.7 shows the increase in federal education funding that Minnesota is expected to receive since NCLB became law. Minnesota is expected to receive $42 million more from the entire Elementary and Secondary Education Act (ESEA) in state fiscal year 2005 than it did in the pre-NCLB baseline year of state f iscal year 2002, which includes an increase of $3 million in Title I, Part A funding. To determine whether new funding will be sufficient to cover the cost of new Title I, Part A activities, we think that it is generally appropriate to focus on the $42 million ESEA increase rather than the $3 million Title I, Part A increase. First, Minnesota has received increased funding from portions of the ESEA other than Title I, Part A explicitly for the purpose of covering Title I, Part A costs. For example— beyond the funding provided through Title I, Part A— Minnesota received $7 million for student assessments and $8 million for improving teacher quality. Second, as noted in Chapter 1, the federal government has granted states and districts the authority to transfer much of their new funding from sections of NCLB other than Title I, Part A to their Title I, Part A programs. Finally, the federal government provides funding for all ESEA programs largely to support the efforts of school districts and schools under Title I, Part A. For example, the purpose of the Reading First program (Title I, Part B) is to help ensure that every student is reading at or above grade level by the end of the third grade, which will help ensure that all students are proficient by the 2013-14 school year. 64 In general, the administrative requirements in these other portions of NCLB are relatively minor compared with those in Title I, Part A.

82 NO CHILD LEFT BEHIND ACT In state fiscal y ear 2005, Minnesota will receive $42 million more in federal funding under NCLB than under the pr evious Elementary and Secondary Education Act.

Table 4.7: New ESEA Formula Funding State Fiscal Year 2002-05 Funding Category Increase in Funding (Millions of $)

Title I, Part A $ 3 Assessments a 7 Improving teacher quality b 8 Other ESEA funds 25

Total ESEA funds $42

NOTE: The 2005 funding levels are preliminary estimates from the U. S. Department of Education. In calculating the increase, the 2002 and 2005 funding levels were adjusted for inflation to reflect 2004 prices.

a As specified in federal law, the Minnesota Department of Eductation uses these funds for its own assessment development and administration costs.

b Prior to NCLB, these funds were provided under two programs— the Eisenhower Professional Development program and the Class Size Reduction program.

SOURCE: Compiled by Office of the Legislative Auditor from U. S. Department of Education, "Fiscal Year 2001-2005 State Tables for the U. S. Department of Education," http://www.ed.gov/about/overview/budget/statetables/index.html?src=rt, accessed February 16, 2004.

64U. S. Department of Education, Office of Elementary and Secondary Education, No Child Left Behind: A Desktop Refer ence , 2002 (Washington, D. C., 2002), 23.

 

Table 4.8 summarizes our discussion of NCLB costs from earlier in this chapter. The eventual costs of some of the major NCLB requirements are difficult to accurately estimate, or it is debatable how many of these costs should be attributed to NCLB. For instance, statewide, school districts could eventually be required by NCLB to spend as much as $20 million annually for school choice-related transportation and supplemental educational services. For

purposes of comparing new revenues and expenditures, it is debatable how much of these costs should be considered "new." Some schools may use Title I funds to pay for supplemental services; others may spend revenues from other sources to pay for these services. If schools redirect Title I funding to supplemental services, these expenditures could be viewed as an alternative approach to serving disadvantaged students, rather than a "new cost." On the other hand, any expenditures incurred to meet new NCLB requirements might be regarded as "new costs" to the extent that they require districts to spend funds on services they would not otherwise purchase.

In addition, school districts face the general challenge of ensuring that 100 percent of their students achieve proficiency by 2014. In Chapter 3, we questioned whether academic research has identified ways to accomplish such an ambitious goal. Even if existing research has identified excellent models for schools to consider, it is doubtful that schools could implement these approaches with consistently excellent results, and there may be some types of students that even the best programs cannot help succeed within NCLB's timelines. Furthermore, we noted that there are debates about how much additional funding would be necessary to dramatically increase student achievement.

FISCAL IMPACTS 83 Table 4.8: New Title I, Part A Costs New Long-Term Annual Cost Category Costs (Millions of $)

School choice and supplemental services (maximum cost) a $20 State implementation of assessments b 8 District/ school implementation of assessments (mid-range estimate) c 11

Curriculum alignment Debatable Costs Corrective action and restructuring for failing schools Unknown Teacher and paraprofessional qualifications Unknown Making all students proficient by 2014 Unknown

Total Title I, Part A costs Unknown a Assumes that all school districts that receive Title I, Part A funding will have to devote a full 20 percent of their allocation to choice-related transportation and supplemental services.

b These assessments include the reading and math assessments in grades 4, 6, and 8, the science assessments in three grade spans, and the listening and speaking assessments for limited-English students.

c These assessments include the reading and math assessments in grades 4, 6, and 8, the science assessments in three grade spans, and the listening and speaking assessments for limited-English students. The districts provided us with estimates that ranged from $4 to $50 per student taking each assessment. For each of the reading, math, and science assessments, we used the median estimate of $17 per student taking the assessment. This cost figure does not include any potential cost savings that school districts could achieve by discontinuing some of their local, non-NCLB assessments. SOURCE: Office of the Legislative Auditor.

 

As states continue implementing NCLB and incurring additional costs, it is possible that Congress may increase NCLB funding rather than hold it at current levels. However, Minnesota's NCLB allocation is expected to decline in state f iscal year 2005, and under President Bush's proposed budget for the following year, Minnesota's allocation is expected to decline even more. 65

Overall, we conclude that: In the future, it is quite plausible that the cost of NCLB's new requirements for Minnesota could exceed the increase in federal funding that the state receives under the act, but this will be unclear until school districts proceed further with implementing the act and the federal government determines future funding levels. Finally, it is worth noting NCLB's own language regarding state and local fiscal obligations:

Nothing in this Act shall be construed to authorize an Officer or employee of the Federal Government to… mandate a State or any subdivision thereof to spend any funds or incur any costs not paid for by this Act. 66

A legal analysis commissioned by the National Conference of State Legislatures concluded that the "plain meaning" of this language is that NCLB does not require states and school districts to spend more on NCLB than the act itself provides. 67 However, it is unclear how this provision might be interpreted and enforced by the U. S. Department of Education and, potentially, the courts.

SCHOOL DISTRICT RESPONSES TO NCLB COSTS

Although many school districts have received substantial increases in their NCLB/ ESEA funding and are only now starting to face most of the new NCLB costs,

84 NO CHILD LEFT BEHIND ACT According to NCLB, states are not required to spend more to carry out the act than the act pr ovides in funding.

65U. S. Department of Education, "Fiscal Year 2001-2005 State Tables for the U. S. Department of Education," http://www.ed.gov/about/overview/budget/statetables/index. html?src=rt, accessed February 16, 2004.

66No Child Left Behind Act, §9527( a). 67Senator Angela Monson (Oklahoma), NCSL President, and Speaker Martin Stephens (Utah), NCSL President-Elect, memorandum to state legislative presiding Officers, chairs of education committees, and legislative education staff, July 7, 2003.

School district superintendents reported in our statewide survey that they have primarily paid for NCLB-required activities through reallocations of existing funds. As shown in Table 4.9, 72 percent of superintendents said that their districts have paid for new, NCLB-required activities in the past two years primarily through spending reductions or reallocations, rather than through new revenues. Likewise, 73 percent of superintendents anticipate that they will pay for NCLB-related activities during the next two years primarily through spending reductions or reallocations. It is possible that school districts will allocate most of their new NCLB funding to direct services for students— to improve student performance— and pay for the NCLB-specific requirements, such as test administration, by reallocating existing resources.

Table 4.10 shows specific changes that superintendents said their districts made (or expect to make) as a direct result of NCLB. Seventy-nine percent of superintendents said that NCLB caused their districts to revise classroom curricula in the last two years. Also, the table shows that a majority said that NCLB caused them to revise the jobs of their existing instructional and administrative staff. Much smaller percentages of superintendents said that they have hired additional staff or increased compensation levels in response to NCLB.

"OPTING OUT" OF NCLB Some legislators have questioned whether Minnesota should "opt out" of the Title I, Part A program— that is, forgo NCLB federal funding and refuse to comply with the law's requirements. Some think that the federal government should not be so inv olved in state education issues, while others believe that the law's requirements are onerous. While legislators might consider a variety of issues when making a decision about opting out of NCLB, we were asked to assess the financial implications. Assessing the financial ramifications of opting out is different than comparing the increase in federal funding under NCLB with the cost of the act's new requirements, which we did earlier in this chapter. If Minnesota opted out, it could lose most of its NCLB/ ESEA funding, not just the increase the state has received under the act. Also, Minnesota would have the option of averting all NCLB/ ESEA requirements, not just the new ones created in this most recent reauthorization of the ESEA. In the following sections, we discuss these issues in more detail.

FISCAL IMPACTS 85 Table 4.9: Ways That School Districts Have Funded (or Intend to Fund) New Requirements of the NCLB Act Percentage of Superintendents Who Identified This As Their Primary Funding Method:

In the Past In the Next Funding Method Two Years Two Years

Spending reductions or reallocations 72% 73% Increases in district's federal revenues 7 4 Increases in district's state revenues <1 1 Increases in district's local revenues 6 6 Other 9 6 Don't know/ didn't respond 6 10

Total 100% 100%

SOURCE: Office of the Legislative Auditor survey of school superintendents, November-December 2003. ( N= 326)

Despite receiving an increase in federal funding, few school district superintendents in Minnesota identified federal funds as their primary means of financing NCLB costs.

 

Loss of Federal Funding In state fiscal year 2005, Minnesota is expected to receive $104 million in Title I, Part A funding and $216 million in overall NCLB funding. (These funding levels have been adjusted for inflation to reflect prices in state fiscal year 2004.) Based on interviews with federal and state officials and technical guidance provided by the U. S. Department of Education, we found that:

Minnesota would lose the majority of its federal NCLB funding if it opted out of Title I, Part A, but the exact amount is unclear.

86NO CHILD LEFT BEHIND ACT Table 4.10: Changes that School Districts Made (or Will Likely Make) As a Direct Result of NCLB Percentage of Superintendents Who Said That Their District: Made This Change Will Likely Make This In the Past Two Years Change in the Next Two Years District Action As a Direct Result of NCLB As a Direct Result of NCLB Revised classroom curricula 79% 87% Reassigned (or redefined the jobs of) existing instructional staff 65 70 Reassigned (or redefined the jobs of) existing administrative staff 61 56

Discontinued some standardized assessments not required by NCLB 44 49

Increased average compensation levels to retain/ attract NCLB-qualified paraprofessionals 17 36

Hired additional instructional staff 29 29 Increased average compensation levels to retain/ attract "highly qualified" teachers 9 16

Hired additional administrative staff 6 8 SOURCE: Office of the Legislative Auditor survey of school superintendents, November-December 2003. ( N= 326)

Some legislators have asked whether Minnesota should "opt out" of NCLB.

On February 6, 2004, the U. S. Department of Education issued an informal statement concerning the federal funds that states would lose if they opted out of Title I, Part A. 68 According to the department, programs with grants that are allocated to states and school districts based on their share of Title I, Part A funding would be "negatively affected." The department specifically identified six such programs—( 1) Even Start, (2) Comprehensive School Reform, (3) Education Technology Grants, (4) Safe and Drug Free Schools and Communities, (5) 21st Century Community Learning Centers, and (6) Education of Homeless Children and Youth. In Minnesota, the portion of the funding for these six programs that is linked to the state's Title I, Part A allocation will total $22 million in state fiscal year 2005. Thus, if Minnesota opted out of Title I, Part A, the state would annually lose at least $126 million—$ 104 million of Title I, Part A funding and another $22 million from the programs listed above.

However, this is not an exhaustive list of the funding that could be lost. The U. S. Department Education's statement left open the possibility of other programs being affected, and our review of the NCLB law indicates that several other NCLB programs appear to be linked to a state's participation in Title I, Part A. For example, the federal government annually provides Minnesota about $7 million to develop and administer the new assessments required in Title I, Part A. If Minnesota opted out and did not implement these assessments, it seems unlikely the state would receive this funding. As another example, a few non-Title I, Part A programs require schools or school districts to be held accountable for making AYP. 69 If Minnesota opts out and does not establish an AYP process as defined in Title I, Part A, funding in these programs may be jeopardized. In the end, the amount of funding that Minnesota would lose if it opted out of Title I, Part A depends on how strictly the U. S. Department of Education (and possibly the courts) interprets these linkages between Title I, Part A and other parts of NCLB. Although it is doubtful that Minnesota would lose its entire NCLB funding ($ 216 million) if it opted out of Title I, Part A, it is clear that the state would lose a large percentage of these funds.

The department's statement also discusses whether states will lose federal funding outside of NCLB if they opt out. The U. S. Department of Education wrote:

Nonparticipation . . . in programs under [NLCB] does not disqualify [a state] from receiving funds under the Carl D. Perkins Vocational and Technical Education Act, the Adult Education and Family Literacy Act, and the Individuals with Disabilities Education Act (IDEA). 70

However, it is important to note that the department provided this statement as technical assistance and not as a formal legal opinion. The department qualified the information by stating, "Each program, within and outside our Department, must be reviewed to determine whether any of the respective program's requirements are linked to or otherwise reference requirements of the ESEA." 71

FISCAL IMPACTS 87 The federal government has provided only limited guidance regarding the amount of funding states would lose if they opted out of NCLB.

68Eugene W. Hickok, Acting Deputy Secretary, U. S. Department of Education, letter to Dr. Steven O. Laing, Superintendent of Public Instruction, Utah State Office of Education, February 6, 2004.

69These programs include (1) English Language Acquisition (Title III, Part A), (2) Small, Rural School Achievement (Title VI, Part B, Subpart 1), and (3) Rural and Low-Income Schools (Title VI, Part B, Subpart 2). Other programs also have linkages to Title I, Part A. For example, the

Improving Teacher Quality grant program (Title II, Part A) has several requirements that are intertwined with those in Title I, Part A.

70Eugene W. Hickok, Acting Deputy Secretary, U, S. Department of Education, letter to Dr. Steven O. Laing, Superintendent of Public Instruction, Utah State Office of Education, February 6, 2004.

 

Cost Savings We examined whether Minnesota could achieve annual cost savings by opting out that would offset the loss of federal funding discussed above. If Minnesota opted out, it would have the option of discontinuing all the activities required in Title I, Part A. Yet, it is quite possible that the state would still carry out many of these activities on its own. In fact, Minnesota now requires some of the NCLB activities in state law. Thus, to consider cost savings that might result from opting out, policy makers should contemplate what type of statewide educational accountability system Minnesota would have without NCLB. In the following paragraphs, we discuss possible cost savings in more detail.

General NCLB/ ESEA Administration. If Minnesota did not participate in NCLB, the Minnesota Department of Education and school districts would not have to carry out general NCLB/ ESEA administrative activities, such as managing grant funds and preparing plans and reports. The department reported to us that it annually spends $2.5 million to carry out these general administrative activities. In addition, we estimated that school districts across the state annually spend about $7 million on their administrative activities. 72

Development of Academic Standards and Curriculum Alignment. By opting out, the state would likely achieve little, if any , cost savings in this area. Specifically, the state would probably not abandon statewide academic standards if it opted out of NCLB. Minnesota was moving toward implementing statewide standards prior to NCLB (and the federal law that preceded it), and state and local officials are now implementing academic standards adopted by the 2003 Legislature. However, without the accountability and sanctions imposed by NCLB, school districts might focus fewer resources on aligning curricula and measuring of proficiency against state standards.

Student Assessments . If the state reverted to its pre-NCLB assessment requirements, it would drop (1) the reading and math assessments in grades 4, 6, and 8, (2) the science assessments, and (3) the listening and speaking assessments for limited-English students. This would potentially save the Minnesota Department of Education and school districts roughly $19 million annually, in the long run. In the event that the Legislature decided to abandon all the other assessments required by NCLB, the department and districts would save a total of about $39 million annually. However, to the extent that school districts would have used these NCLB assessments to gather information about their students' academic skills and achievement levels, some districts may decide to start administering other assessments on their own to gather similar information. The cost of these new, non-NCLB assessments would offset some of the savings achieved by opting out.

88NO CHILD LEFT BEHIND ACT Minnesota would likely continue to bear the cost of some accountability activities even if the state opted out of NCLB.

71 Ibid. 72 We based our estimate on the median estimate that we received from eight school districts. (Minneapolis did not provide an estimate for the total general administrative costs it will incur.) The median estimate was 6 percent of a district's Title I, Part A allocation, and we multiplied this by the statewide district allocation for Title I, Part A in state fiscal year 2004. While the cost of general ESEA administration applied to all programs (not just Title I, Part A) we used the Title I, Part A allocation as the basis of our extrapolation.

 

AYP Determination and Reporting. Minnesota could av oid the costs associated with the AYP determination and reporting process prescribed by NCLB if it opted out, but these costs are relatively small statewide. Furthermore, if the Legislature decided to keep some sort of statewide accountability system in the absence of NCLB, state and local officials would still need to carry out some performance reporting activities.

Sanctions and Supplemental Services . Without NCLB, the state would not be required to impose consequences against under-performing schools and school districts. School districts would no longer be required to spend an amount up to 20 percent of their Title I, Part A allocations on supplemental services and school choice-related transportation— an amount up to $20 million statewide. In addition, they would not have to carry out corrective actions and restructuring for schools that fail to make AYP for at least four consecutive years.

Teacher and Paraprofessionals Requirements. Statewide, there could be significant savings if school districts did not have to comply with NCLB requirements, although the exact magnitude of the savings is unclear. Nine school districts estimated that their savings per K-12 pupil would range from less than $1 to over $60 if teachers and paraprofessionals only had to meet qualifications specified by the state and school districts. In addition, school districts would not have to meet the NCLB goals for increasing the amount of high quality professional development that teachers receive.

Pa r ental Inv olvement. If Minnesota opted out of NCLB, Minnesota would not have to provide the parental inv olvement activities required by the Title I, Part A program. Based on information provided by nine school districts, we estimated that school districts annually spend about $2 million statewide on these activities. 73

Overall. The potential cost savings from opting out of NCLB might be somewhat larger than the cost of the new NCLB requirements outlined in Table 4.8. But,

Without knowing the size of NCLB's least predictable costs, it is unclear whether Minnesota's annual cost savings from opting out of Title I, Part A would exceed the loss of federal funding. Without question, Minnesota could avert some costs if the state opted out of NCLB. While it is possible to make reasonable estimates of some of these costs, it is diff icult to accurately estimate how much Minnesota would save if school districts did not have to meet NCLB's requirements for staff qualifications and performance-related sanctions. It is even more diff icult to estimate what costs might be saved if Minnesota was not required to ensure that all students are proficient by 2014. However, NCLB funding accounts for less than 4 percent of school district budgets, so it would not take a large percentage increase in education spending for NCLB costs to exceed NCLB revenues.

FISCAL IMPACTS 89 A majority of school district superintendents oppose opting out of NCLB.

73 The median estimate was 2 percent of a district's Title I, Part A allocation, and we multiplied this by the statewide district allocation for Title I, Part A in state fiscal year 2004. No Child Left Behind Act §1118( a)( 3)( A) requires districts to set aside not less than 1 percent of their Title I,

Part A funds for parental involvement activities.

 

Although it is unclear whether the cost savings that Minnesota would achieve by opting out of Title I, Part A would be sufficient to offset the loss of funding, most school district superintendents do not want the state to opt out. In our statewide survey of superintendents, we found that:

Seventeen percent of Minnesota's superintendents said that they favored Minnesota "opting out" of NCLB, while 51 percent opposed it and 31 percent offered no opinion. 74

TRACKING NCLB COSTS IN THE FUTURE To make effective decisions about Minnesota's K-12 education system, legislators need a good understanding of the costs that NCLB is imposing on the state. The act will have a large impact on the way schools and school districts operate. In addition, the Legislature may want to continue examining the option of opting out of NCLB, or it may wish to challenge the federal government under the provision of the act that says that states are not required to spend more on NCLB activities than the act provides in funding. 75 However, the exact magnitude of many of the NCLB costs will only become clear after Minnesota is further along in implementing the act.

RECOMMENDATION The Legislature should require the Minnesota Department of Education to annually prepare a report identifying, at a minimum, the expenditures incurred by school districts (regardless of funding source) to comply with NCLB-required sanctions for low-performing schools.

This report does not recommend that the Minnesota Department of Education track all NCLB-related state and local costs on an ongoing basis. Determining which costs to attribute to NCLB requires many judgments, and we are skeptical that such judgments could be made consistently for costs incurred by numerous education agencies in many separate cost categories. However, we think that some potentially large, readily-defined NCLB costs should be tracked regularly. As a starting point, we suggest that the department collect and summarize data regarding local expenditures for NCLB-related school choice, supplemental education services, corrective actions, and restructuring. For example, such a report could indicate whether school districts are spending the full amounts they are required to set aside for school choice and supplemental services— or less, due to low demand for these services. In addition, such reports could indicate the nature and cost of the corrective actions and restructuring sanctions that some districts will be required to implement in the future.

90 NO CHILD LEFT BEHIND ACT The Minnesota Department of Education should track NCLB-related costs.

74Office of the Legislative Auditor, Surve y of Sc hool District Superintendents (November-December 2003).

75No Child Left Behind Act, §9527( a).

List of Recommendations In cases where the Minnesota Department of Education determines that an AYP determination was made in error, it should ensure that the error does not adversely affect the school's or school district's sanction status in subsequent years (p. 50).

The Minnesota Department of Education should provide the 2005 Legislature with a plan that outlines how value-added measures of student achievement could be incorporated into the annual AYP determination process (p. 51).

The Minnesota Department of Education and the Office of Educational Accountability should report to the 2005 Legislature on any unresolved issues regarding the validity and reliability of Minnesota's education accountability system (p. 54).

The 2004 Legislature should require the Minnesota Department of Education to submit a plan to the House and Senate education committees that outlines how it will monitor the quality and effectiveness of supplemental educational services providers (p. 56).

The Legislature should require the Minnesota Department of Education to annually prepare a report identifying, at a minimum, the expenditures incurred by school districts (regardless of funding source) to comply with NCLB-required sanctions for low-performing schools (p. 90).

Further Reading General Borman, Geoffrey D. "Title I: The Evolving Research Base," J ournal of Education for Students Placed at Risk 5 (2000), 27-45.

Center on Education Policy. F r om the Capital to the Classroom: State and F eder al Efforts to Implement the No Child Left Behind Act . Washington, D. C., January 2003.

From the Capital to the Classroom: Y ear 2 of the No Child Left Behind Act. Washington, D. C., January 2004.

Congressional Research Service. Elementary and Secondary Education: Reconsider ation of the F eder al Role by the 107 th Congress. Washington, D. C., December 31, 2001.

Davenport, Ernest C., Mark Davison, and Yi-Chen Wu, Adequate Y early Pro g r ess Simulation: Final Report . Minneapolis: Office of Educational Accountability, University of Minnesota, January 26, 2004.

Education Commission of the States. No Child Left Behind: The Challeng es and Opportunities of ESEA 2001 . Denver, CO, March 2002.

Erpenbach, William J., Ellen Forte-Fast, and Abigail Potts. Statewide Educational Accountability Under NCLB. Washington, D. C.: Council of Chief State School Officers, July 2003.

Grissmer, David, Ann Flanagan, Jennifer Kawata, and Stephanie Williamson. Impr o ving Student Achie vement: What State NAEP Test Scores Tell Us. Santa Monica, CA: RAND, 2000.

Hill, Richard K. and Charles A. DePascale. Reliability of No Child Left Behind Accountability Designs. Dover, NH: National Center for the Improvement of Educational Assessment, February 7, 2003.

Linn, Robert L., Eva L. Baker, and Damian W. Betebenner. "Accountability Systems: Implications of Requirements of the No Child Left Behind Act of 2001," Educational Researc her31, no. 6 (August/ September 2002), 3-16.

Marion, Scott, Carole White, Dale Carlson, William J. Erpenbach, Stanley Rabinowitz, and Jan Sheinker. Making Valid and Reliable Decisions in Determining Adequate Yearly Progress. Washington, D. C.: Council of Chief State School Officers, December 2002.

Minnesota Department of Children, Families, and Learning. Consolidated Plan for the Elementary and Secondary Education Act (ESEA) . Roseville, MN, June 12, 2002.

Minnesota Department of Education. Consolidated State Application Accountability W orkbook. Roseville, MN, January 31, 2003.

Consolidated State Application: September 1, 2003 Submission. Roseville, MN, September 1, 2003.

Minnesota House of Representatives, Research Department. Federal and State Testing Requirements for K-12 Public School Students . St. Paul, November 2002.

Schwarz, Richard D., Wendy M. Yen, and William D. Schafer. "The Challenge and Attainability of Goals for Adequate Yearly Progress," Educational Measurement: IssuesandPractice 20, no. 3 (Fall 2001), 26-32.

U. S. Department of Education. Improving Teacher Quality State Grants, TitleII, P art A: Non-Regulatory Guidance . Washington, D. C., January 16, 2004.

LEA and School Impro vement: Non-Re gulatory Guidance . Washington, D. C., January 7, 2004.

No Child Left Behind: A Desktop Refer ence 2002 . Washington, D. C.: Office of Elementary and Secondary Education, September 2002.

Public School Choice: Draft Non-Regulatory Guidance . Washington, D. C., December 4, 2002.

Report Cards, T itle I, P art A: Non-Regulatory Guidance . Washington, D. C., September 12, 2003.

Standards and Assessments: Non-Re gulatory Draft Guidance . Washington, D. C., March 10, 2003.

Supplemental Educational Services: Non-Regulatory Guidance . Washington, D. C., August 22, 2003.

NCLB Costs AccountabilityWorks, prepared for The Josiah Bartlett Center for Public Policy. Financial Impact of the No Child Left Behind Act on the State of New Hampshire & Review of the Cost Analysis of the NHSAA. Concord, NH, February 2003.

Driscoll, William and Howard Fleeter. Pr ojected Costs of Implementing the Federal "No Child Left Behind Act" in Ohio: A Detailed F inancial Analysis Prepared for the Ohio Department of Education . Columbus, OH: Levin, Driscoll, and Fleeter, December 12, 2003.

Mathis, William J. The Federal "No Child Left Behind" Law: Should vermont Take the Money? Vermont Society for the Study of Education, October 22, 2002.

"No Child Left Behind: Costs and Benefits," Phi Delta Kappan 84, no. 9 (May 2003), 679-686.

National Association of the State Boards of Education (NASBE). "Cost of President's Testing Mandate Estimated as High as $7 Billion," http://www.nasbe.org/archives/cost.html.

National Center on Education Finance, National Conference of State Legislatures. " Adequacy and Education Finance," http://www.ncsl.org/Home/tabid/118/Default.aspx?programs/educ/PubsAdequacy.htm.

New Hampshire School Administrators Association. Analysis of Cost Impact of ESEA– No Child Left Behind Act on New Hampshire. Penacook, NH, November 19, 2002.

Peyser, James, and Robert Costrell. "Exploring the Costs of Accountability: Claims that the No Child Left Behind Act Represents an 'Unfunded Mandate' Wilt Under Close Scrutiny," Education Next( Spring 2004), 23-29.

Rebarber, Thomas, and Thomson W. McFarland. Estimated Cost of the T esting Requirements in the No Child Left Behind Act. AccountabilityWorks, February 2002.

U. S. General Accounting Office. T itle I: Characteristics of Tests Will Influence Expenses: Information Sharing May Help States Realize Efficiencies. Washington, D. C., May 2003.

FURTHER READING

February 20, 2004 James Nobles, Legislative Auditor Office of the Legislative Auditor 140 Centennial Building 658 Cedar Street St. Paul, MN 55155

Dear Mr. Nobles: Thank you for your office's extensive program evaluation of the federal requirements under the No Child Left Behind Act of 2001 .

In January 2002, President Bush signed la ndmark legislation that brought additional accountability measures to public schools. Building on 1994's Improving America's Schools Act signed by President Clinton, the No Child Left Behind Act calls for closing the achievement gap, improving accountability through additional statewide assessments, undertaking school improvement, expanding educational options, and enhancing teacher quality while providing states and districts with additional federal funding.

The Pawlenty administration strongly supports the goals and tenets of No Child Left Behind . While Minnesota has high overall achievement rates, the State has not been as successful in closing the achievement gap between white students and students of color. While many will state that the goal of 100% pr oficiency is impossible, we must strive to provide a high quality education to all children. If the State is not going to support such a goal, then it must tell the public which children will be left behind. The Pawlenty administration is not prepared to do this. In a recent speech commemorating the 20th anniversary of A Nation at Risk , the seminal report on American education, U. S. Secretary of Education Rod Paige, echoed many of the same themes. He said: "Education is a civ il right, just like the right to vote or to be treated equally. And it's the duty of our nation to teach every child well, not just some of them. Yet in the greatest, most prosperous nation in the world, we had created two education systems -separate and unequal -that found it perfectly acceptable to teach only some students well while the rest -mostly minority and mostly low-income -floundered and flunked out."

Some might say that the law is unjust to schools, or that its costs outweigh its benefits. However, this should not stop the State and public schools from addressing the significant issues faced in education – partic ularly the achievement gap. Last November, more than 100 minority leaders and educators looked beyond the politics and signed a joint letter supporting No Child Left Behind . In the letter, they wrote: "No Child Left Behi n d …is a huge step forward in the movement toward full participation in American democracy…. Like other steps before it – including Brown v. The Board of Education and the Individuals with Disabilities Act -NCLB might justifiably be labeled as a mandate not "fully funded." But just as we then didn't use insufficient funding as an excuse to maintain legally segregated schools or to exile special education students from public school classrooms, we must not use funding to escape our responsibilities now. . . . Rolling back any part of the requirement to know more and do more about the large achievement gaps that have long blighted American education sends the wrong message and simply cannot be an option." The above statement is particularly meaningful given that research, as well as MDE's experience working with schools identified for improvement, indicates that successful change is not the result of some untried, magic solution. Rather , these schools have found that they need to focus resources around best practice initiatives such as a comprehensive curriculum with scope and sequence, an alignment of their curriculum to state standards, communication among teachers, use of data to identify strengths and weaknesses, on-going classroom-based assessments for diagnostic purposes and to monitor continual progress, and the use of instructional strategies that meet the needs of diverse student populations. As the State has started implementation and works with the federal government to gain approval of its plan for compliance, a number of changes requested by the State have been approved. These changes will provide greater flexibility to school districts and charter schools. We believe the federal government will continue to work with the State and provide more flexibility as we work toward full implementation and achieving No Child Left Behind's substantial but important goals.

The Department of Education takes seriously the recommendations made by the Office of the Legislative Auditor in the program evaluation. Most of the recommendations are in process of being implemented and some will be considered for implementation as the State continues to work with the federal government. The Department's responses to the recommendations are outlined below. OLA Recommendation: In cases where the Minnesota Department of Education determines that an AYP determination was made in error, it should ensure that the error does not adversely affect the school's or school district's sanction status in subsequent years.

2 MDE Response: When determining a school or district's AYP status, the department uses data submitted by the district through the state's MARSS (Minnesota Automated Reporting Student System) accounting system and their annual assessment reports. It is the district's responsibility to ensure data are accurate since the data are used to make numerous decisions, including general education aid.

In addition, districts are given two opportunities to make corrections to their assessment data. Prior to any public release of test information, districts receive early correction rosters to ensure that the student demographics and test participation data reported for a given school and district are correct. The department also instituted a month-long period for review of the AYP determination, data cleansing, and appeals prior to finalizing the AYP status for any school or district. The department has requested the Legislature to designate these preliminary determinations as non-public data so that schools and districts can have a full month to review the data prior to any public release. In 2003, the majority of districts met the timelines even though it was the first year of implementation. The department expects districts will take full advantage of the opportunities already in place and improve the accuracy of AYP determinations for schools and districts. Since there are multiple opportunities to correct school and district data prior to finalizing AYP status, the department does not agree wi th the recommendation to hold schools or districts harmless for uncorrected data errors found to be the responsibility of the school or district. In the event that the error come s not from a local district but from MDE, the error will be corrected and schools and districts will be held harmless. OLA Recommendation: The Minnesota Department of Education should provide the 2005 Legisl ature with a plan that outlines how value-added measures of student achievement could be incorporated into the annual AYP determination process.

MDE Response: The Pawlenty administration strongly supports a value-added system for measuring student achievement. Specifically, Minnesota schools will be able to use value-added measures as another academic component in the AYP formula. The specific criteria and process are currently under development. Implementation will depend upon action by the Legislature. MDE will continue to work closely with the Legislature on the development of the value-added system. OLA Recommendation: The Minnesota Department of Education and the Office of Educational Accountability should report to the 2005 Legi slature on any unresolved issues regarding the validity and reliability of Minnesota's accountability system.

MDE Response: As noted in the OLA report, the department has taken strong steps from the very beginning of its implementation to make its accountability system valid and reliable. By using an index measure, Minnesota's system acknowledges improvements in student performance across achievement levels. By requiring a specific number of students to have been assessed prior to making an AYP determination for any group and applying a confidence interval to the calculation, the department has ensured that the AYP determinations are made in the most valid and reliable manner possible.

The department has continued to work with the U. S. Department of Education to improve the state's accountability system. Changes made specifically to address the concerns regarding special education students and English Language Learners have made Minnesota's accountability system more valid and reliable. The department will continue to work on refinements as implementation progresses and changes are warranted. OLA Recommendation: The 2004 Legislature should require the Minnesota Department of Education to submit a plan to the House and Senate education committees that outlines how it will monitor the quality and effectiveness of supplemental educational services provider.

MDE Response: In 2003, MDE decided that the federal criter ia for certifying supplemental services providers were not rigorous enough. In order to ensure that th e State, school districts and charter schools were protected against "fly-by-night" set ups, MDE received rulemaking authority from the legislature to adopt more rigorous rules for certification. MDE plans to evaluate supplemental services providers based on outcomes of the services that were provided. At this time, the program is in its first year of operation; plans are still being developed and staffing levels will be reconsidered when more schools are required to offer supplemental services. Once those plans are in place, MDE will notify the legislature. OLA Recommendation: The Legislature should require the Minnesota Department of Education to annually prepare a report identifying, at a minimum, the expenditures incurred by school districts (regardless of funding source) to comply with NCLB-required sanctions for low-performing schools. MDE Response: As your office noted in Chapter 4, it is difficult to predict some of the costs of NCLB. It is also difficult to make determinations of what costs should be associated solely with implementation of NCLB and what costs should be seen as part of providing an effective and efficient system of public education.

If the Legislature requires MD E to quantify the costs of the NCLB-required sanctions on an annual basis, the burden will ultimately fall to the districts and schools to submit additional reports not required by NCLB. MDE certainly will monitor whether or not the school districts set-aside and use an amount equal of 20% of their Title I, Part A allocation to provide intra-district school choice and access to supplemental educational service providers.

MDE does not currently have the capability to quantify the costs as suggested and would have to rely upon districts to provide that information. This report would be a state-imposed requirement, not a NCLB requirement, and would place additional burdens on districts.

Again, thank you for your office's hard work on this program evaluation. If you have any questions, please do not hesitate to contact me. Sincerely, /s/ Cheri Pi erson Yecke Cheri Pierson Yecke, Ph. D. Commissioner

Recent Program Evaluations
Residential Facilities for Juvenile Offenders, February 1995 95-01
Health Care Administrative Costs, February 1995 95-02
Guardians Ad Litem, February 1995 95-03 Early Retirement Incentives, March 1995 95-04
State Employee Training: A Best Practices Review, April 1995 95-05
Snow and Ice Control: A Best Practices Review, May 1995 95-06
Pollution Control Agency's Use of Administrative Penalty Orders, Update July 1995 95-07
Development and Use of the 1994 Agency Performance Reports, July 1995 PR95-22
State Agency Use of Customer Satisfaction Surveys, October 1995 PR95-23
Funding for Probation Services, January 1996 96-01 Department of Human Rights, January 1996 96-02
Trends in State and Local Government Spending, February 1996 96-03
State Grant and Loan Programs for Businesses February 1996 96-04
Post-Secondary Enrollment Options Program, March 1996 96-05
Tax Increment Financing, March 1996 96-06 Property Assessments: Structure and Appeals,
A Best Practices Review, May 1996 96-07 Recidivism of Adult Felons, January 1997 97-01
Nursing Home Rates in the Upper Midwest, January 1997 97-02
Special Education, January 1997 97-03 Ethanol Programs, February 1997 97-04
Statewide Systems Project, February 1997 97-05 Highway Spending, March 1997 97-06
Non-Felony Prosecution, A Best Practices Review, April 1997 97-07
Social Service Mandates Reform, July 1997 97-08 Child Protective Services, January 1998 98-01
Remedial Education, January 1998 98-02 Transit Services, February 1998 98-03
State Building Maintenance, February 1998 98-04 School Trust Land, March 1998 98-05
9-1-1 Dispatching: A Best Practices Review, March 1998 98-06
Minnesota State High School League, June 1998 98-07
State Building Code, January 1999 99-01 Juvenile Out-of-Home Placement, January 1999 99-02
Metropolitan Mosquito Control District, January 1999 99-03
Animal Feedlot Regulation, January 1999 99-04 Occupational Regulation, February 1999 99-05
Directory of Regulated Occupations in Minnesota, February 1999 99-05b
Counties' Use of Administrative Penalties for Violations of Solid and Hazardous
Waste Ordinances, February 1999 99-06 Fire Services: A Best Practices
Review, April 1999 99-07 State Mandates on Local Governments,
January 2000 00-01 State Park Management, January 2000 00-02
Welfare Reform, January 2000 00-03 School District Finances, February 2000 00-04
State Employee Compensation, February 2000 00-05 Preventive Maintenance for Local Government
Buildings: A Best Practices Review, April 2000 00-06
The MnSCU Merger, August 2000 00-07 Early Childhood Education Programs,
January 2001 01-01 District Courts, January 2001 01-02
Affordable Housing, January 2001 01-03 Insurance for Behavioral Health Care,
February 2001 01-04 Chronic Offenders, February 2001 01-05
State Archaeologist, April 2001 01-06 Recycling and Waste Reduction, January 2002 02-01
Minnesota Pollution Control Agency Funding, January 2002 02-02
Water Quality: Permitting and Compliance Monitoring, January 2002 02-03
Financing Unemployment Insurance, January 2002 02-04
Economic Status of Welfare Recipients, January 2002 02-05
State Employee Health Insurance, February 2002 02-06 Teacher Recruitment and Retention: Summary
of Major Studies, March 2002 02-07 Local E-Government: A Best Practices Review,
April 2002 02-08 Managing Local Government Computer Systems:
A Best Practices Review, April 2002 02-09 State-Funded Trails for Motorized Recreation,
January 2003 03-01 Professional/ Technical Contracting,
January 2003 03-02 MinnesotaCare, January 2003 03-03
Metropolitan Airports Commission, January 2003 03-04 Preserving Housing: A Best Practices Review,
April 2003 03-05 Charter School Financial Accountability,
June 2003 03-06 Controlling Improper Payments in the Medical
Assistance Program, August 2003 03-07 Higher Education Tuition Reciprocity,
September 2003 03-08

Evaluation reports can be obtained free of charge from the Legislative Auditor's Office, Program Evaluation Division, Room 140, 658 Cedar Street, Saint Paul, Minnesota 55155, 651/ 296-4708. Full text versions of recent reports are also available at the OLA web site: http://www.auditor.leg.state.mn.us