Content
and Student
Achievement
Standards
+
Testing
+
Professional
Development
+
Accountability
Reporting
+
Rewards, Sanctions
and Targeted
Assistance

Accountability

Contact: Gale Gaines
Joseph Creech
Released: July 1998
SREB (404) 875-9211

Getting Results: A Fresh
Look at School Accountability

When the Southern Regional Education Board asked state and regional leaders to establish educational goals in 1988, school accountability was one of the goals: “All states and localities will have schools with improved performance and productivity, as demonstrated by results.” The SREB has worked with states during the last 10 years on this goal. SREB conferences and reports on accountability have given legislators and educators ways to learn from each other.

Since 1988 the public and policymakers have continued to press to improve schools, “do accountability right” and show results in student achievement. Accountability for student learning has become crucial to the future of public education. SREB states have begun ambitious school-accountability programs that are making a difference in improving schools and student achievement. The recent passage of comprehensive K-12 accountability initiatives by the South Carolina and Delaware legislatures illustrates regional policymakers continued focus on improving student achievement.

The SREB has worked with state leaders on school accountability for the last 10 years and has identified five policy areas that are crucial parts of a comprehensive school-accountability program: content and student achievement standards; testing; professional development; accountability reporting; and rewards, sanctions and targeted assistance. Experience has shown that none of these areas stands alone. They all must be done well and must align with one another in order to raise student achievement and improve schools.

With the cooperation of the SREB states’ chief school officers, the SREB recently convened key staff of state education agencies to discuss their experiences in implementing accountability programs. Before the meeting teams from each state were asked to identify the “essential characteristics” of the five policy areas. In two work sessions agency staff from the SREB states discussed these traits and came to consensus. This report reflects the “essential characteristics.”

In coming years the SREB will continue to champion school accountability for student achievement. The SREB will convene state policymakers, their staffs and state education officials to focus in practical detail on the five policy areas and their alignment and will publish information to help state and local officials as they make decisions on accountability.

Mark Musick, president
Southern Regional Education Board

Content
and Student
Achievement
Standards
+
Testing
+
Professional
Development
+
Accountability
Reporting
+
Rewards, Sanctions
and Targeted
Assistance

Accountability

Content and Student
Achievement Standards

Content and student achievement standards should be:

• Developed with considerable input from a variety of sources
• Concise and understandable
• Rigorous and challenging
• Reasonable and attainable
• Focused and organized by grade level or course
• Measurable whenever possible

Content and student achievement standards are the most important elements of a good system of school accountability. Content standards define what students should learn and student achievement standards define how well students should learn it. Accountability for student learning is impossible without a clear, focused “road map” of what and how well students are to learn from kindergarten through graduation.

The advent of “high-stakes” accountability programs throughout the SREB region in the last 10 years proves that elected officials and the public want to see results in student achievement and school improvement. Developing appropriate content and student achievement standards are an important part of showing policymakers and the public a “return on their educational investment.” Unfortunately, many states have developed content and achievement standards without involving the public and teachers. Many also have not considered how their standards are to be implemented, measured and tied to accountability.

Content and student achievement standards should be developed with considerable input from a variety of sources. In the past a small number of educators controlled standards and curriculum development. Standards often were developed apart from the day-to-day reality of classroom teachers, parents and the public.

Effective standards for content and student achievement are developed with a “consumer orientation” that includes communication with teachers, parents and the public. Giving these groups constructive roles helps ensure their support. Ideally, the communication begins with listening carefully to classroom teachers and parents.

Both Texas and Virginia recently revised content standards with considerable involvement of the public. Allowing a variety of people to help develop content standards was challenging and sometimes controversial. However, it appears that most teachers, parents and members of the community understand and support the new standards.

North Carolina and Maryland are working with the National Assessment Governing Board to compare their state content standards and state assessments with the challenging curriculum frameworks on which the National Assessment of Educational Progress (NAEP) is based. Comparison with a credible, challenging assessment can provide states with valuable information for the continuing development of their curriculum and state assessments.

Content standards and student achievement standards for high schools should consider what colleges and employers expect. Students benefit most from high school courses aligned with the expectations of higher education and the workplace. Success in the work force depends on the ability to learn after graduation from high school. That learning may be on the job, in vocational and technical schools, or in colleges and universities.

Content and student achievement standards should be concise and understandable. State content standards should define clearly what children are expected to learn and be able to do at different stages from kindergarten through 12th grade. For content standards to be useful, the language must be clear to teachers and parents. An effective review includes classroom teachers and parents to verify the clarity and usefulness of content standards.

States and schools also should design information specifically for parents so they understand what is expected of their children and how they can help their children learn. The West Virginia Department of Education developed parent guides that illustrate specific content standards for each grade level. These easy-to-understand guides provide useful information on how parents can help their children learn at home and give tips on how they can communicate better with the school about their children’s progress.

Content standards must be specific enough. Vague standards can be interpreted in many ways and, in turn, taught inconsistently. Those that are too general are impossible to assess reliably. Content standards should be detailed enough to help guide local curriculum development and teacher instruction. An appropriate balance involves specific content and opportunities to apply that knowledge by solving real-world problems. Standards also should use examples in order to help parents and teachers understand what students are to learn and at what level.

As with content standards, effective student-achievement standards are understandable to teachers, parents and others. Student achievement standards should identify clearly the levels at which students and schools are expected to perform. Texas uses the labels “exemplary,” “recognized,” “acceptable” and “low-performing” to describe schools’ achievements. Kentucky uses “novice,” “apprentice,” “proficient” and “distinguished” to describe students’ performance. Maryland describes performance in its schools as “excellent,” “satisfactory” or “standard not met.” Whichever terms are used, these examples clearly describe achievement and offer understandable information that can motivate schools and students to improve.

Many SREB states are wrestling with defining the roles of the state and of local schools in an era of high standards, accountability and greater flexibility. In order for student achievement to improve, the state’s role of developing standards and accountability programs must be balanced with local control over teaching methods and allocation of resources.

Clear content standards can help provide focus for classroom teachers. Including substantial information about how to teach the content detracts from the content standards’ primary purpose of expressing what is to be taught. As a state deputy superintendent put it, “The primary responsibility of the state is what and how well students learn, not how the teachers teach.” It is important, however, that the state assist local schools by reviewing instructional materials, providing training and otherwise helping teachers adhere to state content standards.

The Virginia Standards of Learning is a 100-page document that clearly provides challenging content standards for grades K through 12 in mathematics, science, English, history, social studies and technology. The document recognizes the importance of the local school’s role when it states: “The standards are not intended to encompass the entire curriculum for a given grade level or course, or to prescribe how the content should be taught. Teachers are encouraged to go beyond the standards and to select instructional strategies and assessment methods appropriate for their students.”
Florida, Texas and North Carolina recently gave local schools and school districts considerable authority and flexibility in the use of state resources and the delivery of instruction. However, these states retained the state responsibilities of defining content and student achievement standards and assessing them according to state measurements.

Content and student achievement standards should be rigorous and challenging. Content standards should require an increasing depth of knowledge at each grade level and course. Meaningful student-achievement standards are demanding enough to compare favorably with those in other parts of the nation and the world and are expected consistently, no matter what school a child attends. As one state agency official put it, “Children will live up or down to the expectations that are set for them.”

The significant differences in student achievement standards from state to state make comparisons difficult. For example, what if more than 80 percent of one state’s eighth-grade students meet its achievement standard for mathematics, while in another state fewer than 30 percent of the students are doing well enough by its standard? Or what if one state says that almost 90 percent of its third-graders meet its standard for reading and another state reports that fewer than 30 percent of its third-graders do? If the residents of these states became aware of these differences, wouldn’t they ask, “What’s going on here?”

States should develop a review process that includes comparisons with other states’ rigorous standards for content and student achievement. Without understandable and valid comparisons of content and achievement standards, the public will remain justifiably confused and cynical. Sharing standards information among SREB states is vital to maintaining credibility through accurate, appropriate comparisons.

Content and student achievement standards should be reasonable and attainable. Setting standards that are unrealistic at a particular grade or within a course is unfair to teachers, parents and communities that are struggling to improve student achievement. It is important to balance the need for challenging standards with the need for reasonable expectations. A state department of education official illustrated the difficulty in achieving that balance: “If we don’t set a high school exit standard high enough, then we don’t satisfy colleges and universities and wind up with remediation issues. If we set the exit standard too high, then we have students that don’t meet it and drop out of school.”

Part of the solution is to establish graduated levels of expectations, such as exemplary and acceptable standards for student and school achievement. This enables teachers and parents to see improvement over time. A state testing director pointed out that school improvement depends on raising each student’s achievement, “We need to understand that this is about individual children, and the way to achieve balance is to use incremental levels and move children up.” Students are more likely to achieve high standards if they are given fair opportunities to improve and can chart their progress over time.

Content standards should be focused and organized by grade level or course. The most effective standards for content in the core academic areas are defined in elementary and middle schools by grade level or small clusters of grades. At the secondary level, grade and coursework should define content standards. Standards that do not specify when content is to be mastered are useless, for example, to a fifth-grade teacher, a high school history department chairperson or a parent of a third-grader.

Proper organization of content standards by grade level (K through eight) and course subjects (nine through 12) provides continuity, eliminates gaps and repetition, and tells teachers and parents when students should master material. Well-designed standards illustrate how knowledge and skills are built over time so that teachers and parents can see how students are to progress.

Often content standards include too many topics and do not consider the limited time that students have in a given grade or course. The result is a lack of focus in the classroom that makes it harder for students to concentrate on important subject matter. A recent national study that compared American curricula with those of other nations characterized American math and science curricula as “... a mile wide and an inch deep.” The study found that teachers could not explore any topics in depth because they have too much material to cover.

A state agency official from Kentucky emphasized that “the word focus should be underlined 40 times.” A deputy superintendent from another state said the most common characteristics of successful schools in that state’s accountability program were “... a strong school principal; a stable, veteran faculty; and, most importantly, that teachers teach the state content standards.” He added that those characteristics were frequently absent among low-performing schools that “... all too often ... are simply not teaching the stuff that they’re supposed to.”

Content standards should be measurable whenever possible. Content standards provide a solid base for aligned assessments. State content standards that cannot be measured reliably will not provide accurate, useful information for accountability purposes. Certainly, not all learning that is important is measurable, but standards can focus on topics for which accurate measures are available or can be developed.

If local curricula and classroom assessments are in sync with state content standards, students will have a better chance to perform well on state tests. Local curricula that align with state content standards then can expand on the topics outlined in the standards. Classroom assessments can include more detailed tests of higher-order skills that are difficult to measure reliably on
once-a-year state tests.

Content
and Student
Achievement
Standards
+
Testing
+
Professional
Development
+
Accountability
Reporting
+
Rewards, Sanctions
and Targeted
Assistance

Accountability

Testing

State testing should:

• Be reliable and valid for accountability purposes
• Have a clear purpose
• Be aligned directly to content standards
• Be useful for school improvement
• Be operationally feasible

School performance based on the results of state testing programs has become an important and controversial part of state accountability programs. Intense public scrutiny and challenges to the high-stakes consequences schools face test the political will of the policymakers who initiate accountability programs. In addition, results from state assessment programs increasingly form the basis for important decisions about resources, including financial rewards for schools, jobs and salaries. Assessments must be trustworthy.

States must ensure that adequate time, resources and oversight go into planning, training for school personnel and implementation of testing programs. Careful planning of assessments that are cost-effective and linked clearly to content standards can strengthen the long-range support by policymakers and the public.

Public confidence in public education and future investments by policymakers rest largely on continuing evidence from schools that students are learning more and that schools are improving. Well-designed testing programs in accountability systems enable educators to provide that proof.

State tests should be reliable and valid for accountability purposes. A state testing system must incorporate the key technical considerations of reliability, validity, fairness and bias. Failure to adequately address these critical considerations weakens an accountability program that includes high-stakes and might face legal actions. As a director of a state testing program pointed out, “If you deny something valuable to people that they expected, you’re probably going to get a lawsuit.” Here are some questions that policymakers need to ask regarding these technical considerations:

What is the purpose of the test, and how will the results be used?

If the test is to be used for “high-stakes” decisions about schools or individuals, it must be technically sound and legally defensible. On the other hand, if test results are to be used informally to help student learning, the technical issues of reliability and validity are less important.

Is the test valid? Does it measure what it is supposed to measure?

Tests used in accountability programs must assess what state content standards expect students to know and be able to do. Content standards should specify, and tests should measure, the knowledge and skills necessary to move to the next grade level or the next course in a sequence.
Good tests measure what all students have had an opportunity to learn. Fourth-graders are not expected to take tests designed for eighth-graders. In the same way, students in schools that have implemented state content standards have more opportunity to learn than students in schools that have not.

Is the test reliable? Can the results be trusted?

Because tests can provide only estimates of what students know and can do, no test is 100 percent reliable. Tests should cover enough topics to sample adequately what students know and to report consistent scores. If the testing is similar from year to year, comparisons can be made over time. For example, the performance of eighth-graders in 1996 can be measured against that of eighth-graders in 1998.

The types of questions on a test also affect its reliability. Tests that contain only multiple-choice questions generally are highly reliable in a measurement sense, because more questions can be asked in a specified time period and the test can be administered consistently. Performance-based assessments that use open-ended responses are less reliable because they may not be administered under the same conditions each time (from school to school or student to student).

Is the test fair?

A test is not fair if it puts members of different racial, ethnic or gender groups at a disadvantage by assuming background knowledge that all students may not have. It is not fair if it measures content knowledge and skills not covered in the curriculum. Test security and ethical standards also affect fairness. Steps to guard against cheating and mismanagement include setting and enforcing clear ethical standards, creating multiple forms of tests, establishing sound administrative procedures and providing training for administering tests.

Local schools’ staff, if trained properly, can provide adequate security in the administration of tests. State and local monitoring methods can assist in ensuring fairness and, as one state’s testing director put it, can fulfill the need to “trust but verify.” Training and other safeguards are likely to limit problems. If unethical behavior occurs, states must be ready with clear consequences and procedures.

Effective testing and accountability programs use ongoing, independent and systematic reviews to ensure that they are meeting their designed purposes. State policymakers who oversee accountability programs then can be reassured of the programs’ integrity and long-term viability.

State testing should have a clear purpose. The continuing focus on improving student achievement has driven much of the increased emphasis on state testing in the last decade. Yet confusion and debate remain over these tests’ purpose and design. States often are tempted to have one test serve many purposes, which can weaken its effectiveness and reliability. “State tests that try to do too much risk the possibility of doing none well,” said one staff member of a state agency. Worse yet, technical weaknesses that result from overextending state testing expose the states to legal challenges.

State tests should be aligned directly to content standards. It makes sense for states’ assessment systems to be based on state content standards and for test items to link clearly to the standards they are designed to assess. This link helps ensure the validity and technical integrity of the tests and accountability systems. State tests developed specifically to measure student achievement on state content standards will show whether students are being taught what the state has determined they should learn. Schools, teachers and students will benefit as a result.

One result of high-stakes accountability is that teachers often focus on what is tested rather than on content standards. If the state tests reflect challenging content standards, the relationship can be positive. However, if state tests do not measure higher-level knowledge and skills, teachers often attempt to improve student performance on tests by focusing on isolated bits of information, minimal skills and test-taking tips.

State tests should be useful for school improvement. State testing and classroom assessment can complement each other and work together to raise student achievement and improve schools. Teachers, parents and the public must understand how testing works and why it is useful.
The results of state tests will be useless unless they clearly show each school how its students are performing and where their instructional strengths and weaknesses lie. Every school has unique demographic characteristics that are important considerations when planning for school improvement. States such as Florida, Maryland and Texas report student results by race, ethnicity and gender, giving more depth to the picture.

Effective testing systems give parents details on test results that enable them to determine their children’s levels of competency and to see how they compare with other children. Parents who have full, useful reports on their children’s achievement have a starting point for meaningful conversations with schools.

State tests should be operationally feasible. Because state testing is a critical part of accountability, enough resources should be provided to ensure that state tests are adequate. Although a state testing program should be cost-effective, policymakers must recognize that, as a state associate superintendent for accountability put it, “In the testing and accountability business we can’t afford to be penny-wise and pound-foolish.” Continuous test development, training, administration, scoring and external technical evaluation are all necessary and cannot be done “on the cheap.”

Less-expensive “off-the-shelf” tests are tempting to some policymakers but may not reflect state content standards enough to be a good measure of student achievement. A commitment to invest fully in a technically sound program for state testing is a fundamental step toward improving schools and student achievement.

States also must provide enough time for tests to be developed. While it is necessary to get accountability programs “up and running” and to begin “seeing things happen,” too often unrealistic time lines are imposed. A state accountability director complained: “We get caught in the middle of legislative mandates and time lines that do not necessarily jibe with the right way to do things. They need to pay attention to what makes sense and will work, and then they’ll get what they want.”

If enough time is provided to plan tests, develop them, conduct field tests and revise the tests to reflect changes in content standards, educators are more likely to be familiar with tests and have confidence in the program. It is especially important to notify schools and parents promptly when significant changes are made in testing, accountability and content and student achievement standards. Teachers and schools must have a chance to prepare for more challenging standards. Inadequate notice exposes a state to legal action.

Content
and Student
Achievement
Standards
+
Testing
+
Professional
Development
+
Accountability
Reporting
+
Rewards, Sanctions
and Targeted
Assistance

Accountability

Professional Development

Professional development should be:

• Aligned with content standards and assessment
• Focused on results in student achievement
• Flexible and responsive to school needs
• Accessible and convenient
• A part of the day-to-day work in schools
• Adequately funded
• Coordinated among local schools, higher education and state agencies

Quality professional development for teachers and principals is a key part of improving student achievement and reaching higher standards for accountability. New standards, assessments and accountability efforts cannot succeed unless states and schools make professional development a priority. As one state agency official put it, “You can make all the changes you want at the state level, but nothing is going to happen unless change takes place at the classroom and school levels.” Yet professional development usually lacks the attention and funding that most states give to other school-reform efforts.

Educators who have no access to useful, quality professional development will be incapable of dealing with challenging content standards, higher expectations for all students and new methods of assessment, instruction and accountability. Teachers may need new knowledge and skills to teach a more demanding curriculum and assess student achievement. Similarly, administrators need new leadership skills so that they can help schools focus on student achievement and can support accountability efforts.

Unfortunately, several barriers prevent educators from getting meaningful training in how to improve student achievement; many educators have had no opportunity to learn about the changes in content standards, assessment and instruction. Among factors that limit teachers and principals efforts to improve their skills are irrelevant, fragmented and low-quality professional development; inadequate funding; and lack of time.

Recent actions to give local schools more flexibility in their use of resources and instructional delivery are a mixed blessing if there is not support for professional development. Many local schools need help to plan and deliver effective professional development. Local district central offices, higher education and state agencies should lend their expertise to better meet the unique training needs of local schools. Most SREB states require local school “improvement plans” that typically include professional development plans directed at improving instructional weaknesses. In order for student achievement to improve, state and local districts must help local schools meet their professional development needs.

Professional development should be aligned with content standards and assessment. Professional development for teachers and administrators tends to be fragmented, lacks focus and often has no direct relationship to state content standards and assessment of student achievement. It also tends to shift from one topic to another instead of addressing identified needs systematically. Aligning professional development with state content standards and assessment results is a step toward improving student achievement. Student achievement goals cannot be met if teachers do not understand the higher expectations and how they can help their students reach them.

Many teachers have neither majors nor minors in the subjects they teach. Research supports the need for teachers to be well-prepared in the subjects they teach. Recent studies reveal alarming statistics on out-of-field teaching in middle grade and high school subjects, especially in mathematics and science.

A deputy superintendent raised the issue vividly: “How many of you would want a pediatrician performing brain surgery on you? Or a tax lawyer defending you in criminal court? The same kind of thing is going on in classrooms: Too often in math, science, and reading and other subjects, teachers are not prepared to teach the subject.” This lack of academic preparation becomes acute as states raise content standards.

All teachers need ongoing professional development to deepen their knowledge about the subject matter that they are teaching and to provide information about new state tests and accountability programs. It is also important that they have opportunities to learn instructional techniques
that help their students improve.

Professional development should be focused on results in student achievement. As a staff member of a state agency put it, “The relationship between professional development and improved student achievement should be clear and unambiguous.” An increased focus on rigorous content standards, student achievement and accountability has put more pressure on schools to show results. Accountability initiatives have made it increasingly important to help teachers and principals gauge, student by student and classroom by classroom, whether students are learning.

Professional development should be flexible and responsive to school needs. A careful analysis of results on tests that are aligned with content standards is an integrated part of a school improvement plan that identifies instructional strengths and weaknesses in each school. Efforts to improve schools and professional development must be tailored to meet these individual needs; generic solutions will not increase student achievement. Programs must help principals and teachers understand how to collect, analyze, report and use student achievement data to determine what instructional changes are needed.

Unfortunately, school districts and schools commonly bring in outside “experts” for professional development who offer “one-shot” solutions that might not meet the real needs of teachers and schools.

Professional development should be accessible and convenient. Making time for professional development is both important and difficult. Teachers and principals spend most of their time actively engaged with students and have little opportunity to do anything beyond teaching and administrative duties. The nature of their work tends to limit teachers’ opportunities to share their knowledge and ideas with other teachers.

Although many states and local districts offer “professional days,” administrative duties and meetings with parents take up much of that time. As one official at a state agency put it, “Just adding a day or two at the beginning or end of the school year just doesn’t do it.” Professional development also takes place after the school day ends. Unfortunately, that time is limited and comes when teachers frequently are tired and unfocused. Programs that take teachers out of the classroom leave students without their regular teacher and are unpopular with parents and policymakers.

There is not an easy solution to finding appropriate time, but states are beginning to explore alternatives. One possibility is to lengthen the year to include training for some personnel. This option includes intensive professional development during the summer and follow-up clinics on weekends.

Professional development should be a part of day-to-day work in schools. A professional-development coordinator at one state agency agreed that, although time is an issue, professional development has to become part of the day-to-day activity of a school: “Professional development has to be continuous, ongoing and an integrated part of the school day, and it is not. We have a generation of teachers who don’t understand what encompasses professional development. They think it’s out of the classroom, sitting down and hearing a speech or being at a workshop. They don’t realize that it can be observing another teacher, interacting professionally with other teachers in planning, reading an article on the Internet or sharing activities.”

Training teachers and administrators to integrate professional development into all aspects of their everyday work will help solve the problem of time. If given the chance, teachers can interact with one another regularly and can build confidence in their ability to seek out and share information that may help solve problems unique to their school.

Professional development builds teacher expertise at the school level, which can increase the sharing of information and result in a better understanding of and acceptance of new standards and strategies.

Adequate, consistent funding should be provided for professional development. Professional development generally is not a funding priority. Funding that is available comes from a variety of sources that often are dedicated to specific programs or initiatives. This method of funding clouds the focus on student achievement and results.

Ambitious student-achievement goals demand that schools have access to a variety of professional development programs that meet their needs. Resources can be dedicated to develop a quality “menu” of professional development offerings aligned to state content standards and assessments. Schools and individual teachers then can select from those offerings to fulfill their needs.

Coordination among local schools, higher education and state agencies can strengthen professional development offerings without duplicating costs. One group alone can not meet all of the professional-development needs of teachers and administrators effectively. Those who provide professional development must be in tune with the focus on higher standards and student achievement so that a well-defined, coherent approach is possible. Close coordination will result in better-focused and efficient professional development.

Content
and Student
Achievement
Standards
+
Testing
+
Professional
Development
+
Accountability
Reporting
+
Rewards, Sanctions
and Targeted
Assistance

Accountability

Accountability Reporting

Report cards should:

• Focus on student achievement and educational results
• Be useful for school improvement as part of a total accountability system
• Be concise and understandable for a variety of audiences
• Provide timely and accurate information
• Show trends
• Give school-, district- and state-level information
• Include data on groups of students within schools when appropriate

When SREB states passed report card laws in the late 1980s and early 1990s, the purpose was to inform the public of the quality of the schools and to build support for educational reforms. There was hope, too, that reporting performance information would cause school improvement. One person put it this way: “What gets measured gets taught. What gets reported gets taught twice as well.”

Early report cards were recognized as initial efforts that would be fine-tuned over time. With improvements in reporting came the realization that report cards alone would not drive improvements in schools and student achievement. Just as report cards for students gauge performance, so do school accountability reports. But just as a student’s report card alone does not cause improvement (the student must learn with the help of parents and teachers), reports alone will not cause school improvement. Teachers must understand what is expected — what students should know and be able to do — and must be given the tools through professional development to meet the challenge.

Report cards should focus on student achievement and educational results. When SREB states began accountability reporting, what was included often was what was available. Early reports were heavy on “input measures” that described the characteristics — but not the quality or performance — of a school or district and its community. Reports commonly included financial information, student characteristics and information about teachers and other staff. This information helped to define the context in which the school operated and outlined factors believed to affect student performance. The limited performance data included in most reports were not tied to challenging standards.

Good reports focus on how students are performing and reflect standards and assessments that are part of the overall accountability system. Other variables, such as demographics, are interesting but should not excuse students’ poor performance or be used to lower expectations. Reports by the SREB states use measures such as test scores, attendance and dropout data to show student performance. Just as a car’s condition cannot be determined simply by checking the oil, a school’s educational health cannot be measured fully without considering several factors.

Report cards should be useful for school improvement as part of a total accountability system. A state policymaker once said, “Report cards show us where we are so we can develop a road map to improvement.” As more experience is gained in reporting, it is becoming clear that reports must show schools and districts where they are, where they need to go and where their strengths and weaknesses lie. A meaningful report card will define clearly the goals of the state, district and school. It also will describe measures on which the school is to be graded and how they tie to content and performance standards.

But preparing and distributing school reports are not enough. If the reports do not lead to action, they are a waste of time, money and effort. Reports will be effective only if teachers and principals know how to interpret the information and have the authority to translate it into classroom practice for school improvement. This capability has become more important as states have charged local districts with figuring out how to meet performance standards.

Report cards should be concise and understandable for a variety of audiences. The terms “concise” and “understandable” seem straightforward enough, but their meaning depends upon the audience for which the report card is intended. Schools, school districts, policymakers, parents and the community all may need a school report card, but these groups are unlikely to require the same level of detail.

School and district staff and advisory councils that make recommendations about a school’s operation need a detailed version that helps them identify specific strengths and weaknesses and plan improvement. Businesses, parents and the community are unlikely to need or want an extremely detailed report. They usually want to know whether students are learning and whether school performance is improving, and they want to see this information quickly.

SREB states have found that tailoring the reports to specific customers helps keep the reports short, focused and useful. Customer surveys or focus groups can help ensure that each audience’s information needs are met. Comments from parents in Georgia led the Department of Education to prepare reports in two formats: One contains comprehensive data and is aimed toward educators and a more user-friendly version is designed specifically for parents. To determine what parents really wanted in a report, the department held six focus groups and used the results in designing later reports.

Report cards should provide timely and accurate information. Because of the focus on accountability and school improvement, good report cards focus on student achievement. Information collected consistently each year will make comparisons possible and will build confidence in the reporting system.

Discrepancies found in the early report cards taught states the importance of verifying the data. Doubts about the data’s accuracy made school and district staff suspicious and distrustful and raised questions from state policymakers. The public lost confidence in the reports as part of larger reform efforts. While most states have developed procedures for checking the information, there is still room for improvement. As information in the reports has become the basis for rewards and sanctions, data have come under even more scrutiny. Ensuring the information’s quality and timeliness requires states continuously to analyze and upgrade data collection systems.

Timely reporting on student achievement is difficult because of the time it takes for the results of performance assessments to become available. Testing students early in the school year makes results available during the same year but can disrupt the early weeks of school, when routines for the year are developed. Testing later in the year means results arrive later.

Texas has helped districts and schools plan improvements by releasing parts of the accountability reports, such as dropout and attendance data and test scores, to the districts as the information becomes available. Local educators can put pieces of the report card together by late June and use them in planning for the upcoming year. Entire accountability reports with district and school ratings are released in August, after most planning is complete.

Report cards should show trends. Because accountability systems are all about improvement in student achievement, it is important to report consistent, reliable data over several years to show progress. Long-term reporting encourages schools and districts to look beyond the current class of students and shows that accountability is a process of continuous improvement.

The accountability system that Kentucky adopted in 1990 included a standard that schools were to meet in 20 years as well as goals for progress to be measured every two years. Schools were expected to progress one-tenth of the way to the standard every two years, and reports reflected this measure. Legislation passed in 1998 calls for a review of the testing and accountability systems, including the 20-year timeline.

Maryland uses 1990 as a base year for dropout and attendance data, then reports the two most recent years so that a clear picture of progress — or lack of progress — can be seen. Schools and districts are moving toward a standard that all are expected to meet. Reports in Alabama, Louisiana, South Carolina and Tennessee also include multiple years of data.

Report cards should include state-, district- and school-level information. In addition to marking their own progress over time, schools and districts should compare themselves with other schools and districts and measure themselves against statewide data. While these comparisons do not necessarily reveal whether individual students are doing “well enough,” they do set the context in which the school is progressing.

Report cards should include data on groups of students within schools when appropriate. When SREB states began reporting information at the school level, officials found that district averages masked the performance of individual schools. One state department of education official described a typical district in her state that contains four high schools: “You can examine the data for the district as a whole and perhaps draw some conclusions. But when you look at the data for each individual high school, you find that no school is really described by the average data for the district.” A similar realization arose as school reports began providing information on groups of students — by race, ethnicity and gender, for example — within schools. In one example, a high school met a satisfactory level on a measure, but data on groups of students revealed that several failed to meet the standard. This problem normally would not show up in school averages.

Many states have adopted goals that aim for all children to learn at high levels and reach their full potential. This concept recognizes each school’s responsibility to bring all children to high levels of achievement. Florida, Maryland and Texas, for example, have reported information on groups of students by race, ethnicity and gender for several years. Texas also reports on economically disadvantaged and special education students. With the exception of the special education students, each group of students is expected to meet the standards in order for a school or district to be rated as “acceptable,” “recognized” or “exemplary.”

There is some controversy about reporting data on groups of students.

Care must be taken to report accurately and to avoid negative consequences, such as labeling groups that historically have not performed well. In addition, if the groups are too small, results may reflect negatively on individual students. Maryland, for example, does not report on any groups (such as Hispanic fe-males or Native American males) of fewer than five students. Schools should emphasize improvement for all students and for each group.

Content
and Student
Achievement
Standards
+
Testing
+
Professional
Development
+
Accountability
Reporting
+
Rewards, Sanctions
and Targeted
Assistance

Accountability

Rewards, Sanctions and
Targeted Assistance

Rewards, sanctions and targeted assistance should be:

• Fair, consistent and equitable
• Based on clear rules
• Balanced with one another
• Based on both absolute standards and improvement
• Supported with adequate and sustained financial resources

In addition, targeted assistance should:

• Focus on producing results
• Develop the ability of school staff to plan for and achieve continued improvements toward high standards

Rewards, sanctions and targeted assistance — most SREB states are implementing one or more of these actions in an effort to improve student achievement. Not all states, however, have tied these programs to comprehensive systems of accountability.

Accountability is not a new concept in education, but the definition has changed. In the 1970s, schools were considered adequate if they had ample classroom space, enough teachers and current textbooks. In the 1980s, attention was given to educators’ skills and knowledge. Teacher preparation programs were reviewed, licensure laws were changed and states began requiring teachers to pass competency tests. Today the focus of accountability is on student learning. Schools are held responsible for all students’ progress toward rigorous state standards and are given the flexibility to determine the best way to reach those standards. School-by-school results are used to decide which ones qualify for rewards, sanctions and targeted assistance.

Do rewards, sanctions and targeted assistance work over the long haul? The jury is still out, but several states have learned some lessons about them. Fairness, consistency, clarity and balance are essential in all areas of accountability systems, including rewards, sanctions and targeted assistance.

Rewards, sanctions and targeted assistance should be fair, consistent and equitable. In addition, teachers, parents and the community must perceive them as such if states are to build confidence in the school improvement process. Because of the high-stakes nature of these efforts, through which schools can be closed and teachers can lose their jobs, well-thought-out policies and procedures must be applied consistently to all schools in order for the public and policymakers to support the efforts.

The measures used to determine whether schools and districts qualify for rewards, sanctions or assistance must be technically sound and valid for those purposes. Kentucky and Tennessee were among the first SREB states to develop reward and sanction programs. Kentucky postponed sanctions amid questions about the testing system it was using. Legislation recently passed will change the testing system and the method used to determine eligibility for rewards and assistance. Likewise, Tennessee postponed tying students’ test scores to individual teachers. These early lessons point to the need for ongoing evaluation as accountability systems are implemented.

Rewards, sanctions and targeted assistance should be based on clear rules. School and district staffs, teachers, parents and the public need to understand what measures are used to judge their schools and how these align with the state content and student achievement standards. A well-designed marketing plan is necessary to explain the standards and measures and how they relate to criteria for rewards, sanctions or targeted assistance. “The biggest mistake we made was not hiring a public relations firm,” said one state official. “We were selling a product and we failed to do that well. Communication is so important.”

Rewards, sanctions and targeted assistance should be balanced with one another. Nearly all SREB states have programs of sanctions or intervention on the books, but few states have implemented them based on student achievement results. Most sanctions have been applied for financial reasons. Half of the states in the region have given financial rewards for meeting or exceeding student achievement targets, and most provide targeted assistance in the form of additional funds, expert help or both.

Some say that the threat of sanctions is more effective than the hope of rewards in inspiring schools to improve. Others argue that rewards, if adequately funded, have more impact. Most would agree that an effective accountability system requires balance between the two, with additional assistance for underachieving schools. But balancing rewards, sanctions and assistance raises questions. Is it fair to prohibit individual teachers from receiving bonuses for raising student achievement if they can be fired for negative results? Is it fair for certain teachers to be scrutinized because assessments are given only in the grades they teach, while teachers schoolwide are eligible for bonuses when achievement exceeds expectations? How much assistance is enough before sanctions are imposed?

Balance also comes into play when determining how many schools are the top performers or most-improved and how many are the lowest performers or least-improved. The wide variation in how states define this balance reflects their different program philosophies. The number of schools receiving rewards in the region in 1997-98 ranged from 2 percent in Georgia to about 40 percent in Kentucky. In Georgia schools are considered for rewards only if they identify and then meet performance objectives. All schools in Kentucky are eligible for rewards, and all are expected to exceed a performance target.

Initial actions in two states in 1997-98 illustrated a similar variation in identifying low-performing schools. In North Carolina about 7 percent of all schools failed to meet the standards that required an expected level of improvement and a majority of students to perform at grade level. Only 20 schools, about 1 percent, received additional assistance. Louisiana officials required each school district to identify, using its own criteria, the lowest-performing 20 percent of its schools in preparation for an accountability plan under consideration. The idea was that districts should begin thinking about these schools because many may be designated as “academically unacceptable” under the new system.

Rewards, sanctions and targeted assistance should be based on both absolute standards and on improvement. Many states, such as Kentucky, Maryland, Texas and Virginia, have adopted rigorous standards that all schools are expected to reach. As described in Maryland: “We report on whether standards are met. All schools will be ‘graded’ as either meeting or not meeting the standards, whether it is the city of Baltimore or one of the county systems. The fact that a school has a high number of at-risk students cannot be used as a reason to explain why the same standards should not be expected.”

While some schools might be in sight of achieving those standards, others may be overwhelmed by how far they have to go. Effective programs of rewards, sanctions and assistance should have rigorous performance standards but also should recognize improvement. Schools that typically perform poorly on state tests or other measures may make more progress toward reaching the standards than traditionally high-performing schools because they have farther to go. Recognizing progress gives schools an incentive to continue improving. It was put succinctly at a recent meeting, “It is important to have rewards that everyone has a shot at — not just money, but recognition, too.”

Most SREB states consider achievement gains over a one-, two- or three-year period. Maryland adopted goals that all schools must meet over five years and Kentucky originally set 20-year goals, but both assess progress toward these goals every two years. Some states look at absolute standards as well as gains in student achievement. Tennessee uses a three-year cycle to determine distribution of rewards. Improvements in student test scores are expected to exceed the national rate of increase. Schools also must meet attendance, promotion and dropout-rate goals.

It is not feasible for every school that improves to qualify for a financial reward, but giving schools recognition can increase public awareness of and support for local school-improvement efforts. A department of education staff member described driving through communities and small towns in his state where schools had been recognized: “You will see it on signs everywhere — at the gas station, stores, restaurants, in the middle of town. It is a matter of pride in the community.” The opposite also is true, and public pressure for improvement should result when a community’s schools are labeled “impaired” or “not improving.”

Rewards, sanctions and targeted assistance should be supported with adequate and sustained financial resources. There are no precise answers as to what is “enough” financial support for these programs. Each state needs to determine what it wants to accomplish, what it will cost to get there and what it is able — or willing — to pay.

States often allocate initial funds based on available revenue or on the estimated number of schools that will qualify for rewards or assistance. As more students begin to qualify, policymakers must decide either to increase funding or to spread existing resources and assistance services more thinly. For example, Kentucky “distinguished educators” — teachers assigned to assist in schools with declining student performance — successfully assisted all 53 schools that were “in decline” after the first two-year accountability period. During the next two-year period, 177 schools were eligible for help. Funding specifically for the distinguished-educator program did not increase, but the state Department of Education was able to add some funds from its operations. The legislature approved increased support for the current biennium; funding rose more than 40 percent to $5.8 million this year.

Funding a rewards program can be expensive, and its success often depends upon whether teachers believe the awards provide enough recognition for their effort. State allocations range from $500,000 in Tennessee to $127 million in North Carolina. The amount schools receive varies widely. Rewards in Maryland, South Carolina, Tennessee and Texas range from $500 to $79,000 per school. In Georgia, Kentucky and North Carolina, awards are based on an amount per teacher that ranges from $750 per teacher to $2,300. Florida’s School Recognition Program will provide initial awards late this fall.

Targeted assistance should focus on producing results. In SREB states a school identified as “troubled” or “impaired” usually is required, often with the help of an outside expert, to develop an improvement plan. The school then is given a year or more to improve before sanctions are applied. Additional assistance provided to “low-performing” or “not-improving” schools should focus on instruction and should identify and address specific weaknesses.

Broad assistance not focused on problem areas is unlikely to improve student achievement. One state official found that schools lacked focus in their improvement attempts. They were doing dozens of things to raise student achievement, but none very well. When developing plans for improvement, schools that need help often uncover more problems than they can address at one time. Without assistance targeted toward specific improvement goals, many try to improve everything at once. Eventually these schools lose focus and see few results. One educator responsible for providing assistance said schools in her state plan a host of activities to address a problem “when what they really need to do is implement only those activities that will give the most benefit.”

Targeted assistance should develop the ability of school staff to plan for and achieve continued improvements toward high standards. One goal of additional assistance is to improve student achievement enough to remove a school from the ranks of “troubled” schools. But removing a school from an endangered list is only half the job; the other half is helping teachers learn to keep the school off the list and to work toward higher state standards.

Most teachers have little experience in using data to determine where improvements are needed. Assistance to “troubled” schools should ensure that teachers know what is expected, how to use performance data to identify problem areas, and how to develop and carry out a successful plan of action. Without this focus, the school is unlikely to continue progressing toward high standards.

Content
and Student
Achievement
Standards
+
Testing
+
Professional
Development
+
Accountability
Reporting
+
Rewards, Sanctions
and Targeted
Assistance

Accountability

Looking Ahead

Accountability for student achievement will continue to be a priority for policymakers in the SREB states. As a part of a continuing effort to support state actions in school accountability, the SREB will convene state agency staff and policymakers and their staffs to share information and strategies to work toward solutions to common problems. In the coming years the SREB will focus on supporting state actions to refine and align the five key parts of accountability programs.

Content and Student Achievement Standards

• How can standards be communicated effectively to teachers and parents?
  How can other states’ and national standards and standards review processes be used to develop or improve states’ own content standards?
• How can textbooks and other instructional materials be aligned better with state content standards?
• How can student achievement be factored into evaluations of teachers and school administrators?

Testing

• What are the appropriate roles of state testing and classroom assessment? How can they complement each other?
• How does a state ensure proper test administration and ethical standards in testing?
• What are the best uses of criterion-referenced and norm-referenced testing in a state testing program?
• How are the testing needs and requirements for special-needs students best met?
  

Professional development

• Can professional development be coordinated to focus on state
  content standards and results in student achievement?
• How can professional development connect better to individual schools’ needs identified from student achievement data?
• Can professional development be accessible and convenient to
  teachers?

Accountability reporting

• Do report cards include information on standards and expectations for all students?
• Are report cards being used effectively by schools, parents and communities to improve student achievement?

Rewards, sanctions and targeted assistance

• Are rewards and sanctions clearly aligned with progress toward and achievement of state standards?
• What forms of assistance to low-performing schools work best?
• How can states better help schools learn to continue improvements once targeted assistance is no longer available?