About the author
Eva Travers, professor of education at Swarthmore College, is working with Research for Action's "Learning from Philadelphia's School Reform" - a research and public awareness project examining key issues associated with Philadelphia's current wave of education reform.
Achieving 'proficiency': the devil is in the details of state plans
by Eva Travers
Item: A child is an above average student in math but fails to score "proficient" on the state standardized test. Yet if the same student had taken the state test in a neighboring state, achieving "proficiency" would have been a cakewalk.
Item: In Florida, only 24 percent of schools statewide met the annual targets for school performance that are now required under federal law in every state. Next door, in Alabama, not renowned for high quality public schools, 95 percent of schools met the state's school performance targets.
Welcome to the inconsistent world of the No Child Left Behind Act (NCLB), which had been billed as a tool for educational equity.
In 2002, the federal government, with President Bush's signature, adopted NCLB, a reauthorization of the Elementary and Secondary Education Act. The stated goal of NCLB is "to ensure that all children have a fair, equal and significant opportunity to obtain a high quality education."
NCLB includes uniform language, regulations and consequences for all schools, but the implementation of NCLB requirements is left to individual states. Thus, "the devil is in the details" of how states interpret and respond to the federal law.
A key feature of NCLB is the requirement that by spring 2014, 100 percent of all students in public schools reach academic "proficiency" on state assessments, aligned with state standards in math and reading (and science beginning in 2006-07). To ensure that "proficiency" is reached by all students within a school, test scores must be reported for "subgroups" - broken down by race or ethnicity, as well as for economically disadvantaged, special education, and English language learner students - so long as each of these subgroups" comprises a certain minimum number of students in a school.
NCLB requires yearly reporting by all schools and all states of the percentages of students that achieve at least three levels of achievement: "basic," "proficient," and "advanced." The federal government publishes proficiency rates by state each year. On the surface, there is uniformity in the federal reporting procedures.
Variations in state tests
But state-to-state comparisons are problematic because the federal legislation grants each state considerable flexibility in designing important details of their testing and accountability plans. While flexibility in developing plans that meet the needs of students in various states is good educational practice, comparing the number of students reaching "proficiency" across states becomes meaningless.
States use different tests to assess "proficiency" - most of them designed to match their states' particular academic standards. The difficulty of state tests varies considerably, when measured against a common assessment yardstick.
Here is one stunning example of how much easier some state tests are than others: eighth grade math students who score at the "proficient" level in Montana score on average at only the 36th percentile on a nationally normed test called the NWEA, while "proficient" eighth grade students in Wyoming score at the 89th percentile on the NWEA.
Pennsylvania's state test, the PSSA, is considered to be rigorous relative to many other state assessments; according to Michael Casserly, director of the Council of Great City Schools, "The PSSA has a reputation as one of the nation's more challenging assessments."
Subjective performance levels
The method used to determine performance levels for standardized tests, including the key "proficiency" designation, is subjective. It is based on the combined judgments of teachers, content experts, parents and/or representatives from business and higher education in each state. They determine what the cutoffs, or "cut scores," will be for "below basic," "basic," "proficient," and "advanced" on their state tests.
In Pennsylvania, "cut scores" were deliberately designed to emphasize rigorous achievement levels. The state added a small statistically based increase to the "cut scores" determined by the group that set the initial performance levels.
Another source of great variation among states is that states had flexibility in determining the schoolwide level of proficiency initially required to achieve "adequate yearly progress" (AYP), as well as the proficiency rates required in subsequent years to move schools toward the goal of 100 percent proficiency by spring 2014.
Some states have set their "proficiency" standards for schools at high levels; others have set the bar lower. In Pennsylvania in 2004-05, the bar is to be raised: 54 percent of students at each school will be expected to be "proficient" in reading and 45 percent in math.
Rules on subgroups vary
Finally, each state has flexibility in determining how many students in a school are needed to constitute a "subgroup" for purposes of reporting scores.
In Pennsylvania, 40 students are required, in one of 13 possible subgroups identified by the state, before a school is held accountable for the scores of that subgroup of students. Most states require 20-40 students to constitute a subgroup for purposes of reporting scores, though the number can be as low as five or as high as 100.
The smaller the number of students a state requires to constitute a subgroup, the more subgroups must be reported and targets must be met, and the harder it is to reach AYP.