Sunday, June 3, 2001

It didn't take North Springs High School physics teacher Tim Maley and his statewide colleagues long to figure out there was something wrong with the science portion of this year's mandatory high school graduation test.

Maley noticed a "remarkably high number of errors" on a section of the test that stumped thousands of students, keeping many who couldn't pass it --- some who missed by only one answer --- from earning a high school diploma.

"It just seemed like the test was written with bad science," he said. "It's like they did not understand what they were writing about."

Maley estimated about 10 percent of the questions on the science section, which about 30 percent of students fail each year:

> Had no best answer because of errors in information provided to students.

> Had multiple correct answers.

> Were ambiguous or were misstatements of science.

Others who contacted the Georgia Department of Education also cited problems, although not as many.

Department officials acknowledge the acceleration formula and periodic table were wrong, because of a printing error, and two questions were thrown out in scoring the test because of those mistakes. Some other problems, state staffers said, involved questions that were being "field-tested," items not counted but used to see how high schoolers answered them. Such questions can be used on future tests. Officials said the questions would be corrected if they were going to be considered for use in upcoming exams.

David Harmon, the department's testing director, stressed that no students failed the test because of errors in the exam and that complaints this year were no worse than in the past.

Still, such problems raise troubling questions as Georgia and other states prepare to increase their emphasis on standardized tests.

"The testing industry doesn't have the capacity and is not geared up to fill the demands that are coming up that complaints this year were no worse than in the past.

Still, such problems raise troubling questions as Georgia and other states prepare to increase their emphasis on standardized tests.

"The testing industry doesn't have the capacity and is not geared up to fill the demands that are coming up from states right now," said Carl Glickman, a University of Georgia education professor and school reform expert.

The New York Times reported last month that the tiny testing industry is coming off its three most problem-plagued years, just in time for President Bush to push legislation requiring states to test each child in grades three to eight in reading and math. More and more states are using standardized tests to decide promotion or making passage mandatory for graduation. Some are using them to assess the performance of schools.

Under Gov. Roy Barnes and his school reform plans, Georgia is about to do both.

The graduation test --- the bane of thousands of juniors and seniors each year --- will die in a few years, to be replaced by end-of-course tests in several subjects. While Barnes' reform law doesn't mandate that students pass the tests to earn a diploma, it's still unclear exactly how the results will affect students.

Elementary and middle school children have already begun taking new curriculum tests, which will soon be used in grades three, five and eight as a kind of gateway to the next grade.

The testing and grading system is at the heart of Barnes' plan to make schools accountable for educating children. The goal is for the tests to be perfect, but Barnes acknowledged, "Any test devised by humans, you can have an error on it."

Still, he thinks increased testing will make it easier to spot flaws. State School Superintendent Linda Schrenko, usually a Barnes critic, agrees.

"The more you test, and the more you try to use a new version of the test every year, the more the possibility of errors," she said. "The only thing good about this is that by giving it so many times across the state, the chances are greater you will catch them."

Harmon said there are safeguards to assure mistakes don't happen. Questions developed by North Carolina-based Management Inc. are reviewed by committees of subject area experts, curriculum directors and teachers, including panels of educators recommended by superintendents.

Georgia teachers make sure the questions relate to what is being taught in state classrooms and look for things like regional or cultural bias to make sure they can be weeded out, he said. If they get final approval, they wind up in the item bank for possible use on the test. If errors are found, a question is quickly pulled, company officials said.

The two errors the state Department of Education acknowledges on this year's science test were printing mistakes made after the questions had been approved by the committees. They were not in the questions, but in formulas and information provided to students to help them answer questions. When the errors were brought to the department's attention, two questions relating to acceleration were thrown out on the scoring, so students were graded on 68 questions rather than 70.

Harmon said there were also "several" errors on "field test" questions that do not count toward a student's score.

Maley and others said in some cases, all the answers to questions could be correct. In others, there simply wasn't enough information available to answer the questions.

"The entire high school career of our students depends on the outcome of the Georgia High School Graduation Test," wrote Brenda Hodgin, assistant principal of Thomson High School in McDuffie County. "This test is too important to our students to have mistakes in the test.

"Every year we see mistakes, either in the instructor's manual or in the test questions themselves. Yet, the state (department) does not hesitate to allow a student to fail the exam with a score of 499."

Students need a score of 500 to pass.

Harmon said the probability of passing the test was the same with 68 questions as with 70. Most Georgia students who take the graduation test eventually pass all five sections and graduate.

And Georgia's problems appear minor compared to those in other states, such as Minnesota, where scoring errors kept students from graduating with their classmates.

Still, to experts like Glickman, even rare errors are yet another example of why using single tests to determine a child's future is "bad science."

"If a kid gets a 498 in science, does that mean the kid doesn't know enough to graduate if they pass all of their classes?" Glickman said. "Policy-makers say these are just isolated incidents. But it's not an isolated incident if you are the child or the parent. It's devastating if you don't pass this test."

Georgia began its graduation test in hopes that it would assure employers and colleges that a state diploma meant a graduate had learned certain things.

However, thousands of students still enter public universities, two-year colleges and technical schools needing remedial classes in math, reading and English.

Maley said his problem is not necessarily with the idea of a standardized exam for graduation.

"I have nothing against the high-stakes tests. If the population says we need these gateways to make sure we are educating our children properly, we need to provide them," he said. "But they need to be well-thought-out tests. I have seniors who failed the test by one question. If I were a parent and my child failed by one question, I would be very concerned about the test."

ONE QUESTION, MANY ANSWERS
Below is a science question on the state graduation test that North Springs High School physics teacher Tim Maley told state officials was unsolvable with the information given.
With what force will a .25 kg baseball hit the catcher's mitt if it is thrown with an acceleration of 5 m/sec/sec?
A. .05N B. 1.25N C. 5.00N D. 12.50N
Maley's response: The force the ball exerts on the catcher's mitt is independent of the acceleration with which the ball is thrown. To answer the question correctly, the students would need the distance the baseball moved, the time elapsed or the acceleration the ball experienced while in contact with the catcher's mitt. As worded, all answers are possible.

The Atalanta Journal-Constitution

-- Anonymous, June 03, 2001