School Beat: WASB: Melissa Hammann reports: Student Achievement Data Update

(Ed.note: This essay is the last in the book of Melissa Hammann titled, "School Issues That Count.)



Student Achievement Data Update: January 2008

A strong reason for attending the Wisconsin Association of School Boards Convention on Wednesday was to observe the Education Forum presentation by Dr. Jeffery Braden entitled “A Policymaker’s Guide to Student Achievement Data.” I was impressed with the generally high quality of the speakers at the convention. Dr. Braden was fabulous, and he had plenty of handouts. I learned quite a bit about the origin or our own Wisconsin Knowledge and Concepts Exam (WKCE), the variety of test protocols available to measure student achievement and, the biggie, whether or not the tests actually do measure student achievement. Here’s some highlights learned from Dr. Braden’s presentation.

The Wisconsin Student Assessment System (WSAS) has 4 Goals.
Provide clear expectations of standards for student learning.
Provide student achievement data relative to the expectations or standards.
Use assessment methods that promote high quality curriculum and instruction.
Provide feedback to students, teachers and parents to assist in educational planning.
The great drawback of the WKCE has been and continues to be the timing of #4. Feedback in May from testing in October is essentially useless in the goal of assisting in educational planning for the school year the test is taken. Educators have no way to direct their classrooms toward areas in need of improvement because they don’t know what those areas are until the end of the year. Our district recently instituted MAP testing to alleviate this issue. It provides immediate feedback, is a growth model test and aids in individualized (differentiated) instruction. The Evansville School Board is drafting a proclamation to the DPI urging them to replace the current WKCE model with growth testing protocols. The WASB had this in their resolutions as well. Dr. Braden is on a committee working to encourage the federal government to utilize such value added growth model testing as the standard in the federal No Child Left Behind (NCLB) policy.

An interesting point addressed in the discussion regarded the classroom time spent taking the WKCE. This is a frequent complaint of many stakeholders in this process including parents, students and teachers. However, the most time intensive tests at grade 4, 8 and 10 only take 1.5% of total class time, with the other years in grades 3-9 averaging less than 1% of total class time. Dr. Braden compared that to the industry standard of 18-22% of time spent completing performance evaluations and 4-5% of the workday spent measuring product outcomes (quality assurance). That was an inspired comparison, which helps non-educators understand it in familiar terms. This analysis only included classroom time TAKING the test. Test preparation time is not included. Dr. Braden emphasized that it has been determined that time spent in class specifically on test preparation doesn’t pay off. A well-functioning school that teaches to the standards to begin with will perform well on the test.

The discussion finally got to the point where proficiency levels and their origin were discussed. This was fascinating to me. How good is good and who says so? Wisconsin has recently taken quite a beating in the press about their lax standards and all was explained here! The levels of proficiency were determined by a method called “bookmarking.” Experts for each subject were gathered by grade level and provided with bookmarks labeled Minimal, Basic, Proficient and Advanced. They were given a book with questions in their subject arranged in order from easiest to hardest and asked to place each bookmark where they thought each performance level should fall in the order of question difficulty. Naturally, there were differences in where this point was for each expert, so the next step was to get together and agree where the levels should be placed. This was done for each grade level from 3-10 for each subject tested within the WKCE. Wisconsin got into trouble when they ran numbers to evaluate where our schools would fall within the context of this first “bookmarking” session. Apparently, it wasn’t pretty, with only about 10% of the students scoring proficient or advanced using the original bookmarks. So, in a classic political maneuver, they went back and changed the bookmarks to inflate the scores. A little later in the presentation, Wisconsin’s performance on the WKCE was compared to it’s performance on the National Assessment of Educational Progress (NAEP), a National Test. The most bleak data shown were the 8th grade reading data: 2006-2007 WKCE showed 85% proficient or better where 2006-2007 NAEP showed only 33% proficient or better. I commented, “So that means that the teachers and experts who participated in the original bookmarking sessions in Wisconsin were right!” Dr. Braden agreed, remarking that since there were no National numbers to run, there were never any inflated bookmarks created for the NAEP test. Further information comparing WKCE scores with NAEP results can be found by Googling “National Report Card.”

This score inflation bodes no good news for Evansville. Our performance on WKCE test has waned over time. 85% proficient is really 33% proficient and Wisconsin sold out the true measurement to get those crucial federal dollars. As we tick toward 2014, when, of course, 100% of all the children in America will be proficient or advanced, I reckon the definition of proficient will evolve to mean mediocre.