A new front is opened in the war on standardized testing.
Seven HISD teachers and their union are suing the school district to try to end job evaluations tied to students’ test scores, arguing the method is arbitrary, unfair and in violation of their due-process rights.
The lawsuit, filed in federal court late Wednesday, could have far-reaching implications as more districts and states use student test data to grade teachers.
The Houston case focuses on the district’s use of a specific, privately developed statistical model that analzes test data to try to gauge a teacher’s effectiveness.
In some cases, according to the lawsuit, teachers see their scores fluctuate from year to year, while other results are based on tests not aligned to the state curriculum. The lawsuit also argues that all teachers aren’t treated equally, and they can’t adequately challenge their ratings because the formula is too opaque.
For example, the lawsuit says, Andy Dewey, a social studies teacher at Carnegie Vanguard High School, received such high scores in 2012 that he qualified for the district’s top performance bonus; his results the next year dropped significantly.
“Mr. Dewey went from being deemed one of the highest performing teachers in HISD to one making ‘no detectable difference’ for his students,” the lawsuit said.
Dewey has told the Houston Chronicle previously that he does not understand why his scores vary when he and his fellow social studies teachers — they are rated as a team — don’t change their instruction significantly from year to year.
[…]
The system at the center of the lawsuit generally is called “value added.” It uses complex statistics to estimate how well students should perform on standardized tests based on their own past performance. Teachers whose students score better than expected get the best ratings, whether or not the students passed the test.
To do the analysis, HISD contracts with a North Carolina company, whose model is called the Education Value-Added Assessment System, or EVAAS.
You can see a copy of the lawsuit here, the press release from the AFT is here, and some background is here. The Texas AFT has an illustration of the EVAAS formula here. I am not opposed in theory to the idea of value-added evaluations. This is basically what the sabermetric revolution in sports has been all about, coming up with ways to measure performance and determine the value of players in various sports. In sports, however, the relationship between the various metrics – runs created, points per possession, DVOA, etc – has been demonstrably linked to the teams’s actual on-field performance. They also show what sort of things a given player needs to do in order to be valuable. Finally, there are multiple systems that have been created to measure value, and they have risen or fallen based on their usefulness and accuracy. I don’t know how much any of this is true for EVAAS. I do know that teachers should have a clear understanding of what is expected of them, and they should have some input on their evaluation. I’ll be very interested to see how this goes. The Trib and K-12 Zone have more.
A system created by a professor at NC Stat echo seems to have links to Grier. For what it is worth he NSF argued the method was flawed.
A system created by a professor at NC Stat who seems to have links to Grier. For what it is worth the NSF argued the method was flawed.
Texas Business Evaluation Model (TBEM)
Today Governor Rick Perry announced a new plan for evaluating businesses in Texas. It’s called the Texas Business Evaluation Model or TBEM. The TBEM was created by the Business Evaluation Study Team (BEST). This assembly of outstanding individuals was brought together 18 months ago to create a system to validate the effectiveness and efficiency of Texas businesses. BEST members represent interests from many constituencies. These include state and local government, PreK – 12 education, higher education, and the judiciary. The Texas Business Evaluation Model, a first in the nation effort, assigns a letter grade (A – F) for each business based on a complicated system of metrics. The grade assigned to the business entity would be derived from the sales of a specific date in the first quarter of each calendar year. The metrics will factor in data from sales and compare these sales from one year to the next. The initial year of evaluation will set a base line and will not be used to assign a grade to the business. However, a grade will be assigned in subsequent years. This model should improve competition as it will be mandatory for all registered business entities in the state. Business owners who object to this new program will have their licenses revoked and be required to liquidate their assets, immediately. In the second year of the TBEM the results of the evaluation will be widely publicized in local and national media outlets.
Does this sound like a good idea?
The level of testing they are doing now and the teacher evaluations is like applying sabremetrics to kids playing t-ball. Yes statistics have a place but at these age levels more focus should be on subjective criteria – do the kids like the teacher? Are they learning? Are they improving their test scores? But don’t base employment / bonuses on kids test scores.