Bright Lines, Big Effects: Unintended Consequences of Student Proficiency Thresholds
Prof. David Figlio
Gordon Fyfe Professor of Economics and Education
University of Rochester, NBER, and Stanford University
Many sectors use discrete classifications of continuous variables to allocate resources, determine services, assign individuals to treatment. These discrete classifications are ubiquitous in education; every state in the USA uses proficiency cutpoints in standardized testing, and states and districts use these cutpoints for grade retention, remediation, course assignment, school accountability, etc. Where we draw the line is highly consequential, but while we know a lot about the consequences of cutpoints for people on either side of the line, we don’t know the effect of drawing a line in a specific place.
We exploit a situation in which there was a year in which (still valid) test scores weren’t available for student assessment due to an implementation glitch associated with the introduction of a new standardized exam in the state of Florida. In this case, the researcher sees the test score that would have been used had educators had access to it. This allows us to observe the counterfactual – what would have happened to kids with precisely the same test scores in testing regimes where information was known to schools versus unknown to schools. Furthermore, simultaneous with the introduction of the new standardized test, Florida dropped some of the “high stakes” consequences of the proficiency thresholds for students. This unusual natural experiment allows us to directly study effects of “bright lines” on the treatment of students and proximate and long-run student outcomes both when the stakes are high for students as well as when the stakes are lower for students.
We find that proficiency thresholds matter – for students on both sides of the cutpoint. The temporary removal of information about student test scores and measured proficiency affected both the students who would have been deemed just proficient and those who would have been deemed just missing the proficiency standard in a remarkably similar manner. We present evidence that educators pay much less attention to important information about student performance during years when the proficiency threshold is observed then when it is not observed, suggesting that the performance threshold substitutes substantially for educator judgment when observed.
