>NCLB: The Untestable in Full Pursuit of the Unattainable?

>
Senator Harry Reid (D-NV) will host a town hall meeting with teachers, administrators, and parents on how to improve the No Child Left Behind Act, Wednesday May 30th at 3:30 pm at Hollingsworth Elementary School (Las Vegas). Video conference equipment will extend participation to Carson City (Nevada Department of Education) and Elko (Great Basin College). (contact 702-388-5020) {Reid Press Release}

Every once in awhile the AFT and the NEA agree on something — both consider NCLB a mess of abysmal proportions. The AFT notes that: “Guidance for states has been unclear, untimely and unhelpful, and the U.S. Department of Education’s attempts to make the law more flexible have brought about only minimal improvements without addressing NCLB’s larger flaws. Underlying all these issues is the pervasive problem of funding, which is far less than what was promised and far less than what is needed.” [AFT] The funding philosophy doesn’t reflect a “carrot and stick” approach as much as it does a “two by four or the whip;” and failure to make “adequate yearly progress” seems analogous to Charlie Brown’s failure to get Lucy to hold the ball for his place kick.

Under the present guidelines schools are given a plethora of ways to fail, and only one way to succeed. Underpinning these “evaluations” are two relatively simple statistical concepts: validity and reliability. No one has yet offered me what I consider credible evidence that the national standards meet those two criteria. In fact, there are some serious analyses that question the reliability and validity of standardized testing, especially for youngsters with limited English skills. [MAEC] One doesn’t need to be an expert to note that norming bias is a real problem if most of the children tested in the sample are English proficient, and only a small number are not.

If the national standards don’t meet the tests for validity and reliability, how then are the states supposed to do so? This may also help explain why some schools may be meeting the state criteria and yet failing to meet federal standards. [NEA]

The NEA’s solution is to allow schools to use more than standardized test scores to measure student performance. One of the more obvious problems is test attendance. Under the present system if a school doesn’t have 95% of the students taking two standardized tests it can “fail to achieve adequate yearly progress.” This has led to some curious results like those of Ridgewood High School, New Jersey, with an average SAT score of 1174, being declared “in need of improvement.”

Alternative programs and schools with high numbers of students who have limited English proficiency illustrate other problems with the standardized test evaluation system. Gray Elementary School in Las Vegas was declared in need of improvement because 36 non-English speaking (and reading) youngsters who had only been in the country for a short time were included in the results. [NEA] Unfortunately, there are several other examples including an Oregon high school penalized for providing an alternative education life skills program for developmentally disabled kids who tend not to perform well on standardized tests.

The exclusive emphasis on standardized testing recalls that about once per decade the American public is treated to something akin to E.D. Hirsch’s “Cultural Literacy,” in which we are told that if the kids would just memorize 5,000 “facts” success will be a natural result. There are two problems with this assessment: (1) The assumption that there is always 1 right answer; and (2) The assumption that there is 1 right way to teach it. [Trumbell] Hirsch is just a half step from Robert Hutchins’ “Great Books” curriculum, a list of books by white males for white males. Adler and Van Doren updated the list in 1972 but added precious little to the diversity of thought and talent included. [Wik] Jane Austen’s Pride and Prejudice makes their list at #91. William Wells Brown, Frederick Douglass, W.E.B. DuBois, and Booker T. Washington are nowhere in sight.

Given the propensity for ‘reformers’ to attach themselves to Great Lists, of facts, books, skills, or concepts, standardized test drafters need to be especially cognizant of Content Bias problems. Those problems have not been addressed with Navajo and Hopi youngsters [ERIC] and other Native American groups haven’t fared much better. [ERICdigests] Signithia Fordham’s study of African American youngsters is even more troubling; observing that both the test and the testing process are skewed in ways that diminish the actual achievement of African American children. [RTS]

Either the test makers must design and deliver standardized tests that are free of content, norming, and cultural biases, and which are both statistically reliable and valid — or, if this is impossible, alternative assessments must be provided. If there are no alternatives to a biased testing program then what the public is getting is good old fashioned GIGO, “Garbage In, Garbage Out.”

This situation may say more about the people who think NCLB is working than it does about the schools and students being evaluated. [ABC] If the assessment drives the instruction, but the assessment itself is unreliable and invalid, then what does that make the instruction? If schools are working to validate the assumptions of intrinsically invalid measurements how do we assess progress? — Toward what end?

Proponents are fond of inserting “accountability” into the discussion as often as possible. However, “targeting the money to reduce the achievement gap” [EW] depends entirely what’s being measured. And, here we go again — if our measurements are unreliable and/or invalid then what gap are we reducing? Armed with our highly questionable measurements of student achievement we nevertheless charge forward into the accountability challenge to get kids to do better on their tests — with “scientifically based methodology.” The Department of Education has made it perfectly clear that the Federal government prefers phonics instruction for reading: Ideology trumping Methodology. This fight has been going since Rudolph Flesh published Why Johnny Can’t Read in 1955.

Those who have vivid (or is that Livid?) memories of basal readers — See Spot Run — with carefully controlled phonics based vocabularies will recognize why the pendulum did a swing toward whole language instruction. Further, phonics can be a frustrating method for youngsters who come from households that don’t possess much reading material in the home. [NAU] This hasn’t stopped the march toward the Ideal Decodable Text, i.e. a reading book 80% of which contains so-called decodable words. That phonics instruction helps doesn’t seem to be the issue. It’s how much phonics must be emphasized. There’s just one problem with this approach: Research hasn’t determined the number or the percentage of text pages that must be “decodable” to be effective. [TOR] The recent questions about the Reading First program’s questionable conflicts of interest and contracting issues don’t make this discussion any more constructive. [DB] And, to make matters even more opaque, at least reading instruction has templates for “whole language” and “phonics based” instruction, there are no such applicable templates for science, math, and social studies.

Embedded in NCLB are issues of the un-measurable being taught via the unknowable for the benefit of the unwilling — which brings to mind Oscar Wilde’s definition of a fox hunt: “The unspeakable galloping in full pursuit of the uneatable.”

Comments Off on >NCLB: The Untestable in Full Pursuit of the Unattainable?

Filed under education, NCLB, Reading First, Reid

Comments are closed.