>Silver Bullets At Moving Targets: Educational Issues in Nevada

>

The quotation “When all you have is a hammer every problem looks like a nail,” is variously attributed to Bernard Baruch or Abraham Maslow, but no matter the attribution, the observation is applicable to political discussions surrounding public education in the state of Nevada (and elsewhere.)  Conservative critics of public education have found a double faced hammer and applied it without much regard for the tool’s efficiency. Face one is that “teacher tenure” isn’t correlative to “student achievement.” The other face contends that “test scores” are an efficacious way to measure the value of educational services.

The first face argument is illogical. Let’s take Michelle Rhee’s comment at face value: “There’s no correlation between tenure and student achievement. We don’t think there’s a need for tenure. There are federal protections from discrimination in place now that afford teachers and all employees those recourses.” [LVSun] If there is no correlation between teacher “tenure” and student achievement — why are we discussing it?  Is it merely because some amorphous “WE” doesn’t think contractual job protections are convenient? Or, is this simply a political question: Republicans don’t like unions, teachers belong to unions, ergo Republicans don’t like contractual job protections for teachers?  No matter what the underlying premise — if we adopt the statement that there is no correlation between so-called tenure systems and student achievement, then the entire debate isn’t germane to the discussion of educational improvement. {previous post}

The second face argument is simplistic. Now, let’s return to Rhee’s proposal for teacher evaluation: “Fifty percent of a teacher’s evaluation should be students’ value-added growth (a measure of a student’s year-over-year improvement on tests). The rest should be classroom observation; student evaluations of teachers; looking at schoolwide gains, so there’s a team component; and then things above and beyond, like reaching out effectively to parents or coaching soccer.” [LVSun]  Some questions were posited in the previous post, and should be explored in more detail.

(1) If 50% of a teacher’s evaluation is based on the scores achieved on standardized tests, then the emphasis in local curricula would (of necessity) be based on the content of the standardized tests. In order for standardized tests to be a useful measure of student performance the tests themselves must be (a) statistically reliable, (2) statistically valid, and (3) conforming to the required curricula to be measured. Reliability requires that the results must be capable of repetition.   Validity is more difficult to assess.  The two minute drill for understanding statistical validity would include the following subtopics: (a) Construct Validity (does the experimentation/test conform to the theory being tested?); (b) does the test cover the variables being studied?; (c) does the test actually focus on the variables? (facial validity); and (d) is the test “externally valid,” i.e. can the results be generalized from one population to another.

We can stipulate that most tests manufactured for use in public schools by major publishing houses are statistically reliable and generally statistically valid. What we cannot say with any statistical certainty is whether or not we are measuring what we value in public education.

What Do We Need To Know?

Consider for a moment the content of a popular e-mail making the rounds since at least 2007 that purported to be a measurement of 8th grade achievement in 1895.  One of the questions is: “Find the cost of 6720 lbs. of coal at $6.00 per ton.” The implied response is implied to be ‘our kids can’t do that!’ However, If we look at this question and the others in the arithmetic section it becomes very obvious very quickly that only basic arithmetic is involved in the testing — there is no algebra, no geometry, much less any hint at trigonometry or calculus — and this purports to be an “exit” examination for “graduation” in an era when the 8th grade was the terminus of nearly everyone’s education. By contrast, the 8th grade NAEP examination for youngsters in 2010 included: 20% number properties and operations, 15% measurement, 20% geometry, 15% data analysis, statistics and probability, and 30% algebra. [NAEP pdf]

In the grammar section of the test a student was asked to “Define Case, Illustrate each Case.”  There are four common cases in modern American English — a noun can be the subject of a sentence, a noun can be a direct object, an indirect object, or a noun can be possessive.   Now what?  I can write: “David threw the ball.” (subject = David.) I can write “Johnny kicked David.” (direct object = David.) I can write “Johnny threw David’s ball.” (David’s = possessive.)  Those tempted to ask, “So what?” are excused.  Case is extremely important if one is constructing a sentence in Latin, in which there are 7 cases, three genders, two numbers, and a plethora of endings attached to let the reader know the case, or function of the word. However, to construct a perfectly understandable sentence in modern American English is it not necessary to know whether a noun is nominative, genitive, dative, or accusative; much less if it might be ablative, vocative, or locative.

Only in the last question on the supposed 1895 exam did the test sound remotely modern, when it asked the student to write a 150 word essay demonstrating competence in written English. At that point it would be revealed if the individual taking the test knew that SYNTAX, and not necessarily grammar, is at the root of modern communication in English. The meaning of three words strung together, like ‘man bites dog,’ is not determined by the endings attached to the three words (denoting cases), but by the order in which they occur in the sentence. Thanks to invasions by at least three Germanic tribes, several Viking clans, numerous Christian missionaries, and William The Conqueror’s retinue, we have a language that has soaked up vocabulary with the same rapacity as it has discarded complex grammatical rules.  If we are truly concerned with competence in written English, then our testing should be valid in terms of what is (or is not) essential to know in order to communicate.

We no longer require youngsters to “parse” sentences, but we do require that 8th graders in Nevada be able to “analyze elements of plots, differentiate between plots and subplots, explain the use of foreshadowing, explain methods of characterization, draw inferences between and among themes in a written example, analyze an author’s point of view, explain how figurative language is used, and compare and contrast the use of mood and tone in written samples.” We also want to know if Nevada 8th graders can identify the main idea and supporting evidence in primary documents, scientific explanations, speeches, interviews, and proposals. Additionally, Nevada 8th graders should be able to recognize persuasive techniques within a text including “transfer, unfinished claims, rhetorical questions, loaded language, appeals to logic, and appeals to ethics.” [NDoE pdf]  The reader is now left to ponder: Which is more important? To be able to diagram a sentence using either the X-Bar or Kellogg-Reed schematic? Or, to be able to identify the main idea and supporting evidence in a text of a speech or editorial?

Given the increasing complexity of what is now required of 8th graders beyond the recognition of “parts of speech,” and basic arithmetic, we need to add another layer. We could test for competency in reading, mathematics, science, and civics and be happy to compare and contrast student achievement (value added?) from one school term to the next. However, if we base 50% of a teacher’s evaluation on test scores in these areas what might be the implications for other subject matter areas?  Do we really want a state-load of 8th graders who don’t know the difference between a saxophone and a trumpet?  Who wouldn’t recognize the Mona Lisa if it were hung in their living rooms?   Who have absolutely no idea of the difference between a penalty kick and a free kick in a soccer game? Who can’t name major food groups and put food items in the appropriate categories? Who can’t differentiate between micro- and macro- economics?  One thing we can guarantee is that if a teacher’s evaluations are predicated on test results in specific subject matter categories then those categories will receive the most time, effort, and attention in our classrooms. When the test determines the teaching, the teaching will resemble the test.

(2) If the gains are to be measured YOY, then how do we interpret results from students who transfer between schools, or between school districts YOY? Or, for that matter, how do we evaluate the results on standardized tests from students who attended a school in another state during the past school year?  It’s probably reasonable to conclude that migrant students will be difficult to track, and that schools with a high percentage of migrant students may show very different test results from those schools which do not have a high percentage of enrollment in that category.  Is Teacher A, in a school with a high migrant population, less “effective” than Teacher B, teaching in a school with a low percentage of migrant students? The test results could be skewered by Teacher A emphasizing in her classroom items to be tested locally, items covering subject matter or skills not emphasized in the teaching/testing programs in the school in which Teacher B is located.  The easiest resolution would be the application of nationwide educational knowledge and skill standards to all age and subject matter groups — however, this solution flies in the face of efforts to leave the control of public schools in the hands of local boards of education.

(3) If YOY improvements are required, then it’s not just that the student would score better on a 4th grade level reading test a year later, but that the student’s score must improve on a 5th grade level test?  The problem here is perhaps best illustrated by the old visual trick of the expanding money bags on charts.  If we want to illustrate the price of milk in 1900 (about 7 cents per quart) in comparison to the price of milk today (about 92 cents per quart national average 2010) we’d see an increase of 85 cents. That is a 1214% increase. In order to be a “valid” graphical reference the graphic for the second price (92 cents) would have to be 1214% larger than the initial money bag (7 cents). But! To keep the proportions of the money bag constant in the graphic the visual increase must be about 4 times larger. [HTL]

By year over year improvement, do we mean that a student would perform better on a test given in both one year and then the next? Or, do we mean that the student would have to perform better on a new test given a year after the first, different one?  How much “harder” is the second test than the first one given?  If our hypothetical student scored a 90% on his 5th grade examination and a 90% on his 6th grade one has there been “no” improvement?

(4)  If standardized test scores are to constitute 50% of a teacher’s evaluation, then how do we apply this to teachers in Special Education classrooms? Especially in those classrooms serving profoundly handicapped youngsters?  In 2008 there were 48,332 youngsters counted in the “students with disabilities” category in Nevada. [NVDoE pdf]  52% had learning disabilities, 16% had speech impairments, 4% were diagnosed with mental retardation, 4% with serious emotional disturbances, 5% with autism, 6% with hearing impairments, 2% with multiple impairments, and 7% with developmental delay problems. About 80% of the these youngsters spend about 40% of their classroom time in “regular educational environments.”  The question that first asserts itself is: If a disabled pupil spends 40% of his or her day in a “regular” classroom, to whom do the test scores accrue? If we are to determine 50% of a teacher’s competency based on test scores, then how do we calculate that percentage if the pupils in question are incapable of taking, much less passing, a standardized test of knowledge and skills?

And, then things can get really messy?

According to adviser Rhee, test score based evaluations could be enhanced by adding extraneous matters, “and then things above and beyond, like reaching out effectively to parents or coaching soccer.”   OK, if Mrs. Blankfire makes one call every night to one parent of each child in her 2nd grade classroom during one month, does this count for more, or less, than Mr. Drainpipe’s exhaustive weekly 6th grade newsletter?  Is Mrs. Blankfire’s coaching schedule for T-Ball more, or less, “effective” than Drainpipe’s Chess Club?  In the mean time, what are we to do with Mrs. Breakwater, who having six children of her own to raise, doesn’t have time (or the money) to spend on either T-Ball or Chess? But, Breakwater’s student work is always graded more promptly than either Blankfire’s or Drainpipe’s?  Pity the school administrator who has to sort through these issues.

Exactly what DO we mean by “reaching out?” Must the reaching be directed solely in the direction of parents of children enrolled in the classroom?  What if one of our hypothetical teachers decided that an appropriate method of connecting with the community was to volunteer at least one night a week to read to children in a shelter for abused women and their children?  To spend Sunday afternoon at the local soup kitchen? To volunteer in voter registration drives? To meet with unemployed adults who needed to improve their skills in job interviews?  How do we construct the metrics? Does one call to a parent of a child enrolled in a teacher’s classroom equal four Sundays spent at the Soup Kitchen?  Does one season of T-Ball equal one entire year of coaching job interviews?  What happens if the volunteering is ever so slightly more controversial? How do we measure the “community outreach” of a person who serves as a volunteer for PETA? For NARAL? Do we inquire if the teacher attends a church, a synagogue, a mosque, or a temple once a week? Just what are those things “above and beyond” that we want to incorporate into a teacher’s evaluation?

No Silver Bullets

Illogical and simplistic suggestions are not solutions. We aren’t going to “solve” educational problems by first ignoring the complexity of the learning process, and then compounding the errors by asserting quick fix suggestions as received wisdom. There are differences between and among school districts concerning what should be taught, and within that context between and among what specific knowledge and skills should be measured.  Without a standardized national curricula (not necessarily a good thing) there can be no reliable or valid measure of educational achievement on a national level. Nevada school districts can agree upon a statewide curricula, but questions of “how, and how much,” remain components reserved to local educators. As long as there is local control of local school districts there will be elements of emphasis creating disparities which, in turn, create variance in test score results. 

Local public schools in Nevada take on everyone, from 7-18 years of age, regardless of variables like the value of education in the home, the living arrangements and environment, the social-economic status, the financial capacity to engage fully with educational opportunities in the community, and a host of other social factors.   We might assume that a given percentage of youngsters coming from a home in which “pointy-headed intellectuals” are disparaged nightly aren’t going to find school a pleasurable place to be. We can assume that if a parent never asks about homework — there will never be any (according to the little scholar at least.) We can assume that parents who are both working two jobs to maintain the family finances aren’t going to have a great amount of time to even ask about the homework much less monitor it. We can assume that employers who don’t grant time off for attendance at parent-teacher conferences aren’t helping the situation. We can assume that parents who are more interested in their student’s shooting percentage than in the SAT score aren’t contributing as much as they could either.   Neither eliminating “tenure” nor improving “test scores” are the Silver Bullets altogether too many pundits are parading as “solutions.”  Ignoring out-of-school factors won’t make them go away.

Local public schools enroll pupils who were low birth weight babies or who have significant non-genetic prenatal influences on their learning capacities. If we are truly concerned about “improving” educational achievement, then scrimping on pre-natal care opportunities is counter productive. Children arrive with inadequate vision, dental, and medical care. Ignoring increased access to medical intervention is also counter productive. Some children will arrive from homes with what is politely called “food insecurity,” we could return to Maslow’s Hierarchy of Needs, and note that physiological needs must be the first satisfied if “self actualization” is on the agenda. It’s hard to learn without sleep, food, and health.  Are the kids coming through the classroom door from neighborhoods polluted by industrial contaminants?  Are they entering after enduring a night punctuated by family relations problems and stress?  Are they bringing with them dysfunctional attitudes and issues from the neighborhood in which they reside? [NEPC] These are not excuses for poor educational performance, but these factors are indicative of how difficult the education process can be, and how formidable the challenges are to statistically measuring the performance of either students or teachers. There are no silver bullets.

Comments Off on >Silver Bullets At Moving Targets: Educational Issues in Nevada

Filed under education, nevada education

Comments are closed.