The U.S. Chamber of Commerce (ICW) doesn’t think much of the higher educational systems of Nevada, Louisiana, and Idaho — these states received “F’s” in its report card. [NNB] It’s not that these three states don’t have some work to do to improve their public institutions of higher education, they do. However, even a cursory glance at the technical appendix demonstrates the problem with research for which the desired outcome is determined before the data arrives.
One quick example: Nevada receives low scores for “student access and success,” a portion of which is based on the percentage of Pell Grant recipients — the technical notes don’t explain whether this is a positive or negative factor. In terms of 4 yr. institutions, the state is then “rated” on completion rates, retention rates, and credentials produced. The “credentials produced” is problematic:
“We developed a simple weighting scheme that reflected information about the cost of graduate instruction and typical program lengths. For simplicity, we assumed that graduate instruction is, on average, about twice as expensive to deliver as undergraduate, upper-division courses. The weights are therefore equivalent to two times the ratio of the coursework required for graduate degree to the program length of a BA. While we acknowledge that different disciplines are more expensive, we did not distinguish between program types.” [ICW]
That last sentence should be a reminder about why assumptions should be carefully considered before statistics are swallowed without chewing. The assumptions also color how different “credentials” are weighted. “Research” PhD’s (not defined) are weighted at 1.5 times a B.A, while “professional “PhD’s or “first professional degrees” are weighted at 2.0 times the B.A. The assumptions are clearly skewed towards the land of Cost Benefit Analysis.
At least they were honest about it:
“We readily acknowledge that this weighting technique is not perfect. In the absence of more granular cost data or a consensual method for weighting degrees, however, we believe this approach is reasonable and simple to understand. Other researchers should experiment with alternative weighting schemes.” [ICW]
It’s simple, whether or not it’s reasonable is debatable. The first obvious question should be: Why are the degrees weighted in the first place? Why is a Master’s degree weighted at 3/4ths of a BA? (Because it’s twice as expensive to provide, but takes 1/4 to 1/2 as long to complete?) Thus, an institution with a good reputation for teacher training, which graduates a significant portion of its enrollees with Master’s degrees in specialized fields such as speech therapy, educational psychology, special education, and/or requires a subject matter BA/BS before attaining a teaching credential in a Master’s program, would be weighted below an institution which emphasizes BA/BS and PhD programs?
Granular cost data really isn’t going to be all that helpful, especially not if the point is to put subject specialists in secondary school classrooms. Or, if the point is to produce individuals with Master’s level education in bio-medical technology fields.
There are criteria by which “failing” institutions may be rated, but ratings based on variations on cost benefit analyses are particularly troublesome:
“Even when conducted with the best of intentions, the method is still problematic, because it substitutes calculation for informed and considered judgment. Although we need not abandon such analysis altogether, we must recognize how and why it is subject to misuse and abuse.” [Strategy+Business]
The problem with applying any form of cost benefit analysis is that it reduces the value of any given proposal or variable into economic terms, as if we could rationally monetize benefits and risks in a mathematically neutral way. We can’t.
(1) “But quantitative analyses are never neutral. To be useful, any data, including economic data, must be considered in the context of the decision that is being made. Also, no matter how clever the mathematics, certain key inputs in a cost-benefit analysis cannot be translated into economic value. ”
(2) “…analysis is an act performed by human beings. Complex systems and modeling experts have long asserted that human beings operating in isolation or within the mental models of their professional training simply aren’t able to be objective enough to come up with the right answers to the kinds of problems that cost-benefit analysis addresses. Certainly, within tightly defined boundaries — when they practice the scientific method, for instance — people can approximate objectivity. Yet even then, they cannot help making value judgments at every step. ” Translation: People make the models and then people make judgments predicated upon the models they’ve created.
(3) The data is almost never complete. In the Chamber of Commerce “study” for example, they “regret” the absence of granular data, but then proceed to draw conclusions as if the data were available.
Some important questions are left unanswered by the USCOC’s attempts at a rating system. For example, might tuition rates be reduced if more state funding were available? Is it the function of institutions of higher education to focus on providing the private sector with a ready-made work force? Or, is the function of a university to conduct independent research toward improving the knowledge base and informational environment for the general use of the private sector businesses which may avail themselves of the research products?
One other problem illustrated by this generalizes rating system is the lack of emphasis on the economic impact of the education received on the individual. The Technical Appendix tells us that 2 year institutions had their certification programs weighted to 3/4th or 1/4th of an Associates degree. However, if one’s prospective employer wants a person with a 48 week training program in Diesel Technology — then the value of that certificate is of greater value to the individual seeking the job than an AA degree which may or may not increase the pay level than the certificate.
There is some good news in the offing. The upcoming budget from the Sandoval Administration will contain “flat budgets,” i.e. no more cuts to education. [KTVN] The bad news is that a flat budget does nothing toward back-filling previous cuts.
One might hope that the Chamber wants to spur Nevadans toward more funding support for the University of Nevada System, however a person could probably bet that the failing grade will be further “proof” that public sector education is “failing” and therefore the institutions should be “punished” by seeing further reductions in public support.
Perhaps one of these days the notion will catch on that education has VALUE, the price of which we will never be able to capture statistically. And, all the bi-variate regressions in the world will never go much further in the direction of objective results than asking alumni if they were pleased with their educational opportunities.