James Ray Watkins, Jr.
First, against the WCE: Does anyone really believe that something as elusive as good writingmuch less the ability to writecan actually be precisely measured like atmospheric air pressure? Could any of us actually write well on command at a prescribed time and place not of our choosing? What does an exam result, however objectively determined, really tell us about pedagogical effectiveness? The awful history of standardized tests is no secret, and the long-standing attempt to create a more humane and effective way to represent student learning a worthy goal. Eliminating the WCE seems a small but necessary step in the right direction.
Second, against the EWP: Given that the technology and support personnel needed for the EWP may well never be an administrative priority, why should I agree to another thankless task? The EWP is little more than a system of collecting texts; worse, its a collection without discrimination, seeming to imply that all students writing is equally successful as writing. Standardized testing may be limited, but the EWP, disastrously, abandons the idea that writing can be judged effective or ineffective; it's part and parcel of a kind of cultural relativism that I can never abide. If all writing texts are equal in our institutional eyes, then what are we trying to do in the classroom? Do we want students to feel good or do we want them to learn? The EWP is a bureaucratic nightmare at best; lets go back to the WCE, which at least has the virtue of doing no harm.
If all of this seems a bit overwrought, well, I hope that hyperbole will underline just how much is at stake whenever we attempt to reform the way we assess student performance. This debate really matters to us, personally and professionally, in a way many of our other controversies do not. Suffice it to say that my representation is less of an exaggeration than many of us would like to believe. I want to be clear about my own position, too, so let me take a moment to articulate my thinking on these matters.
It seems obvious to me that assessment is both desirable and possible. I have a strong sense of what good writing looks and sounds like, even if I have a hard time articulating exactly what I mean when I make that judgment. If my students leave with nothing else, I want them to leave with some sense of what I mean when I say this is or is not good writing. As someone who believes strongly in the new technologies of writing, nothing makes me angrier than the cowardice of administrations unwilling to make the financial commitment necessary to make these technologies effective and useful. I certainly would never support the corrosive idea that all values are equally valid. Fighting a war in order to secure oil contracts, for example, seems to me wrong in just about all times and places.
Finally, while self-esteem is a wonderful thing, I see my teaching as centered on helping students think. I am trained as an intellectual, not a psychologist. I would never advocate returning to a time in which thought was radically divorced from feeling, but that is not the same thing as saying that my main task is to help students emotionally. Call me old-fashioned, but I believe that a pedagogical emphasis on self-esteem tends to encourage narcissism, and to obscure what I consider central to education: that is, comprehending the challenges of obligation and responsibility that are vital to the future of an affluent culture. Individualism without the courage and sacrifice of commitment is, well, everything that I see wrong with modern American capitalism in a nutshell.
Whats more, I think the EWP is historically significant and potentially one of the most effective educational reforms Eastern has ever attempted. I think we all ought to be fighting for it's future with as much energy as we can muster. If we were to get organized in support of it, I believe it could both make our jobs easier and help to make our teaching more effective.
Even more, the EWP should become one of our most prized accomplishments at EIU, something that could bring us national attention and acclaim. We may not have been first, but we were early and we have bragging rights.
I want to argue, in other words, for a third, positive position that would reject the WCE for the reasons cited above, while recognizing the limit's of the EWP as it currently stands. My hope is that in doing so I will contribute to generating the political support the EWP needs in order to succeed in becoming an effective adjunct to our teaching. In my view, the EWP is the best way to institutionalize our values and an effective way to support the teaching of writing. In order to explain that position, I have to cast a pretty wide net, so if you will be a little patient with me, I would like to do that now.
One way to think about the EWP/WCE imbroglio is to ask which model is most comparable to the kinds of assessment students are likely to encounter outside of school: tests or portfolios. Imagine if we suddenly advocated a WCE-type assessment system for ourselves. Whats good for the goose is good for the gander, as my mom often said. This would mean that at the end of our tenure process we would be given a nationally or state or perhaps a locally designed test on "being a professor"an obviously absurd proposition. (Yet who wants to bet that this idea isnt out there somewhere?)
It's admittedly a rough analogy, but the arguments against a professor test are precisely the arguments against a student writing test: it's reductive, it lacks the context that would give it meaning, it gives you data and numbers instead of description and information, and so on.
I bring up this admittedly silly metaphor because I think it's important to remember that outside of credentialing systems (the Certified Public Accountant exam, and the like) the corporate worldrarely an advocate of progressive assessmenthas never adapted standardized tests wholesale, and that should tell us something in itself. You don't get a raise because you passed a "management test" or whatever, simply because long experience says these things are thin measures of performance, much less knowledge. It's also important to remember that this truism is a familiar facet of our students parents work experiences, and we could use this as a starting point in attempting to rally them to a more substantive assessment system than testing and grades.
Recent developments also suggest that the private educational foundation system (e.g. the Carnegie Foundation, among many others), which plays a decisive role in directing corporate funding in education, is steadily moving away from standardized testing and promoting portfolios at every level.
The move away from the standardized test (as tentative as it is right now) is part of a larger cultural shift, not reducible to left/right, populist/authoritarian dichotomies. This shift is better explained in terms of the organization of the economy as such the standardized test is perhaps best seen as a by-product of the assembly line; as our culture shifts away from primary manufacture, it's not surprising that we also move away from the testing systems developed at the same historical moment. The ongoing growth of the secondary and tertiary sectors of the economy, the so-called technology and service revolutions, respectively, seems to be a driving force towards portfolio assessment, at least for the professional cadres.
Politically speaking, we have to deal with these foundations and with changing economic realities, and that seems to me to mean that eventually we have to deal with the attendant assessment models. I think for a lot of reasons we had better start now to figure out how to do that on our own terms, lest we be forced to do so on terms not of our own choosing. Again, if we do our own writing and advocacy well, I believe we can count on parental support in this endeavor.
I won't do more than mention the eugenicist roots of the very concept of a standardized test, or its military history in the First World War's need for pilots, for example. And of course there is the immigrant story of the northern European's fear of all those inferior southern Europeans, and so on. (Can you tell I'm French and Southern in ancestry?) More modern forms of racism have played an important role in this, too, to say the least. The history is undeniable, but I think those so-called politically correct arguments tend to promote the idea of a "democratic" standardized testing system, which I see as an oxymoron. But it is important to remember that we are dealing with the sort of technology that does not have humane origins. Ironically, it does have "populist" origins, but that's another story too!
Generally, I think all technology is deeply bound to the social conditions in which it was developed. The standardized test is in this sense an artifact (or anachronism) related to Victorian models of economic scarcity. Similarly, the standardized test was once an effective tool against inherited wealth, thus the populist support. Horses and buggies worked well in their time too, but we can afford better now. In any case, the history that once made the standardized test useful is simply over now, in several senses. The choice we face is not between standardized tests, or testing and grading as such, and no testing; it is between two competing forms of assessment, between pedagogically useful richness and bureaucratically convenient simplicity. And I think our ambivalence is, as Pogo might say, about us more than it is about students.
Most important to me is the very purpose of assessment in the context of writing pedagogy. More than once, colleagues have suggested to me that the purpose of the EWP ought to be to generate feedback to faculty charged with teaching writing; those who were approving essays thought to be of too low quality might receive polite notes, advising them of the fact. It's my sense that the portfolio system is in a kind of limbo right now exactly because people are justifiably afraid of getting those notes, and of the long-term implications of such notes. Yet the very idea of these notes, I believe, contradicts the aims and goals of the EWP.
In my view, the old WCE system was redundant insofar as it attempted to evaluate what had already been evaluated both by individual teachers and by departments. Our own system of professional evaluation, negotiated and institutionalized through our collective bargaining agreements, in other words, already assesses teachers effectively. In any case, there is no reason to muddy the assessment waters by trying to assess teachers and students using the same instrument.
I would also contend that standardized testing implicitly embodied an institutionalized mistrust of teachers and intellectuals with a long history in our culture. It is also based on a kind of scientific idealism, the notion that any and every phenomenon can best be evaluated via the model of scientific inquiry. Once again, that is too long a story to tell in detail here, but suffice it to say that I believe scientific inquiry has been enormously productive precisely because it is so restricted in focus, and because it does not seek to answer every human question or problem. Predicting the motions of planets is one thing, probing the human mind quite another.
Whats more, I do believe that trust in teachers can and should be institutionalized. Again, assessments of teachers and of student learning are related but not identical processes. The WCE did a poor job of helping us understand and improve either; given support, the EWP at least has a chance of improving student learning.
So if we don't (institutionally) mistrust teachers and if we reject positivism (really I would say we are rejecting a scientific stereotype) then what do we do when we want to do assessment? In my view we use a portfolio system to produce qualitative descriptions of the program. Again, I want to emphasize that the EWP should be used to assess (not reductively evaluate) programs and not teachers. What should happen is that the writing program at EIU as a whole would be given a narrative, not a note, so to speak, that describes to us in some detail what our students can and can not do, and that gives us some ideas as to what we might do differently in order to achieve our (collectively defined) goals.
One side story is that the only department that can be singled out in the current system is the English program, and many of us are afraid that the English department itself, rather than individual professors, will be given the notes.
That too has pretty awful long-term implications. Yet if we had enough support from faculty, that could be easily fixed why put the course numbers in the database at all? Much of the current system has similarly small, easily fixed problems that lack only the faculty's (organized, vocal) support to be dealt with adequately. Others would have to be worked out in the long run, but all are dependent on faculty support and organization and commitment. The WCE had the advantage of being both familiar and relatively cheap; nonetheless, we can afford to do better, even if we have to pay more and accept the risks of innovation.
We can't create a technology system that never crashes, for example, but we could create one with enough support personnel that when it did crash it was back up in minutes, not hours or days. Given that level of support, all the problems I have thus far heard could be dealt with effectively. Other colleagues have worried that students will switch papers and disks between their office and the office of assessment, or even turn in work that is not their own. With an adequate and secure infrastructure, however, the portfolio could be turned in and verified via e-mail, as it was originally designed to be. I do not mean to imply that the system could ever be perfect, of course, but with adequate administrative foresight and finance it could be made very effective. By and large, the system is already designed in such a way that students have little interest in cheating: why would students wish to have our program misrepresented?
I know for many in our community this programmatic assessment would never be enough, but it is enough for me simply because, again, I trust teachers and I think the current individual teacher portfolio assessment system is strong. Again, thanks to a long history of organized effort, our teacher assessment system is to a great extent under the democratic control of the faculty, something that could never be said for the WCE. I think getting the EWP fully under the sway of shared governance is an achievable goal. I also don't think you can reduce this hugely complicated problem of "students can't write" down to classroom methods, or to bad teachers, or to any one thing.
I don't even think that assessment should be first on our list of the many things that we need to work on in order to improve student writing. And I don't think we much help our students by creating a system that attempts to identify and single out "bad" or "ineffective" teaching or teachers or root out grade inflation. All of these phenomena are quite predictable artifacts of the current system, secondary not primary causes of the current state of student writing.
If, however, you say to me "let's work on the problem of students who can't write," then I would say let's start with public school funding. It's not that complicated in theory, and yet it's enormously complicated in practice, but we can at least acknowledge the fact that most public school writing teachers have, say, 4 or 5 classes of 20 to 40 students. In the conditions common in public schools, teaching writing effectively is simply not possible, and no assessment system in college can touch that problem. Teaching of all kinds is an expensive proposition, and arguably as a culture we have never been fully committed to it.
One reason I dislike testing and grading so much is that it is emblematic of our hollow commitment to the ideal of an educated society. I still believe in that ideal, and at the center of that ideal to my mind is the realistic understanding that education is a long, hard, life-time proposition, and that an educated society would have very differant priorities than our own. To use the cliché, people would come before money.
If all that were not enough, I also would argue that standardized testing undermines our effort to create a self-governing institution. I believe, in other words, that we have every reason to get on top of this portfolio system and to use it to strengthen democratic control of the schoolsthat means, in short, bottom up governance. If we don't, well, it will be used in the same way that standardized testing is still usedfor top-down governance. And without democratic schools we will simply never be able to meet our students' needs, or even to come close to achieving whatever we mean by an educated society.
Again, this is based on a very fundamental trust of teachersand of peoplethat in the end many simply may not agree with. It's something we work towards, of course, this trust, not something that is ever achieved once and for all. But, again, being an old-fashioned kind of guy, I am trying to work towards that goal, rather than away from it, even as I recognize its idealism.
All that said, I don't have much of a problem with using quizzes in the classroom, simply because, given our students' backgrounds in the public school system, these sorts of tests can be motivational. But I don't think that they are precise measurement tools. As CNN says all the time, "this is not a scientific poll." I feel the same way about gradingI use it while I recognize that saying that a student is a C writer tells me very little that would help me help that student. And I would never collect quizzes or grades and then add up the numbers and use the data to produce a description of good or bad teaching, or say that they tell me something crucial about what students "know." Don't even get me started on distributed or embodied cognition theory, or on the notion that "knowledge" is not simply something that is "inside our minds"!
These ideas have other important implications for our program, it seems to me. If, for example, English 1002 is a writing course and not a literature course, which I think it is, then ending the course with a final or using reading exams as a major assessment tool seems pedagogically confusing, at the very least.
I am right now preparing a survey that will attempt to give us some better sense of freshman technology skills, so I do think surveys can be a useful tool, but they have very sharp validity limits, and I would never call the results more than an approximation. I will certainly never design a survey that can be used to single out individual teachers for their pedagogical choices, tempting as that might be given my belief in the newer writing technologies!
I say all this because I want to emphasize that while I understand the history and the economics behind testing and grading, I also recognize the complexity of our situation, and that it is unlikely in the extreme that as a culture we will suddenly abandon our quest for objectivity in assessment. But I feel the same way about that situation as I do about many other bad but common ideas: just because I have to live with them doesn't mean I have to start thinking they are good!
I agree fully, in other words, that we need to provide honest, even blunt answers to the question of "how am I doing?" or "how is my kid doing?" As a culture, education professionals have all too rarely had the courage to do that. The thing is, the bluntest, most honest answer is: learning is not the kind of thing we can usefully talk about in that way! I understand this desire for clarity. If a doctor tells me that I have cancer I desperately want to know an exact answer to when I will die, but I also understand that responsible doctors have to admit there are limits to the certainty they can provide.
I would like to finish this already long text not with conclusions but with an invitation to further reading and discussion. In the end assessment is as much about historical and political sensibility as it is about facts and argument. My own sensibilities have a myriad of origins of course, but it seems worthwhile to at least touch on some of my more important influences, in hopes of contributing to a strengthening of our political commitment to the EWP. Some of these things you may well be familiar with, I certainly don't claim originality in my research!
To my mind, the best text on scientific objectivity as a cultural phenomenon, which is not the same thing as saying that objectivity is impossible, is a book by Ludwik Fleck, Genesis and Development of a Scientific Fact. It's a very exacting history of the Wasserman test for syphilis, of all things, and it very precisely lays out the limits and strengths of positivism without embracing nominalism and or a radical skepticism that would reject scientific reasoning wholesale. My favorite historical text on education, testing and the foundations is by Clive Barrow, Universities and the Capitalist State: Corporate Liberalism and the Reconstruction of American Higher Education, 1894-1928. A slightly different but compatible description of the economics of universitiesand English departments in particularis in English Departments and the Circulation of Cultural Value by Evan Watkins (no relation).
Several years ago (1995) Nicholas Lemann published a piece in the Atlantic Monthly calledThe Structure of Success in America that does a fine job of telling the story of the standardized test (SAT, the Army test, Princeton, etc.) that he then turned into a book. The standardized testing industry, huge and well organized and not interested in education particularly, is certainly a part of this story as well.
If you want to investigate the movement away from standardized testing more broadly, I would suggest Fair Test's website, http://www.fairtest.org/. Fair Test is an advocacy organization, but I think they present a very reasonable argument, and if you dig around in there you can find reading on testing and assessment research that is very persuasive. They also have a series of FAQs that sum up the arguments about and research on standardized testing. On the portfolio end, one of my dissertation chairs (there were two!), Margaret Syverson, is very involved in portfolio development and research. Her model, which she calls the Online Learning Record, is described at http://www.cwrl.utexas.edu/~syverson/olr/.
Peg has more information on this sort of thing than you can shake a stick at! She has also done some fascinating work on non-positivist objective evaluation, as odd as that might sound.
this discussion at least offers some sense of where I am coming from and of
my own fairly complex rejection of the WCE and embrace of the EWP.
If we are going to have any success in this "problem" of getting our
students to write well (my professional aim, in the end), then we are going
to need a lot of solidarity and mutual support. I am hoping that this text is
read as my way of trying to argue that, whatever our differences, we do share
this goal of improving student writing. If any of you are interested, then I
also hope you will continue to give me your advice and ideas on how that might
be accomplished. After all, I am still more or less a newcomer!
| E-mail the author
Agora Home Page | Contents Page | EIU Home Page |
English Dept. Home Page | Past Issues