Monday, November 24, 2008

Using Computers to Assess Writing

As Huot discusses in his “Computers and Assessment: Understanding Two Technologies,” he argues against notions of using technology to relieve the drudgery of grading student papers and as a more efficient way of grading papers. He follows Sirc’s assertion that we should use computers as a medium for responding to student writing (239) and argues that they can be especially useful for collecting and responding to portfolios.

I find using computers to respond to student writing useful for additional reasons. One is a personal reason: my handwriting is akin to hieroglyphics to anyone who has never studied them. Oftentimes I would wonder why students kept making the same mistakes over and over again. Unfortunately, I made the mistake many novice teachers do by assuming that they were just not interested in improving their writing. After feedback from classes and on student evaluations, I realized that deciphering my handwriting was a large part of the problem. Now, I have students submit their essays on Vista or in email so that I can type comments throughout the paper and give a summary statement at the end. This happens to be more time efficient for me because I type faster than I can write. However, the more important issue is that my students are able to understand my comments.

Another benefit of responding to student writing with computers is to demonstrate academic conventions. I write my comments in blue in their text with brackets around my words to show where I have inserted comments. I used to use the Comments tool until I received too much feedback that students were unable to read them since they had older versions of Microsoft Word. By using brackets to insert my voice into theirs, I reinforce the fact that different voices need to be signaled in the text in some way (a lesson on using sources/voices) and the notion that a writer cannot change the words of another author without telling the audience that they are doing so (using brackets to facilitate the audience’s understanding of the quote). A (“feel good”) side note: I like to use blue, as opposed to red, because it is less threatening and “cooler.” Color, of course, is only one part of the way that students’ respond to a teacher’s comments, in addition to the quantity, tone of voice, explanation (or lack of), praise, and illumination of options. Nonetheless, it is part of my overall strategy to encourage improvement.

A last benefit that I see for using the computer as a medium for response to student writing is that the malleability of the Word document (i.e., REVISION) is fore-grounded. Writing may be more clearly seen as a recursive process through the use of computer-mediated responses. By printing out a document it looks final and finished. On the other hand, submitting it to an instructor and having that instructor change the inside of the document highlights that writing is in flux. The resulting revision of the document also may also highlight this process as recursive. Although, this does depend on how the instructor responds to the student’s writing—if the response is not based in a rhetorical and recursive understanding of writing, the affordances of the document alone will not facilitate this type of understanding just as portfolios can be used without the theory behind it being employed. All in all, I find that computer-mediated responding is the best way for me to respond on both a personal and theoretical level.

Sunday, November 23, 2008

PaStIcHe: examining remixing

"How can we teach, engage in research, write about, and talk across boundaries with others, instead of for, about and around them?" (620).
Jacqueline Jones Royster "When the First Voice You Hear is Not Your Own."

Indeed.

Enter: “Datacloud: The Remix”

In our own scholarship:


[Remix: we might also think of coding data as a form of remixing? We read, stuff ‘emerges’ – emergence though is based on our interpretative lens, our framework-- and where does our interpretative lens come from? the voices in our heads—old mentors, fellow scholars to whom we speak, and of whom we read, those whose approval we seek. We code, stuff emerges. Whose stuff is it? Letting the data speak for itself…sure, but then we act as interpreters of that voice, we always necessarily select what we will attune to. And in that selection, our interpretative lens is evidenced.]

In our teaching:

"at least one set of social forces suggests to students that using citations and quotations from source materials will be valued less than their own original text, a situation that may encourage them to conceal their sources." (p. 378)
[Really? I spend a lot of time *encouraging* students to use sources! Our students often aren't granted the authority to speak for themselves-- they're expected to use sources because their voices aren't always trusted to speak *correctly* on their own! Elbow, ok, maybe he’s a little passé, but he has a place here in any discussion of voice, which is what’s going on in one level in this text. Johnson-Eilola & Selber are reframing students’ voices like Bakhtin’s heteroglossia.]

"students are encouraged to make explicit their borrowings and appropriations. What counts, in this new context, is the ability of students to remix texts in ways that address specific issues, readers, and situations." (p. 380)
[They're arguing for a shift from Invention to Arrangement and Delivery. They're not calling it that explicitly but that's what it is. It’s more than a shift though, it’s perhaps more accurately a reappropriation. Arrangement and Delivery are taking ownership of Invention? Or vice versa?]

" What if the “final” product a student produces—a text—is not concerned with original words or images on a page or screen but concerned primarily with assemblages of parts?" (p. 380)
[this sounds a lot like factory production language though, this is where I’m getting into trouble. Is this again a lack of trust? Are our own students’ personal narratives still valued? Are we telling them we don’t want to hear your stories? Or is my problem here in how I’m defining stories? We don’t want to hear your old stories, you’ve got to make new stories and you need to recognize that they come from outside of you?]

"patch-writing"
[like topoi a little. also like progymnastata, modeling- Graff & Birkenstein They Say I Say - They're arguing for writing as problem-solving.]

" A traditional approach in composition would create at least two hierarchical levels of value among these materials: The quotations, links, and default elements of the template would be valued less than the original text (original images, personal narrative, and summaries). After all, one might say, the other materials were all pre-existing: Johndan merely found them." (p. 390)

[Invention as discovery, so they're situating invention as problem-solving within invention as creative act. Is there room here for recognition of individual subjectivities? Yes, I think so, as long as the students are encouraged to explain their choices, their juxtapositions. But again, I'm running in circles, because that would also reinforce Johnson-Eilola & Selber's claims that we value the original text more than the texts students bring in. -- this would be a means of forcing them to generate original texts. Is there a way out of the loop?]

On plagiarism, originality, and assemblage

I borrow; forgive me:

From T.S. Eliot, The Sacred Wood: Essays on Poetry and Criticism


We turn first to the parallel quotations from Massinger and Shakespeare collocated by Mr. Cruickshank to make manifest Massinger's indebtedness. One of the surest of tests is the way in which a poet borrows. Immature poets imitate; mature poets steal; bad poets deface what they take, and good poets make it into something better, or at least something different. The good poet welds his theft into a whole of feeling which is unique, utterly different from that from which it was torn; the bad poet throws it into something which has no cohesion. A good poet will usually borrow from authors remote in time, or alien in language, or diverse in interest. Chapman borrowed from Seneca; Shakespeare and Webster from Montaigne. The two great followers of Shakespeare, Webster and Tourneur, in their mature work do not borrow from him; he is too close to them to be of use to them in this way. Massinger, as Mr. Cruickshank shows, borrows from Shakespeare a good deal. Let us profit by some of the quotations with which he has provided us—

Massinger:

Can I call back yesterday, with all their aids
That bow unto my sceptre? or restore
My mind to that tranquillity and peace
It then enjoyed?

Shakespeare:

Not poppy, nor mandragora,
Nor all the drowsy syrops of the world
Shall ever medecine thee to that sweet sleep
Which thou owedst yesterday.


Massinger's is a general rhetorical question, the language just and pure, but colourless. Shakespeare's has particular significance; and the adjective "drowsy" and the verb "medecine" infuse a precise vigour. This is, on Massinger's part, an echo, rather than an imitation or a plagiarism—the basest, because least conscious form of borrowing. "Drowsy syrop" is a condensation of meaning frequent in Shakespeare, but rare in Massinger.


Massinger: Thou didst not borrow of Vice her indirect,
Crooked, and abject means.

Shakespeare: God knows, my son,
By what by-paths and indirect crook'd ways
I met this crown.



T. S. Eliot seems to be inferring that imitation is a lesser art; yet still an art, indeed. Imitation is the classical method of learning. And if we ascribe to dialogism per Bakhtin, it seems perfectly legitimate to cut and paste. Haas was making the point with writing interns that it is perfectly ok if they look at one another's work--re: their letters to perspective employers, their resumes and borrow---put into use--one another's ideas. She was saying that in the university, we have these rules, but that's not how they work in the real world where people collaborate, share, help one another.

good god grading

The predictions of the future of computer-based assessment made me imagine a future where a John Henry-esque battle between a TA and a grading machine takes place. That's something I'd gladly lose.

I think Brian referred to the act of student grading as "onerous" in one of his articles, and that made me chuckle a little because the act of grading 50 student papers got in the way of me doing any "real" work this week (and will quite possibly result in my embarrassment whenever we show off our projects next week). At about 15-20 minutes per paper, I have to allot myself about 12 hours for any major assignments that need extensive grading. Since I'm teaching two classes this semester, I've cut back on the amount of assignments that require an in-depth, personal response from me, and transformed these things into group/blog work. So, in some ways, I welcome the electronic grading robots--unless of course we can live in an idealized society where I can focus all of my efforts on teaching one class. But I don't think that's gonna happen.

That being said, all of the work in the classes I teach is submitted and graded electronically. While I'm the one doing the grading, using a computer for this act shaves a lot of time off of the productivity I would have if I brought a giant stack of papers into Starbucks every couple of weeks. Here are the benefits I've seen so far:
  • Organization. I'm huge on this. It helps with grading because I can keep track of all versions of a student's paper.

  • Surface-level stuff. Because the new word can spell and grammar check most stuff in context, this is something that I don't necessarily have to worry about when I'm grading. Though I still need to tell students FOR THE LOVE OF GOD PLEASE RUN THESE CHECKS about a thousand times before they realize how much of a difference hitting F7 can make.

  • Instant word count. It's always at the bottom of the screen, and it's infinitely useful.

  • Grading via computer allows me to write more, and it allows the student to read everything I've written. Since my first semester of teaching, I've learned that students don't necessarily read all of the comments you put on their paper, so I don't write as much as I used to--but I still have the ability to do so. Of course, it seems like most students don't want to see more than their grade, which is why it's fun to piss them off with an evaluative non-grade. Takes a while to break 'em.

Strangely enough, grading does seem like the thankless "busywork" of academia--along with teaching comp classes. In my 90 years at college, I've seen enough TAs and GAs grading papers for real, live professors that I have to fall into John's camp and wonder just how useful and reflective grades can be when the process itself can be shuffled off like this.

I came to the conclusion Friday night--after I finished all of my grading--that I need to be in a field where my students hand in ScanTron sheets and I have way more free time. This is why I'm joining Kent's applied physics program. I'll see you guys there in the Spring.

Rambling about computers and assessment

Brian Huot’s piece, “Computers and Assessment: Understanding Two Technologies” was very interesting to me. As a teacher of composition, I always find the area of technology and assessment to be a tricky arena. As for the piece itself, Huot examines past practices used with computers and assessment. It was scary to read that some teachers actually thought that a computer could replace the human agency in the grading process. People wanted to use the computers to grade papers based on “surface level” details. Word count, syllables, and grammatical uses were caught by the computer and allowed teachers to place to students in the “appropriate” category based on the findings of the computer and the student’s use of those things. I found that appalling. I realize that it is very tempting to try to find a technology or program that would decrease the amount of time it takes to evaluate student writing, but it seems to me that these predecessors of this decade led us straight into standardization. I hate standardization. Using a computer program to “grade” writing is ridiculous to me even with the temptation of less itme spent on reading countless student essays.

Huot reviews some of the main reasons for the need/want of computer grading systems and along with a dispensation of “drudgery” was the concept that computer grading would be more efficient (that seems a lot like a means of relieving drudgery but whatever)…so these teachers wanted to decrease the amount of time it takes to grade to , I am guessing, spend more time on creating a better classroom or some such. I totally understand these reasons and the early programs do seem remarkably like SpellCheck and GrammarCheck, so I guess we have moved into an area that allows some computer objectivity in the writing classroom. I am an advocate for these programs because it allows students to write without worrying as much about their surface level flaws becaes the computer will usually find the mistakes for them.

Huot then goes on to explain how assessment and computer technology works in our world today and the efficiency of using the World Wide Web to share portfolios and ideas within the computer classroom as well as allowing teachers to compress the very large portfolio system. As I am typing, I have a box full of portfolios waiting for a student to retrieve them taking up a large amount of space in my kitchen, so I can totally justify the idea of having portfolios electronically submitted. Also, the computer allows teachers to respond to students in a more time efficient way because they can type their remarks and then immediately send it to the student via email or a system like Vista. Although I have not yet integrated this system in my classroom I plan to because it seems like a very good idea. Overall, I just liked this piece because I think it looks at some of the ideas that led to how the computer is used in assessment today and it discusses some of the very positive traits the computer can give to the writing classroom.

However, as I was reading about the beginning uses of computers as grading systems I was hoping to discover some sort of type that allowed for less time spent on grading the papers due to the use of the computer—but I realize that I don’t really believe in objective grading like that. I would not be here if graduate school application relied only on standardized test scores and writing because I am not a strong surface level writer. The computer programs that are discussed would probably have thrown me out in the first round, but since the application process includes reading a person’s essay by human subjects I believe I was able to join the graduate program. I think I am confusing myself a bit here, but basically I just want to say that I realize it is tempting for a composition teacher to want to find more time in the day (especially if they are teaching 4 or 5 sections of Composition) but the idea of human subjectivity in writing is too important. Value should be placed on what ideas the students are trying to convey and not how well they use their commas, and the revision process is an extremely important tool to teach and learn. Computers and the technology that comes with them are valuable because they can take some grunt work off of the teacher and allow for swifer responses and more creative lessons—so, go computer technology!!!

assessment, testing, grading--wtf?

When I was studying to be a high school English teacher--back when I believed that this was my true calling--I was taught to make a distinction between assessment and grading. Assessment, I was told, was an ongoing process of monitoring student progress. The goals of assessment were to verify that learning had occured, and to ensure the appropriateness of the curriculum. So-called formative assessment included frequent systematic observation (including anecdotal records) and necessitated the ongoing provision of feedback for students. Assessment was to be intimately linked with instructional planning. It was not to be thought of as a thing added on at the conclusion of instruction. It was a way of evaluating whether or not your teaching had been successful, not a way of determining if your students had been successful. It was a way of identifying when concepts needed to be taught for the first time or retaught if learning had not occured.

Grading was different. Grading certified completion. Grading suggested finality. Grading offered conclusive judgment: A B C D F.

Because of my training, I have tended to continue to make the distinction between assessment and grading to this day. I have also noted a kind of tension between the two. Assessment, by this definition, seems to be never ending. It's something you do over and over again--to inform teaching and reteaching, learning and relearning--and to make both teaching and learning ongoing, recursive practices. Yet, in the end, we are still asked to grade. The ongoing process of assessment is forced to a halt by the singular moment of the grade. This leads to a kind of conundrum. E.G: Help your students to work on, I don't know, MLA format. Sit with them, work with them, write to them, comment on their papers: Assess. Then, at some point, put a letter on their work--how about a C (they just don't get it!). Grade. Now try to assess--since that's ongoing--try to go back and help after you've put that C on their work--except now they don't want to talk to you, or worse, you can't go back because they are gone. The semester has come to an end. The grading session is over and students have been "marked."

Aside from this tension between ongoing assessment, on the one hand, and conclusive grading, on the other, I also find that while grading is something that I observe frequenlty, assessment is exquisitely rare. In my admittedly few years of teaching, I've very rarely seen a teacher say, "hey, my students aren't getting this concept. I guess it's back to the old drawing board for me. How can I reteach this content?" By contrast, I've seen an alternative scenario rather frequently: "Hey, my students didn't get this concept. Guess they'll have to accept Cs so we can move on to the next concept (or reading assignment, or paper, or whatever)."

Of course, the dominance of the grading-mentality as opposed to an ongoing assessment-mentality can be see in the standardized testing movement. Kids are tested like mother-fuckers. They're stamped with grades so that they may be "placed appropriately." Of course, in practice, the kids who aren't succeeding simply get flagged and plopped into remedial courses or lower tracks. These kids are, needless to say, disproportionately poor people of color. The remediation they receive--even in cases where it is helpful--also has the effect of labeling these kids as "Other" for the rest of their educational lives. Once they are classed and tracked, they remain in their class and in their track and never rejoin the masses of "normal" children. The grading system, then, helps to reproduce existing social inequities. After all, we NEED kids to fail. We NEED, some of them anyway, to fill the low-end jobs. We need optimum rates of unemployment, and thus optimum rates of failure and drop-outs. Might as well be poor black kids to fill that social role.

So, what's the solution? Well, as Madaus has pointed out, the solution has been to test our way out of it. Keep testing. Keep providing grades. Keep tracking. Don't bother to look at any of the social dynamics that may adversely affect educational outcomes. Don't bother to look at the adverse effects of testing/grading. Don't bother to reduce class sizes, increase teacher-support and training, make room in the curriculum for reteaching. And, above all, don't question why grading--stamping letters on people--is necessary. Don't question how grading is inextricably linked with an economic system that requires failure, poverty, and unemployment.

This begs the question: WHY the eff do we need to grade? Or, at least, why the eff do we need to grade student writing? I'm not sure I've ever been satisfied by an answer to this question. I can see grading people if they're brain surgeons, say. I can see grading people if what we're grading has life-and-death implications. But writing? If a kid can't write a college essay, no one is going to die. If a kid can't write in this context, it doesn't mean s/he won't be able to write in some other context. So, the grade doesn't even tell if the kid is a good writer. And it sure as hell doesn't assess. If I give a kid a B in my College Writing I class, his College Writing II teacher doesn't even see the goddamn B. Nor does that teacher have any goddamn clue from that B, what this kid does well or poorly, what s/he needs help with, whether the B was 'cause I sort of liked the kid's growth over the semester; or because s/he really did "A-work" but missed a few assignments; or because I found him/her slightly more pleasant than the others, even though the writing kind of sucked; or because, when compared to classmates, the kid's papers were just a little worse than some better papers; or because B is the highest grade I give out; or because I have some arbitrary, yet dynamic, idea of what the fuck a "B-paper" is and this kid's paper reminded me of it. The grade is meaningless as an assessment tool.

So, is there any reason to put it on some kid's paper? Is there any reason that a kid NEEDS a writing grade, other than to tell him if s/he's good or bad, right or wrong. And, if this is why, do you really need a goddamn grade to tell you if you're writing is good or bad, right or wrong? Isn't it already obvious where your writing stands when someone says, "I don't know WTF you're saying in this paper"? Or do we give grades simply so the kid can show them to his or her employer? If so, is there any reason that an employer needs to see a letter grade that "reflects" a kid's ability to write rather than some kind of writing sample from the kid, or, better yet, some non-letter-grade assessment about the kid's qualifications for that particular job?

Why do we grade writing? Is it only to be gatekeepers? Is it only to ensure that the right kids "make it" and the wrong kids don't? I still see no reason for it--other than to reproduce and sustain an unfair social system.