Digital Writing, Digital Teaching

Earlier this month, one of my students, Lynette Seitz, and I were interviewed by Heather Smith, CMU’s Assistant Director of Media Relations about technology literacy and our work in ENG 315 this semester.

I appreciate her invitation to record this podcast and it was wonderful to have Lynette’s voice in there, too, as a pre-service teacher who is thinking about incorporating digital writing into her classroom.

You can get the podcast through CMU’s channel in iTunes.

Another great session this week, this time with one of the co-authors of the Writing Next report: Steve Graham.

Here is an overview from the MSU LARC site:

Steve Graham, Vanderbilt University

Evidence-Based Practice in Writing – Drawing on Experimental, Qualitative, and Single Subject Design Research for Answers

Wednesday, April 16, 2008
11:30am – 1:00pm
Room 133F Erickson Hall, Michigan State University

This presentation will examine what we know about effective writing instruction, drawing on three recent reviews of the literature. One of the reviews (Writing Next) was a meta-analysis of experimental and quasi-experimental writing intervention research. Another review was a meta-analysis of single-subject design writing intervention research. The third review was a meta-synthesis of qualitative research conducted with outstanding literacy teachers, designed to identify common practices across studies. Advantages and disadvantages to the use of evidence-based practices in writing will also be explored.

About the Speaker:

Steve Graham is the Currey Ingram Professor of Special Education and Literacy, a chair he shares with Karen R. Harris. His research interests include learning disabilities, writing instruction and writing development, and the development of self-regulation. Graham is the editor of Exceptional Children and the former editor of Contemporary Educational Psychology. He is the co-author of the Handbook of Writing Research, Handbook of Learning Disabilities, Writing Better, and Making the Writing Process Work. In 2001, Graham was elected a fellow of the International Academy for Research in Learning Disabilities. He is the recipient of career research awards from the Council for Exceptional Children and Special Education Research Interest Group in the American Educational Research Association.

And, here are some notes from the session:

• Opening quote: “Kids know the most interesting things” - Mark Twain
  • “It hurt, the way your tongue hurts when you accidentally staple it to the wall.”
• Writing is nowhere in terms of the educational reform movement in this country
  • The things that drive the educational reform movement are reading and math
  • Now, STEM - science, technology, engineering/economics,math
  • Why is writing out in the cold?
    • This is not always bad, as it sometimes results in school practices that are not good
    • But, we need to make the case that writing is important
      • 1. One of the reasons that people are not paying attention to writing is that there is a general perception that we do not know how to teach writing. Policy makers want evidence, and they want particular kinds of evidence.
        • We do know that there are some things that work for all students 4-12 and younger
        • People don’t think that writing is important. So, we have to look at the effects of writing on content area learning. We make the case that writing can be helpful in terms of the STEM skills
        • Reading gets more play in the literacy discussion. We need to look at the effects of writing on reading. How does writing affect reading?
          2. What are the practices going on in elementary and secondary schools
          • Limitations: survey data that could be rosy, but the data is still not good
          • ELA teachers are doing less than one extended writing assignment a month
          • You don’t wan to go into policy making without good research to make recommendations

          3. Theoretical framework — from Patricia Alexander from moving from knowledge about discourse and enhancing motivation

• What are three primary resources we can draw from?
  • Professional writers
    • Unfortunately, the advice can be simplistic and only moves confident writers to expert writers; it doesn’t help other writers
  • Effective practices from experienced teachers
    • Talk to effective teachers or observe good teachers in practice and study them
      • Problem: if I go in looking for one thing, I will likely see it (difficult to separate the wheat from the shaft”
      • Problem: Donald Graves and the example that works. Yet, there are times when this doesn’t work.
      • Problem: generalizability. Evidence is often selective.
      • With scientific studies, we collect evidence, presents findings for all participants, replicability, strength of impact — all this leads to something that should be more trustworthy than insight and experience.
• This presentation, thus, will draw on three sources: experimental, single subject, and teacher practice
  • Other criteria:
    • Four replications
    • Converging evidence (the sun, the moon and the stars align)
    • Recommendations based on higher quality studies are superior
      • Process writing has very poor research, so you need to be cautious about this
      • The more studies, the merrier
  • Effect size:
    • .8 is large
    • .5 is moderate
    • .25 is small, but significant
  • Writing Next looks at overall quality of writing
    • Strategy instruction (planning, revising, editing, and regulating the writing process; 20 studies, .82 effect size (particularly helpful for kids who find writing difficult)
      • Don’t just PEE (post, explain, and expect) students need repeated modeling
      • For instance, the STOP strategy (Suspend judgment, Take a side, Organize ideas, Plan more as you go)
    • Teaching Summarization (systematic and explicit teaching of how to summarize texts); 4 studies, ? (missed it) effect size
      • Teach the six rules of summarization
    • Peer assistance (working together to plan draft and revise); 7 studies, .75 effect size
      • Needs to be a structured in a positive way — having students add questions marks and carats in their peers’ papers
    • Setting product goals (specific goals for the written product to be completed); 5 studies, .70 effect size
      • Need to tell students what you expect without limiting them
      • Product goals and revising
    • Word Processing (using word processing); 18 studies, .55 effect size
      • Some are short studies, but some are up to a year
      • Using the technology which is widely available is important, but it is used infrequently in schools or, when it is used, it is only used for final draft/publication
    • Sentence combining (constructing more complex sentences by combining shorter kernel sentences); 5 studies, .5 effect
      • Work on this together with students, then invite them to apply it back in their own writing
    • Process Approach (extended opportunities for writing, student ownership); 21 studies, . 32 effect size
      • Inviting students to engage in planning and revising is good
      • Bad news: the effect size is scattered all over the place
      • Receiving training from NWP is about a .46 effect, and is insignificant if you did not get that training
      • You can do this in a very poor way, and not get a good effect; this is compatible with a strategy approach that makes the writing more visible
    • Pre-Writing (have students engage in activities such as brainstorming; 5 studies, .32 effect
      • STOP strategy, for instance
    • Inquiry (old research); 5 studies, .32 effect
      • No pre-test done, so these studies may underestimate the effect size
        • Example: set a goal, analyze the data, look at specific strategies, and apply what you learned
          • A student in elementary school looking at conflict on the playground
    • Study of Models
      • Examines examples of specific writers and types of text; 6 studies, .25 effect
        • Model from good readings
    • Writing as a Tool for Learning (writing in the content areas); small but positive effect
      • 26 studies, but I think that it is more effective in science and math than ELA and social studies based on the effect sizes that we see
    • Grammar (explicit teaching of grammar); 11 studies, -.32 effect size
      • Quality of writing is not affected by grammar instruction
      • What this traditionally looks like is that you give a definition, example, and then is used in decontextualized works
      • If we expect it, but do not help students use grammar then it will likely not work
        • Take the kernel sentence: Dog bit mailman

      Recommendations for Struggling Writers (teaching handwriting, spelling, and typing to struggling writers — teaching transcription skills towards automaticity), small positive effect

• Single Subject Design Recommendations
  • Explicitly teach students strategies to construct paragraph; strong positive impact
    • Showing parts of a paragraph to the point that students understand the goals of writing a paragraph
  • Explicitly teaching students how to capitalize, punctuate, etc. helped
  • Reinforce positive aspects of students writing — social praise, tangible reinforcement or both as a means to increasing specific writing behaviors (small positive effect)
    • Traditional means of grading papers doesn’t work — “we get more with honey than we do with vinegar”
    • Couldn’t draw the summary effect from this, however
    • Need to move the feedback beyond the specific paper and help the student move forward in his/her writing
  • Self-monitoring (students asked to count how many errors they made); might be effective for some struggling writers
• Individual Teachers
  • Study exceptional teachers and schools
    • Practice had to be applied by the majority of schools or teachers
    • 10 Practices that might make a differences (had to occur in four or more studies)
    • Dedicate time to writing and writing instruction, with writing occuring across the curriculum
      • Get kids in the game of writing
      • Increasing writing by itself is not enough, it also needs to be motivating and give kids tools to be effective
    • Involve students in various forms of writing over time
    • Treat writing as a process
    • Keep students engaged by involving them in thoughtful acticvities such as planning compositions
    • Vary individual, small, and large group instruction
    • Mode, explain, and provide guided assistance when teaching
      • Teachers need to relinquish control
    • Provide just enough supprt so that students can make progress or carry out writing tasks and processes, but encourage students to act in a self-regulated manner as much as possible
    • Be enthusiastic about writing and create a positive environment where students are constantly encouraged to try hard, believe that the skills and strategies that they are learning will help them write well
    • Set high expectations
    • Adapt writing assignments to meet the needs of students
• Caveats
  • We should not order these practices hierarchically in terms of one being more effective thananother
    • Instead we should order them in a way that we see them working well for us
  • The database is thin
  • Just because a practice has been studied, it does not mean that it will be effective for all teachers in all classrooms.
    • Pay attention and see if it works in your classroom, with your students
  • Little data on those students who are most at-risk: ELL, learning disabilities, struggling writers
  • Lack of data on maintenance and generalization
  • Don’t really know best how best to put all of these things together
    • Think about trying to integrate some of these ideas as part of an overall approach rather than try to fit it into an existing approach
  • Teachers’ views on acceptability of these practices will clearly influence their use — this will include the issue of domain specificity
    • If you don’t accept it as a reasonable practice for you in your classroom it will not work
  • Just because a practices is effective in a study or was used by an exceptional teacher does not mean that it will always work
• Questions
  • 6 traits
    • Most studies were pre- and post-tests with no control
    • Look at journal article on Writing Next
    • 6 plus 1 looked pretty good for what was there
  • In-Service
    • When we asked ELA, science, and social studies teachers about how well their program taught them to teach writing, 70% said it was inadequate
    • We also asked about in-service preparation — you personally, school, conferences — ELA said that 70% were adequate, but 30% were inadequate
    • Most science and other content teachers didn’t feel prepared to do so
    • Not doing it at pre-service level because most states do not require a course in teaching writing
  • We have been doing this work for nearly 25 years and we have not delivered our work in terms of learning strategies approach and outreach
    • We have a distribution problem — we are not providing what we know in pre-service and in-service ed
  • A lot of this is very complicated, so we did the best practice book to give something for teachers to look at
    • We need to have support materials showing teachers how to do this — if you can see it, you can do it

My Reflections

In thinking about Dr. Graham’s talk, there are a number of salient points that I want to consider. First, he went over the 11 strategies from Writing Next and, even though there is evidence to show that all these strategies are effective, it is the individual teacher that makes the difference in writing instruction.

Second, he talked about how students can use word processing to write and revise, and that is very effective for their growth as writers; however, most of the opportunities that students have to write with the computer only involve typing in a “final draft” of something else that has been written out beforehand.

Next, he talked about peer editing and how students must be scaffolded into the process of giving feedback; just having them give comments to one another is not enough as they must use the language of writing in that talk.

Finally, he talked about the writing process approach and having an authentic purpose and audience for students should happen more often than what it is. Typically, the audience is only within the classroom walls, and students don’t share beyond their friends. Yet, he described a project in his children’s school in which students shared their work more widely and that it could be a goal for many, although not all of our assignments.

Here are notes from my Crossroads Writing Project colleagues, Lavon Jonson and Sonja Mack: “Blogging — Maximizing Writer’s Notebooks with a 21st Century Dimension.”

• Background
  • Bringing blogging into the traditional process of using a writers notebook
  • Writing with your students encourages them to write (Graves, etc.)
• Blog Growth
  • In April 2007, 70 million blogs, 90% by teenagers
  • In four years the growth has been phenomenal
• Rationale for use in the classroom
  • Blogging in English Class (on Teacher Tube)
  • At first, it was different, but then students described how it was interesting
• Why use blogging in the classroom?
  • To share items from writer’s notebook (used to share it in a circle on the floor, now we do it on blogs)
  • Edublogs forums (support video)
• Blogs to check out
  • Sonja’s Class
  • Lavon’s Reflections
  • Elementary Classroom Blog
  • Blog about Movies
  • HS English Blog
  • Blog about blogs and education

  What we’ve noticed from our students

• All students are able to contribute
• Comments are more heartfelt

Last Friday, I was invited to lead a “brown bag” session for my English department’s composition program. Titled “Multiliteracies in Composition,” we focused our pre-reading on an article about a second-year college composition course developed at Michigan Tech called “Revisions.” Details can be found in the following article:

Lynch, Dennis A., and Anne Frances Wysocki. “From First-Year Composition to Second-Year Multiliteracies: Integrating Instruction in Oral, Written, and Visual Communication at a Technological University.” WPA: Writing Program Administration 26.3 (2003): 149-171.

We began by watching the Richard Miller’s presentation: The Future is Now. This presented us with a variety of challenging questions about how we might pursue such a vision of the “new humanities” at CMU, including discussions about professional development, our beliefs about the changing nature of literacy, and how, if at all, a shift in our curriculum would happen in the time frame that Lynch and Wysocki describe from their context.

We then continued in small groups with a jig saw reading, where groups posted 2-3 responses or question in their own page on my wiki. After a watching Wikis in Plain English, they understood the basics of posting and were able to see how using a wiki could allow for multiple groups to post their work and then quickly share it with the class. The conversation continued in a large group discussion, including some emerging questions:

• What do students need in terms of literacy in a changing world?
• How do multiliteracies relate to technology and communications?
• What does the multi-disciplinary approach do for departments? What about specialization?
• If everyone talks the same language, do we have our own specialties?
• What does this mean for us in terms of the course? Content? Writing?
• Faculty-only vs. Graduate Assistants–How is this possible or feasible at our University?
• What does this look like across the curriculum? Is it sustainable?
• What about assessment? Individual? Groups? Programmatic?
• Is there still a need for traditional comp courses? Don’t you still need a first year comp?
• How does the continuing focus in professional organizations on 21st century lliteracies contribute to this discussion (last week’s NCTE statement on the future of composition), both for college and life?
• What would the writing center need to/be expected to do?
• Does this perpetuate a two-tiered society, a Gutenberg in reverse?
• How do we support faculty in these collaborations?
• Is the resistance about learning to do old things with new technologies or really coming to understand a new paradigm that the new technologies allow?

We ended with Michael Wesch and his students’: A Vision of Students Today, and just in time for a sunny mid-winter drive home. All told, it was a timely and lively discussion for our department, and I appreciated having the opportunity to facilitate the session. Given the release of the 2008 Horizon Report, it seems as though we are constantly reminded that things continue to change. I hope that this session serves as a spark that continues into further conversations about multiliteracies in composition later this semester.



This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

From the Google Reader….

It makes no sense complaining about the decline of the printed word. As it becomes just another medium, we are moving to a kind of multimedia literacy, where capability with print becomes no more important, or useful, than capability with image.

This is not necessarily a bad thing. There is no rule that says that the written word is superior to other forms of media. While some of us are print-oriented and will always remain so, there are people growing up to whom print is of comparatively minor importance.

The vast majority of these people will enter adult life as well educated as the generations before them. But they will rely less on books and newspapers, and more on television and the internet and multimedia.

We are not witnessing the decline of literacy, simply a new type of literacy. It is pointless to make moral judgements about the superiority of one medium over another.

Literacy alive and well in computer age - Perspectives - Opinion - Technology

Graeme Philipson makes a compelling argument for how our culture’s artists such as Doris Lessing and Elton John — both who decry the effects of the internet — need to change their perspectives about literacy in the 21st century. As a topic always on my mind, I found this opinion article a fresh take on the topic, especially the connection that Philipson makes between our thousands of years of oral history that has, only in the past few centuries, become replaced with print. Just because things are changing again doesn’t mean that we are in decline, it simply means that we need to adapt to the change.

This connects with a conversation that I was having yesterday with one of our college’s public relations consultants. She and I were talking about my research interests and how to make “literacy and technology” something newsworthy, and both struggling to find an angle on it. On the one hand, it seems that discussions of technology and literacy should be self evident. Yet, we continue to see school infrastructures and policies, teacher, administrator, and parent attitudes not reflecting a shift in thinking about this, and, as this EdWeek article points out, the fact that what doesn’t get measured, doesn’t get treasured.

So, my question today is thinking about how to make technology and literacy — not just tech literacy, but instead the changing nature of literacy — a key part of the conversation that the media reports on with schools. Clearly, when they publish the box scores for the test results, people stand up and pay attention. Without being punitive, are there ways that we, as educators, can engage the media to get the story of technology and literacy shown to the general public in a compelling manner?

To be more concrete, I want the tone of the conversation in the media to change from “Why aren’t students passing the tests” to “Why don’t students have one-to-one access to laptops for use in their daily reading, writing, calculating, observing, predicting, analyzing, etc.?”

Philipson shows us a way to shift the conversation on the opinion page. Can we think about ways to do it on the front page, too?

Blogged with Flock

January has been a busy month for me as I have been coordinating field placements for my ENG 315 students and we have begun exploring the use of blogs, wikis, podcasts, and RSS in our teaching lives. When we began this work a few short weeks ago, only a handful of these pre-service teachers had heard of a wiki or a podcast, fewer still knew about RSS, some had a general idea about blogs, and none of them were thinking about how these tools would translate into the teaching of writing in their classrooms. So, we started slow, and now things are moving along quite well.

The second week, we downloaded Portable Apps, and I explained my rationale for why would use these tools — both because they are free and open source as well as the idea that they need to be able to take their data with them. We also started setting up our blogs, and discussed the Common Craft video on blogs, thinking about implications for our classrooms and personal learning. The third week, we turned our attention to understanding RSS and reading each other’s blogs. This week, we set up our Google Readers, and I am now challenging them to begin using RSS reading in their professional responses.

So far, this process is going fairly smoothly and I do not feel that I am detracting from the “content” of the course by focusing on the technologies. In fact, I feel that they are helping me get some ideas across even better. For instance, it is one thing to encourage them to read each other’s blogs; it is quite another to provide them with a combined feed of everyone in their class and invite them to read, through their Google Reader, everyone’s posts. I will be building in some time for people to read and comment each week, as their reading of other blog posts will help them activate their brains for our class discussions.

Also, I am finding that they are all having “aha” moments as we move forward. Some are seeing connections to other classes an projects, and I think that they are all starting to see the ways in which we can connect with one another. For instance, one student explained how she immediately subscribed to all her friend’s blogs and, while it wasn’t purely academic, that solidified in her mind the power of RSS to gather information. In a time where we take for granted that all of our students understand so much about the web intuitively, it is good to know that we can talk about these technologies in relation to the teaching of writing and that they can begin to see some new connections.

Next up, we will be working with Rob Rozema’s class at GVSU to post our “This I Believe” essays to a Ning social network and get comments across classes. Then, after spring break, digital stories. As we continue on in the semester, I am looking forward sharing more ideas. It is interesting to compare the snapshots of two generations of teachers that I am seeing this semester — the pre-service students and the in-service teachers in Project WRITE — and compare how they are engaging with similar technologies in different ways. I feel as if with the pre-service teachers, they can pick up on the technology quickly once it is introduced, yet the conversations about pedagogy are still emerging. for the in-service teachers, we are able to talk about pedagogy very easily, but only after very thorough discussions of how and why to use the technology.

The differences are clear and makes me even more aware of the generational gap that must be happening as new teachers enter schools. They are very excited about the technology, yet can’t talk about it in pedagogically sophisticated ways. Veteran teachers are, as they should be, very concerned about pedagogy. This dichotomy makes me wonder how we can get everyone speaking the same language and beginning to think more about the pedagogy and the technology at the same time, regardless of age or experience. Then, we need to layer in discussions of literacy for everyone, because those are not present yet.

More teaching to be done, for sure and it is a great deal of fun in additional to a continual pedagogical challenge.

This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

Here is a video from Richard E. Miller, the Chair of the Department of English at Rutgers, explaining his thinking about the shift to a new vision of the humanities and how that vision will be enacted through physical space at the university. It certainly suggests some of the changes that we will have to make in our thinking, especially at the universtiy level.

One particular element of this video that makes it compelling is his idea about the missing piece of the Wikipedia puzzle, and what universities have to offer students as they make their way in a read/write web world.

My colleague Rob Rozema from GVSU has invited my students and I to participate in a new Ning social network, Teach English. I am very excited about the opportunity to be involved in this project, and we will also have students from Allen Webb’s course at WMU join in, too.

As we consider what we will do with this network, I think that we have to ask ourselves a key question about its implementation and potential for use: how do we account for and respond to the contradiction in local, state, and federal policies regarding internet use (for instance, no blogging or social networking) and the call to teach these skills in our schools?

In other words, if we teach students how to use social networks, will they be able to use those skills once they are teaching?

Moreover, this raises another issue that my best friend Steve Tuckey and I were discussing a few weeks back — does taking a technology and reappropriating it for use in schools undermine the excitement and potential uses for that technology?

As an example, we talked about the idea of a “cheese sandwich blog,” one that tells basically accounts for the mundane happenings in everyday life. (If we build 20 million blogs, will the readers come?). Contrast that with the more substantive kinds of blogging that many edubloggers are calling for and teaching; that is, a more “academic” form of blogging. Steve asks, what’s wrong with the cheese sandwich?

He asks this not to be sarcastic (well, OK, maybe a little bit), but more to take a critical approach to how we use blogging. From an email conversation, he says, in part:

by trying to call for highfalutin standards of rigor in what our students blog about, we are essentially trying to colonize one of the most democratic spaces with the self-important hierarchy of academia. We try to set up the same old benchmarks for “good writing” in a new environment, all the while touting the greatness of its promise as something “new.” Seems schizophrenic to me. And don’t get me started on how real-time authoring serves to feed the dragon of continuous assessment…

In other words, if we reappropriate “blogging,” into an academic setting, is it blogging anymore? Or, is the definition of “blogging” (or, perhaps, edublogging), such that a higher level of discourse is now becoming expected above and beyond the typical diary/journal/update blogs of the past. And, with microblogs in Facebook and Twitter, are we going to have to think about how to make that academic blogging, too?

Steve was interested in seeing me raise this point with the other edubloggers that are thinking about similar ideas, perhaps in another forum beyond our blogs, too. Perhaps I will write a letter to EJ or something like that. If others have an idea about where and how we might discuss this issues — the appropriate use and reappropriation of blogging for academic purposes — let me know. It will certainly be on my mind as I prepare for next semester.

This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

Yesterday, our RCWP Project WRITE team had the good fortune of being able to work with NWP’s Director of Research and Evaluation, Paul LeMahieu, on an analytic writing continuum workshop. In his talk, which was similar to the session that I attended last summer, he talked about how the continuum has been developed, the pedagogical uses of it, and how we, as professionals who teach writing, need to not just tell those who value tests to “stop,” but to also offer them something better to use instead (we hope to post some notes on the session soon on the Project WRITE wiki).

Particularly useful for the Project WRITE teachers, as he talked about the different categories for assessment on the continuum (content, structure, stance, diction, sentence fluency, and conventions — modeled, with permission, after six traits), he also talked about how this structure of assessment works for most kinds of writing, but not all and not the least of which is multimodal writing. He mentioned how there are not really any models that explore how to assess multimodal composition and how, perhaps, we could develop one through this work in Project WRITE. That is a very exciting component of this project that I had not anticipated when we originally started, and I look forward to pursuing it more soon. (NOTE: I do think that Bernajean Porter has got our thinking moving in this direction for K-12 students, and put up some good criteria and an interactive rubric maker on her Digitales Evaluation site.)

Coincidentally, I have been chewing on this idea now for the past few days as I was trying to help my students in ENG 201 come up with criteria for evaluating their final multimodal projects. As I asked them to reflect on what they have been doing and how they have been working over the past few weeks on these projects, we talked on Tuesday about how the categories of the analytic continuum (which we have been using all semester) just didn’t quite line up with what they were thinking about in terms of what to earn a grade on. Along with some criteria for judging group member performance, they went back to our discussions earlier this semester about rhetoric, and we came up with the following ideas for grading this project:

• Ethos: the credibility of the author is established through professional language, use of appropriate sources, and evidence of author’s perspective (within or in addition to the main multimodal documents)
• Pathos: the texts make appropriate emotional appeals that both engage the reader and provide insight into the chosen topic
• Logos: the texts present a clear and coherent central idea, supported with appropriate evidence and argumentative strategies
• Content and Structure: the choice of mode and media support the message in the texts and elements of multimedia are thoughtfully integrated into the project rather than as a gratuitous add-on
• Design: the choice of design principles (contrast, repetition, alignment, proximity) as well as rhetorical decisions (transitions, word choice, stance) combine to make an attractive and effective presentation

So, it will be interesting to see how this turns out. Students, in groups, will be assessing the other groups’ work and I will be throwing in my grade with the whole bunch to get an average. I haven’t graded anything multimodal yet, let alone a collaborative grading where students are involved in the process. I’ll write more about it once we are done, and look forward to hearing your ideas about how you are teaching and assessing multimodal writing, as well as any resources that you can point to about this messy, yet engaging, component of the writing process.

This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

Last month, TechCruch featured a story on a new web-based tutoring service, PrepMe. I was contacted by Calvin Truong, PrepMe’s Operations Manager about writing a post on the service here in my blog. From the TechCrunch review, it sounds like a different take on the model of submitting a piece of writing only to have it graded by a computer, a model that many, including Nancy Patterson (in this month’s Language Arts Journal of Michigan) and  Maja Wilson, have been critical of:

Prepme is one online test prep company coming out of the University of Chicago’s business incubator. Founded in 2001, the company offers test preparation for the SAT, PSAT, and ACT, using an adaptive algorithm to customize the preparation course for each student.

Unlike Kaplan’s online offering, Prepme doesn’t calculate the best lesson plan once, but continuously as you work your way through the material. Their system keeps track of what questions you get right and wrong, working you harder on the types of questions you miss.

Additionally, customers can connect electronically, using real time chat, with high scoring college students who serve as tutors.

Source: TechCrunch, “Starts-Ups Change How Students Study for Tests,” 9/1/07

When Calvin wrote to me, he wondered if I would blog about PrepMe here. I replied with some initial concerns:

Prepme does seem like an innovative service that takes advantage of computerized scoring while still adding the element of human judgment. Most of the outright computerized scoring systems out there really worry me as a writing teacher (as well as turnitin.com), so this is a clear departure that blends technology and pedagogy…

… although I do think that your service is innovative, I am still concerned about writing items that seem to support computerize scoring, as many of the professional organizations that I belong to have statements that expressly condemn computerized scoring.

NOTE: After some closer reading, I should note that the writing itself appears to be scored by the tutors, while multiple choice items that are like those encountered on the ACT, SAT, and other tests are the ones being computer graded.

To continue the conversation, Calvin wrote back immediately, and with his permission, I share parts of that response here:

About your concerns, I completely understand, and I think we’re pretty in-sync on both points. We’re working on a few initiatives that would speak more generally to trends in education, and since we’re only grading multiple choice tests and hand grading essays to enable detailed feedback it seems we’re on the same page. I would expect there is much less resistance in using technology to automate grading multiple choice exams since this minimizes human error, but perhaps I’m mistaken on this.

As to the interesting trends that may be worth writing about, there are two that we’re working on that may be of interest. The first is our work with the State Dept of Education in Maine, and the second is more generally about what’s happening in online education.

Maine recently enacted legislation that required the SAT to graduate from high school. We’ve committed to a 3 year program with the Dept of Education where we provide free test prep to every public high school student in the state. Here’s a press release from the Maine DoE website: http://www.maine.gov/education/edletrs/2007/ilet/07ilet072.htm. This initiative is interesting in and of itself, and may provide fodder for an interesting discussion. As far as we can tell they’re doing it to not have to invest tremendous resources to create their own state standardized test and to also drive students to consider applying to college. It’s an interesting social experiment and we’re proud to be a part of it.

Another approach might be to talk about the general trend in online education of trying to find the sweet spot between scalability, quality, and cost. We believe that using technology to give you scale while having high quality services with tutors from top universities, at a significant cost advantage is the way to go. In the pre-college market, having tutors at top universities is a quality win because these are exactly the sorts of students that our users want to be and exactly the sorts of students that our parents want their children to be, and this fosters great relationships online. There is some inherent cost in this approach but we believe it’s worth it.

Clearly there are others out there that disagree — some go for the no-compromise in quality, 1-on-1, in person is the only way to go but that has tremendous cost and little scalability. Other companies are trying the low cost online model with outsourced tutors and we believe this sacrifices too much quality in favor of cost.

It may be interesting to consider the implications of this because what is commercially viable may not actually be what is the most pedagogically pure.

All in all, it was an eye-opening discussion and gives me hope that hybrid models of online grading with humans sharing their insights could be a way to go. In my initial training as an online instructor for our state’s virtual high school, it seemed as though we relied more on the multiple choice items and writing that was highly scripted, almost not requiring a human response (even though I was grading it). As we ask students to write and share their writing online, this is not the best model of them composing digital texts per se, but it is a model that we could consider using in our own classrooms to foster peer response on traditional texts in digital environments.

Also, it points to the need in our field to more fully analyze this phenomenon and come up with alternatives that we feel are viable. I am not an expert in the topic of computerized writing assessment, yet am becoming more familiar with the field. A search in Google Scholar for “computer based writing assessment” didn’t yield anything since 2003 (in the first ten pages of hits). The most recent an comprehensive article that I saw was Goldberg, Russel, and Cook’s “The effect of computers on student writing: A meta-analysis of studies from 1992 to 2002,” originally published in the Journal of Technology, Learning, and Assessment. (An article in the current issue, found after I accessed the JTLA website, “Toward More Substantively Meaningful Automated Essay Scoring,” looks interesting, too).

So, I thank Calvin for beginning this conversation, and giving me something more to think about as I evaluate my students’ writing this week (all of it submitted digitally, incidentally) and consider what else might help them become better writers in the future. I also hope that you — as teachers of writing — share your thoughts both in comments here and by emailing Calvin as well.