Category: Peer Review

Another great session this week, this time with one of the co-authors of the Writing Next report: Steve Graham.

Here is an overview from the MSU LARC site:

Steve Graham, Vanderbilt University

Evidence-Based Practice in Writing – Drawing on Experimental, Qualitative, and Single Subject Design Research for Answers

Wednesday, April 16, 2008
11:30am – 1:00pm
Room 133F Erickson Hall, Michigan State University

This presentation will examine what we know about effective writing instruction, drawing on three recent reviews of the literature. One of the reviews (Writing Next) was a meta-analysis of experimental and quasi-experimental writing intervention research. Another review was a meta-analysis of single-subject design writing intervention research. The third review was a meta-synthesis of qualitative research conducted with outstanding literacy teachers, designed to identify common practices across studies. Advantages and disadvantages to the use of evidence-based practices in writing will also be explored.

About the Speaker:

Steve Graham is the Currey Ingram Professor of Special Education and Literacy, a chair he shares with Karen R. Harris. His research interests include learning disabilities, writing instruction and writing development, and the development of self-regulation. Graham is the editor of Exceptional Children and the former editor of Contemporary Educational Psychology. He is the co-author of the Handbook of Writing Research, Handbook of Learning Disabilities, Writing Better, and Making the Writing Process Work. In 2001, Graham was elected a fellow of the International Academy for Research in Learning Disabilities. He is the recipient of career research awards from the Council for Exceptional Children and Special Education Research Interest Group in the American Educational Research Association.

And, here are some notes from the session:

• Opening quote: “Kids know the most interesting things” – Mark Twain
  • “It hurt, the way your tongue hurts when you accidentally staple it to the wall.”
• Writing is nowhere in terms of the educational reform movement in this country
  • The things that drive the educational reform movement are reading and math
  • Now, STEM – science, technology, engineering/economics,math
  • Why is writing out in the cold?
    • This is not always bad, as it sometimes results in school practices that are not good
    • But, we need to make the case that writing is important
      • 1. One of the reasons that people are not paying attention to writing is that there is a general perception that we do not know how to teach writing. Policy makers want evidence, and they want particular kinds of evidence.
        • We do know that there are some things that work for all students 4-12 and younger
        • People don’t think that writing is important. So, we have to look at the effects of writing on content area learning. We make the case that writing can be helpful in terms of the STEM skills
        • Reading gets more play in the literacy discussion. We need to look at the effects of writing on reading. How does writing affect reading?
          2. What are the practices going on in elementary and secondary schools
          • Limitations: survey data that could be rosy, but the data is still not good
          • ELA teachers are doing less than one extended writing assignment a month
          • You don’t wan to go into policy making without good research to make recommendations

          3. Theoretical framework — from Patricia Alexander from moving from knowledge about discourse and enhancing motivation

• What are three primary resources we can draw from?
  • Professional writers
    • Unfortunately, the advice can be simplistic and only moves confident writers to expert writers; it doesn’t help other writers
  • Effective practices from experienced teachers
    • Talk to effective teachers or observe good teachers in practice and study them
      • Problem: if I go in looking for one thing, I will likely see it (difficult to separate the wheat from the shaft”
      • Problem: Donald Graves and the example that works. Yet, there are times when this doesn’t work.
      • Problem: generalizability. Evidence is often selective.
      • With scientific studies, we collect evidence, presents findings for all participants, replicability, strength of impact — all this leads to something that should be more trustworthy than insight and experience.
• This presentation, thus, will draw on three sources: experimental, single subject, and teacher practice
  • Other criteria:
    • Four replications
    • Converging evidence (the sun, the moon and the stars align)
    • Recommendations based on higher quality studies are superior
      • Process writing has very poor research, so you need to be cautious about this
      • The more studies, the merrier
  • Effect size:
    • .8 is large
    • .5 is moderate
    • .25 is small, but significant
  • Writing Next looks at overall quality of writing
    • Strategy instruction (planning, revising, editing, and regulating the writing process; 20 studies, .82 effect size (particularly helpful for kids who find writing difficult)
      • Don’t just PEE (post, explain, and expect) students need repeated modeling
      • For instance, the STOP strategy (Suspend judgment, Take a side, Organize ideas, Plan more as you go)
    • Teaching Summarization (systematic and explicit teaching of how to summarize texts); 4 studies, ? (missed it) effect size
      • Teach the six rules of summarization
    • Peer assistance (working together to plan draft and revise); 7 studies, .75 effect size
      • Needs to be a structured in a positive way — having students add questions marks and carats in their peers’ papers
    • Setting product goals (specific goals for the written product to be completed); 5 studies, .70 effect size
      • Need to tell students what you expect without limiting them
      • Product goals and revising
    • Word Processing (using word processing); 18 studies, .55 effect size
      • Some are short studies, but some are up to a year
      • Using the technology which is widely available is important, but it is used infrequently in schools or, when it is used, it is only used for final draft/publication
    • Sentence combining (constructing more complex sentences by combining shorter kernel sentences); 5 studies, .5 effect
      • Work on this together with students, then invite them to apply it back in their own writing
    • Process Approach (extended opportunities for writing, student ownership); 21 studies, . 32 effect size
      • Inviting students to engage in planning and revising is good
      • Bad news: the effect size is scattered all over the place
      • Receiving training from NWP is about a .46 effect, and is insignificant if you did not get that training
      • You can do this in a very poor way, and not get a good effect; this is compatible with a strategy approach that makes the writing more visible
    • Pre-Writing (have students engage in activities such as brainstorming; 5 studies, .32 effect
      • STOP strategy, for instance
    • Inquiry (old research); 5 studies, .32 effect
      • No pre-test done, so these studies may underestimate the effect size
        • Example: set a goal, analyze the data, look at specific strategies, and apply what you learned
          • A student in elementary school looking at conflict on the playground
    • Study of Models
      • Examines examples of specific writers and types of text; 6 studies, .25 effect
        • Model from good readings
    • Writing as a Tool for Learning (writing in the content areas); small but positive effect
      • 26 studies, but I think that it is more effective in science and math than ELA and social studies based on the effect sizes that we see
    • Grammar (explicit teaching of grammar); 11 studies, -.32 effect size
      • Quality of writing is not affected by grammar instruction
      • What this traditionally looks like is that you give a definition, example, and then is used in decontextualized works
      • If we expect it, but do not help students use grammar then it will likely not work
        • Take the kernel sentence: Dog bit mailman

      Recommendations for Struggling Writers (teaching handwriting, spelling, and typing to struggling writers — teaching transcription skills towards automaticity), small positive effect

• Single Subject Design Recommendations
  • Explicitly teach students strategies to construct paragraph; strong positive impact
    • Showing parts of a paragraph to the point that students understand the goals of writing a paragraph
  • Explicitly teaching students how to capitalize, punctuate, etc. helped
  • Reinforce positive aspects of students writing — social praise, tangible reinforcement or both as a means to increasing specific writing behaviors (small positive effect)
    • Traditional means of grading papers doesn’t work — “we get more with honey than we do with vinegar”
    • Couldn’t draw the summary effect from this, however
    • Need to move the feedback beyond the specific paper and help the student move forward in his/her writing
  • Self-monitoring (students asked to count how many errors they made); might be effective for some struggling writers
• Individual Teachers
  • Study exceptional teachers and schools
    • Practice had to be applied by the majority of schools or teachers
    • 10 Practices that might make a differences (had to occur in four or more studies)
    • Dedicate time to writing and writing instruction, with writing occuring across the curriculum
      • Get kids in the game of writing
      • Increasing writing by itself is not enough, it also needs to be motivating and give kids tools to be effective
    • Involve students in various forms of writing over time
    • Treat writing as a process
    • Keep students engaged by involving them in thoughtful acticvities such as planning compositions
    • Vary individual, small, and large group instruction
    • Mode, explain, and provide guided assistance when teaching
      • Teachers need to relinquish control
    • Provide just enough supprt so that students can make progress or carry out writing tasks and processes, but encourage students to act in a self-regulated manner as much as possible
    • Be enthusiastic about writing and create a positive environment where students are constantly encouraged to try hard, believe that the skills and strategies that they are learning will help them write well
    • Set high expectations
    • Adapt writing assignments to meet the needs of students
• Caveats
  • We should not order these practices hierarchically in terms of one being more effective thananother
    • Instead we should order them in a way that we see them working well for us
  • The database is thin
  • Just because a practice has been studied, it does not mean that it will be effective for all teachers in all classrooms.
    • Pay attention and see if it works in your classroom, with your students
  • Little data on those students who are most at-risk: ELL, learning disabilities, struggling writers
  • Lack of data on maintenance and generalization
  • Don’t really know best how best to put all of these things together
    • Think about trying to integrate some of these ideas as part of an overall approach rather than try to fit it into an existing approach
  • Teachers’ views on acceptability of these practices will clearly influence their use — this will include the issue of domain specificity
    • If you don’t accept it as a reasonable practice for you in your classroom it will not work
  • Just because a practices is effective in a study or was used by an exceptional teacher does not mean that it will always work
• Questions
  • 6 traits
    • Most studies were pre- and post-tests with no control
    • Look at journal article on Writing Next
    • 6 plus 1 looked pretty good for what was there
  • In-Service
    • When we asked ELA, science, and social studies teachers about how well their program taught them to teach writing, 70% said it was inadequate
    • We also asked about in-service preparation — you personally, school, conferences — ELA said that 70% were adequate, but 30% were inadequate
    • Most science and other content teachers didn’t feel prepared to do so
    • Not doing it at pre-service level because most states do not require a course in teaching writing
  • We have been doing this work for nearly 25 years and we have not delivered our work in terms of learning strategies approach and outreach
    • We have a distribution problem — we are not providing what we know in pre-service and in-service ed
  • A lot of this is very complicated, so we did the best practice book to give something for teachers to look at
    • We need to have support materials showing teachers how to do this — if you can see it, you can do it

My Reflections

In thinking about Dr. Graham’s talk, there are a number of salient points that I want to consider. First, he went over the 11 strategies from Writing Next and, even though there is evidence to show that all these strategies are effective, it is the individual teacher that makes the difference in writing instruction.

Second, he talked about how students can use word processing to write and revise, and that is very effective for their growth as writers; however, most of the opportunities that students have to write with the computer only involve typing in a “final draft” of something else that has been written out beforehand.

Next, he talked about peer editing and how students must be scaffolded into the process of giving feedback; just having them give comments to one another is not enough as they must use the language of writing in that talk.

Finally, he talked about the writing process approach and having an authentic purpose and audience for students should happen more often than what it is. Typically, the audience is only within the classroom walls, and students don’t share beyond their friends. Yet, he described a project in his children’s school in which students shared their work more widely and that it could be a goal for many, although not all of our assignments.

This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.