Peer Review in Public

This afternoon, partially as a way to procrastinate from my own writing and partially because I was genuinely interested in the invitation, I participated in an “open review” of Remi Kalir and Antero Garcia‘s forthcoming manuscript, Annotation. Their open review process will continue through August 23, 2019, so jump in! They request that commentary adhere to the following, all good advice for any scholarly dialogue:

Civil. We can disagree. And when we do so, let’s also respect one another.

Constructive. Share what you know. And build upon ideas that are relevant and informative.

Curious. Ask honest questions and listen openly to responses.

Creative. Model generative dialogue. Have fun. Contribute to and learn from the process.

Having read hundreds of academic articles in the past 20 years, as well as offering blind peer review for dozens more, as well as blind reviews of probably two dozen academic books, I thought that this would be interesting. (And, again, I was procrastinating on my own writing, so an engaging intellectual task that can carry me away and still feel like I am getting work done is always welcome). Here are a few things I learned while reviewing their book which, again, you, too, can contribute to through August 23rd.

My Stance as a Reviewer

When I offer peer review to academic articles and books, I am typically using the “track changes” and commentary features in Word or, in some instances, by offering comments and edits on a PDF (my favorite tool for doing that is the iOS app Good Reader). I typically frame these comments as direct suggestions to the author(s) of the article/manuscript I am reading, and I engage in a professional, yet conversational tone.

With my review of Annotation today, I think that I maintained some of that approach, yet I knew that my comments would be captured, in perpetuity, in Kalir and Garcia’s public version of the document. While I didn’t hold back with questions and concerns, I did realize that I changed my tone. Whereas I would try to be explicitly clear in comments and questions (perhaps even providing examples of what I was aiming for with unclear writing) in blind review, I didn’t want that to be part of the public record.

For instance, in the example below, I offered a comment that could spark further dialogue amongst others reading the text, pushing toward some broader implications for teaching and learning. At other points, I was replying to the comments already made by others, and I would specifically say something like “I agree” or “Along these lines.” Also, at points, I directly wrote to Kalir and Garcia in ways that I could do so with colleagues I know, and would be comfortable saying in front of a group of others.

Screenshot of Specific Comment on

My Commenting Style in an Open Setting

Yet, still, it felt strange. In the first few chapters, there were some other annotations/ors, yet they fell away. Even those that remained were offering suggestions for links, not the generative kinds of peer review that (I hope) I have always aimed to offer in the peer reviews that I complete. For instance, I would describe problems and ask questions like:

  • I may simply not be reading this right, but making the comparison of submitting an expense report in relation to the openly annotated future just didn’t ring for me here. Sorry, but perhaps you could find a different example?
  • This is an interesting example, but I don’t know that it fully draws out all the ideas that you mentioned above related to “shifting social norms, changing financial and organizational incentives, and evolving scholarly practices.” Perhaps you could reorganize around — and particularly elaborate upon — these three ideas in relation to SciBot?
  • This is an important, if technical, point, and deserves some elaboration. Why is it important that some are built into the browser, whereas others stand alone. And, for that matter, why have you not mentioned OneNote, Evernote, Google Keep, or SimpleNote anywhere in the text, and especially here before you launch into the important questions you pose below?

By the end of the process — which took me just as long as any other book review — I began to wonder/wander, leading me to other directions.

Reflecting While Reviewing

Of course, during a normal review, the kinds of internal dialogue that I have with myself may make it into the first draft of my comments, but I usually do some editing before a final draft heads off to the editor. Here, I figured that Kalir and Garcia’s invitation to be civil, constructive, curious, and creative would welcome some of these thoughts.

As I went through the process, and saw fewer and fewer reviewers in subsequent chapters, I got discouraged. While this is no fault of the authors, and I know that they have extensively shared their open manuscript, welcoming reviews, it does make me worry a bit about the hive mind, and whether the power of collaboration and collective intelligence is, perhaps, not as powerful as we might hope. A few of my musings, especially as they relate to why scholars may choose not to participate in an open review:

  • This [vision of social annotation and scholarship] is aspirational, and I appreciate it. Yet, I think that you can elaborate more on what actual changes would need to happen to make it a reality. Be specific, and talk about faculty workloads, department/college T&P requirements, and the ways in which “open” is still perceived as subpar.
  • And, yet, there still seems to be reluctance, or at least lack of widespread acceptance [of open review]. For instance, in your attempts to make this manuscript open and accessible (which I applaud), I am still wondering how many total scholars will participate. Even for those of us who saw the invitation to begin with, a gentle nudge was in order for us to participate. And, in the end, I don’t know that my review of this manuscript will “count” on par with doing a review for an established journal or publisher when (and if) I include it in my promotion materials. Of course, for me at least, this doesn’t matter as much as it would to a junior faculty member who needs to decide whether to spend a few hours trying to write her own work, or to participate in a “normal” editorial review board/process as a blind reviewer for an established press/journal. Both of those actions are rewarded in the academy. As much as I respect Remi and Antero (and that’s why I am doing this annotated review), the simple fact of the matter is that I am doing this because I care, not because it will “count.” These are part of the material reality of academe, and I don’t know how we will change that, even with open annotation and peer review. At the end, there is only so much time in the day…
  • So, I have held off until now, but I have to ask… and only partially in a cynical manner… Like the tree falling in the forest, does an annotation really make a sound (ripple, impact, effect, etc)? That is, I appreciate your utopian vision, yet I wonder if you might want to reign it in a bit here. Sorry… not trying to pop the bubble, especially after nearly two hours of reviewing and annotating your manuscript, but I am just being realistic. The first few chapters had a few annotators. Now, here at the end, it is just me. And you two, as the authors. Are we really connected to a “robust information infrastructure?” Or, are the three of us walking alone in the woods?

In the end, I appreciate the opportunity to do this review, and to pause here to reflect on the process. I struggle both with how to structure class discussions in digital spaces as well as how to be a social scholar, so reading Kalir and Garcia’s manuscript was serving many more purposes for me than merely procrastinating on my writing. I am hopeful that the ideas I have offered to them (and those who might continue to annotate over the next month) are helpful. And, of course, I will continue to think about practices of annotation in my own scholarship and teaching.


Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Review of Amplify by Katie Muhtaris and Kristin Ziemke

For a number of years now, I have been wanting to provide elementary colleagues with a book that offers a glimpse inside a classroom that runs as a digital writing workshop, one that truly embraces the principles of writing workshop pedagogy while integrating digital writing tools into day-to-day literacy practices. I have been fortunate to connect with many elementary educators who embrace the writing workshop approach with digital writing tools, but hadn’t yet seen a book that captured, in words and images, what a digital writing workshop might look like.

Amplify! Digital Teaching and Learning in the K-6 Classroom By Katie Muhtaris, Kristin Ziemke
Amplify! Digital Teaching and Learning in the K-6 Classroom By Katie Muhtaris, Kristin Ziemke

Then, last year, I found out that Katie Muhtaris and Kristin Ziemke were working on a book to be based on some of the techniques and strategies they share in their blog, “Innovate Ignite Inspire.” Knowing that they were doing this kind of smart work with their kids, I have been eagerly awaiting their book. The result is Amplify: Digital Teaching and Learning in the K-6 Classroom.

Now, having met Kristin quite some time ago at a Michigan Reading Association conference, I knew that she was an educator who was a bit skeptical about the use of technology, but wanted to integrate tech in productive, responsible ways. Or, as Stephanie Harvey describes it in the foreword of the book, though Katie had been enthusiastically integrating technology in her classroom for number of years and, subsequently, Kristin would “peek in, curious about how tech platforms might enhance learning in her first-grade classroom, but not entirely convinced” (vii).

Just as its title suggests, Katie and Kristin’s book does not supersede or replace existing literacy practices with technology-enhanced lessons. Instead, their goal is, indeed, to amplify best practices in reading and writing workshop, modeling literacy practices for their students, and moving them toward a hybridity of reading and writing in both print and digital spaces. As they explain:

Digital learning is at a crossroads, and it’s time for teachers and students to share our voices in how, why, and when our kids should use technology as a learning tool. We invite you to join us on a journey of discovery, exploration, and empowerment. (xii)

Their core principles are ones with which I, and countless other teachers, would certainly agree:

  • Use a workshop model for instruction
  • Hold small-group and individual conferences
  • Engage kids in cross-curricular content
  • Scaffold learning
  • Encourage collaboration and conversation
  • Drive instruction with assessment

These principles align with their overarching goal — “Technology in the classroom fits easily into this hands-on approach to learning (the writing workshop): our students should be the ones using it” (5).

They back these principles up with numerous examples, and I especially appreciate the way that they create “technology anchor charts” in much the same way they would when exploring a new genre, discussing reading strategies, or documenting a process. Also, they describe how they adapt the workshop model by adding in the element of “play” before a mini lesson. “Play,” they contend, “is collaborative, experiential, tactile, and active,” all ideas that lend themselves well to using technology (33).

The book itself takes the voice that we have come to expect in all Heinemann titles — respectful of teachers’ time, knowledge, and needs for high-quality professional learning and growth. Rather than providing a buffet of tech tools, Katie and Kristin actually focus their efforts on just a few key tools and processes: capturing ideas with Padlet, engaging students in a backchannel with Today’s Meet, teaching them how to record voice and video with a webcam and microphone. Throughout the book, there are suggestions that a teacher can “try tomorrow” with minimal technology knowledge.

As the book comes to a close, they share insights on reflection and assessment. Regardless of any number of digital tools at their disposal, Katie and Kristin remind us that

The simple act of giving ourselves permission to stop and watch opens our eyes to the rich fabric of learning in our classroom. We can examine the quality of the tasks we ask our students to undertake. What impact do they have? Why is this important? How can this be better? (90)

Amplify has provided elementary teachers a glimpse into the workings of what I would call a digital writing workshop and what Franki Sibberson has recently begun to call a “digital reading workshop” in Digital Reading: What’s Essential in Grades 3-8. Though I am curious as to why Katie and Kristin do not use that language, I imagine that they avoid adding the “digital” label to the work that they do for good reason — to keep the focus on reading and writing, thinking and learning. As we all continue to think about ways in which we can purposefully bring technology into the K-6 classroom, Amplify provides us with both the principles and practices for doing so.

NOTE: While I am a Heinemann author and did request a complimentary copy of this book, please know I am writing this review independently, not at the request of Heinemann or the authors.

Update: 12/10/15, 11:33 PM – Katie was kind enough to point out that I transposed two letters in “Padlet,” so that has been corrected.


Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.