�鶹ӰԺ

Reading and understanding academic articles are critical parts of social science education. Academic articles help students’ learning in the following ways. First, they complement instructors’ lectures, sometimes by providing some redundancy and at other times by providing supplementary materials. Second, they promote critical thinking by tracing through scientific investigative processes, which students can evaluate. Third, they promote reading skills by helping students learn to read efficiently and productively.

Despite these benefits, students do not engage with their reading assignments effectively. Most students read at a level too cursory to reap the benefits I outline above, and their absorption of the material is minimal at best. However, instructors teaching classes larger than small seminars cannot devote significant time to reading and dissecting reading assignments in class, as such an endeavor would be both highly time consuming and have uncertain payoffs for students.

While I do not have data to support my intuition, I suspect that students’ engagement with reading assignments has fallen over the years. (There is certainly data that students spend less time on coursework than they used to.) One possible factor is instructors’ increasing reliance on digital forms of academic articles as opposed to printed and bound readers. The former makes it difficult for students to grab a pen, mark up readings and actively engage with the reading. Students are more likely to passively skim the readings on their computer, simply clicking or scrolling to turn the pages.

Technological tools have emerged and are emerging to address these deficiencies. Touch-screen-based tablet computers and e-readers, in conjunction with good software, can closely emulate the experience of reading printed articles with a pen and a highlighter. The affordability of these tools, however, is likely to be still too low for most students. A contrasting approach is taken by tools that transform reading into a more social experience. For example, allows users in a group to collaboratively mark up any PDFs uploaded. Students can mark up a reading by highlighting passages, posting questions or remarks linked to the mark-ups and respond to one another’s mark-ups, questions and remarks. The tool turns reading away from being a solitary endeavor to a collaborative one.

With the onslaught of information that students confront everyday, their attention span is likely getting shorter increasingly. Yet, the benefits that they would reap from actively engaging with academic articles remain high. This presents a compelling reason for instructors both to increase the attractiveness of reading academic articles and to facilitate students’ attempts to critically engage them.

In the initial post, I explored the need for using collaborative reading as a way to facilitate students’ absorption of academic articles. In the follow-up post, I examined the learning outcomes that collaborative reading could produce and how any improvements in the outcomes could be linked to the use of the technology. In this post, I outline a plan for implementing collaborative reading via NB—an online tool enabling students to collaboratively mark up and annote PDF-based readings—in my two courses next semester (Spring 2013).

One course I will be teaching is Political Science 4193-705 “International Political Economy.” A particular challenge for this course is that it is a Global Residential Academic Program (GRAP) course, to be facilitated in a residence hall for 18 mostly freshman students. Although the freshman students will be in their second semester of college education, they will have had significantly less exposure to reading academic articles than upper-division students. This challenge, however, will also provide an opportunity to teach students how to read academic articles not simply to read collaboratively.

Despite the unconventional setting, composition and size of the course, the overarching aim of the course remains the same as other versions of the same course I teach—for students to understand and become knowledgeable about the political and economic causes and consequences of how the international economy functions. To achieve this goal, I will incorporate collaborative reading (through NB or other comparable technological solution) as one of pedagogical tools. Starting in the third week of the semester, I will assign one article a week for students to read collaboratively. Actively utilizing the collaborative reading system will be mandatory, and students will be required to make at least one comment and pose one question per assigned reading. Successfully completing this requirement will constitute 10% of their course grade. Instead of immediately jumping into complex academic articles, however, I will slowly ratchet up the difficulty of the reading. During weeks three, four and five, students will collaboratively read relatively short journalistic accounts about some international economic phenomena. This will get them used to using the tool and to exploiting the social and collaborative aspects of the technology. Then, during weeks six through ten, students will collaboratively read popular academic writings—for example, those contained in the journal _Foreign Affairs_. I will encourage students to identify authors’ central argument and evidence that they advance to support their argument. Lastly, during weeks eleven through fourteen, I will assign articles from top academic journals. By this point, students should be accustomed to collaborative reading and to looking for argument and evidence and thus ready to take on a more difficult task.

The other course I will be teaching is Political Science 7073, a graduate seminar on “Global Political Economy.” Although I have been piloting collaborative reading and developing plans for using it in the context of undergraduate courses, the technology can be usefully implemented in a graduate seminar. Running an effective graduate seminar poses a challenge in that faculty members usually do not know how much of the assigned readings students absorbed and what overlapping concerns, comments or questions students have about the readings and the topics for the week. Collaborative reading will facilitate everyone coming to class in agreement about what the agenda for discussion should be. A possible resistance for implementing the collaborative reading system at the graduate level might be that students have developed their own preferred way of reading, from which they are unwilling to deviate. To acclimate and persuade students of the utility of collaborative reading, I will start with minimal requirements during the early weeks—one comment and one question per reading—and slowly ratchet up the requirements throughout the semester.

In my previous post, I explored the need and value for exploring new models of facilitating students’ reading of academic articles. In this post, I examine some specific learning outcomes that we can pursue through collaborative reading.

The broad outcomes I strive for in incorporating collaborative reading into my class are greater breadth and the depth of students’ reading. I would like to see a higher number of students meaningfully engage the assigned academic readings and students engage the readings beyond the shallow level at which they typically do the assigned readings.

If collaborative reading in fact promotes these outcomes, I expect to see more students coming to class having done the reading and students coming to class prepared to engage in discussions and debates beyond the superficial main points of the reading assignments. I expect parallel outcomes in online venues as well. On both the NB collaborative reading system and course discussion forums on D2L course website, students who do not typically participate in class should be more likely to comment on and raise questions about the readings. Moreover, I expect the quality of students’ posted comments and questions to be higher than what they raise in the classroom. More formally, I anticipate that students will be better able to incorporate readings into their exam answers, because they will be more likely to have done the readings and to have understood them better.

An ideal research design would involved a randomized experiment. Students would be randomly divided into two different sections. In one of the two sections (the treatment section), students would utilize the collaborative reading system. In the other (the control section), students would do the assigned readings in the traditional manner—individually. Then I can compare the breadth of student participation in class discussions and compare the mean exam scores between the two sections.

In lieu of this ideal design, I can employ a research design that exploits over-time variation in each of my two sections. Prior to the midterm, students did not use the collaborative reading system because of the technical barriers of incorporating the system into their workflow. After the midterm, I provided an extra incentive to help students overcome the barrier and to begin using the collaborative reading systems. Although the nature of the material covered is substantially different between the two periods, I can attempt to crudely compare students’ engagement and performance.

The formal assessment tools are numerous:

• Count of students actively participating in discussion in each class session
• Count of students who do not participate in class but posting comments/questions on the collaborative reading system
• Extensiveness of classroom and online discussions, where extensiveness is measured by back-and-forth exchanges among students on a topic
• Quality of student responses on exam questions—students’ summary of readings should be more accurate and students should be able to refer to parts of the readings beyond the main point.

As I argued  at the beginning of the academic year, reading is an important part of social science education. However, most students typically do not read the assigned readings with the frequency, interest and depth necessary to provide the foundation for further learning in the classroom. There are certainly “low-tech” solutions, such as the the “poker chips in the classroom”  that Will Moore implements. I have, however, pursued a different solution during this year, using a novel tool for collaborative reading called NB .

In short, NB allows users to collaboratively mark-up readings that instructors upload to the shared class space. While the discussion interface looks very similar to that of most online forums in existence, the innovation of the service is that students can highlight and annotate whatever parts of readings that they would like comment on. My goal in using this system was that by making reading more collaborative and social, students would engage more with assigned readings and come to class better prepared to participate in discussions.

After pilot-testing the service in my two undergraduate lectures during the fall 2012 semester, I fully incorporated the NB service as part of my undergraduate and graduate seminar during the spring 2013 semester. In this post, I discuss the successes and the lessons coming out of using the service in the undergraduate class.

The undergraduate class in which I implemented the NB service provided a challenging context. While the course and the substance—international political economy—were generally for upper-division students, I taught the class to a small group (18 students) of overwhelmingly first-year students. As part of a residential academic program (RAP), the class was taught in a special classroom in a dormitory that housed the program.

To specifically target the audience, I built up the difficulty of reading assignments throughout the semester. For the first seven weeks, I assigned a news article from the New York Times  or the Economist  each week. During weeks 8 through 12, I assigned popular academic/policy articles from Foreign Affairs . Lastly, in weeks 13 and 14, students read technical academic articles pertaining to the topic of the week. Students' usage of the NB service—indicated by their annotations on each reading—constituted 10 percent of their course grade.

Students generally enjoyed and appreciated using the NB service. 7 out of 12 respondents chose 4 or 5 on a five-point scale on the question of how much they enjoyed using the service, and 7 out of 8 respondents chose 4 or 5 on the question of how much they appreciated  using the service. Providing a more convincing response on the usefulness of the service, 9 out of 12 respondents wanted their future instructors to use the service.

The overall effectiveness in the NB service facilitating students' understanding and absorption of the readings was inversely proportional to these levels of difficulty in the reading assignments. When asked about what types of articles students felt more confident about reading and understanding compared to the beginning of the semester, among 12 respondents, 10 students selected newspaper articles, 8 students selected policy articles and 5 students selected academic articles.1  6 out of 12 students either agreed (3) or strongly agreed (3) that the NB serviced increased their likelihood of doing the reading assignments, whereas 3 students disagreed. 5 out of 12 students either agreed (3) or strongly agreed (2) that the serviced helped them better understand the readings. 6 students neither agreed or disagreed, whereas one student disagreed. Although these numerical results do not indicate an overwhelming success, they do provide sufficiently positive effects to build on in my future classes.

Students' written feedback, however, indicate that I achieved some of the goals I sought to achieve. For example, one student wrote “I liked seeing everyone thoughts about the article. It allowed me to see it from a different perspective.” Similarly, another student commented “I was able to read the comments from others and see how they were reading and understanding the readings so it helped get my brain working in different ways.” Another student commented “Posting the questions and comments allowed me to understand the reading better and feel as though I was part of the discussion instead of just reading an article online and not being able to mark it up.” Not all written comments pointed to the effectiveness of the service, however: “I didn't really read my peer's comments.”

The written comments in the survey also pointed to areas for improvement, although some of the deficiencies are inherent to the technological tool itself. One student commented on the inability to physically take notes on the readings: “The NB system was an interesting tool, but I am better able to complete readings, especially those which are rather lengthy, when given hard copies which I can physically take notes on.” Despite the ubiquity of PDF-based readings, many students still prefer to print out articles to read and mark up. More generally, one student found reading on a computer screen difficult: “With the longer readings, the system maybe even decreased my understanding, due to my lack of concentration for reading things from a computer screen.”

One actionable area for improvement is increase interaction in the comments and questions. While some students responded to one another's questions and comments, most annotations on the readings were singletons, without any further comments following up on them. Although I intentionally chose to not participate in the discussions and annotations, at least one student thought it would be helpful if the instructor participated in the comments and replies. In my future classes utilizing the NB service, I will require students to respond to at least one comment or question in addition to raising their own.

______________________________

Note: Each student could choose multiple answers for this question.