Political Science Educator: volume 27, issue 1
Assignments and Course Design
Ruth Castillo, Emory and Henry College, Sarah Fisher, Emory and Henry College, and Kayce Mobley, Bethany College
Political scientists have long recognized the importance of the news media in shaping foreign policy and public opinion (Entman 2004; Baum 2002). However, the discipline possesses room for improvement for teaching media literacy in the classroom. While some quality media literacy assignments for the international politics classroom exist (Brown 2021), most recent scholarship focuses only on introductory US politics courses (Feezell 2021; Mancillas and Brusoe 2016; Schiffer 2021). Teaching and studying international politics create additional hurdles when compared to US-centric classes. First, foreign press may be the primary news source and thus remain unfamiliar to US audiences. Second, state censorship and state sponsored media further confound the ability to understand news from different parts of the world.
We experimented with an approach to help students think critically about sources that highlighted additional issues related to how students think of credibility in the news. All materials for these lessons are available through a Creative Commons license and can be accessed freely online.[1] In this essay, we reflect upon our experience.
Supplemental resources: 1. Kimberley MacVaugh - Incorporating Information Literacy Skills into Your Syllabus
Vertical Reading Techniques
Common checklists taught at colleges and universities, such as CRAAP, RADAR, and RADCAB, direct students to read the source itself to determine its reliability. This process relies on vertical reading that seeks out surface level information from the source itself by reading the source in question (Caulfield 2018; McGrew et al. 2019; Wineburg and McGrew 2019). Wineburg, et. al. (2020) found that these methods not only failed to help students identify misinformation, but they actually led students to mislabel poor sources as reliable.
Lateral Reading Technique
As an alternative to vertical reading, lateral reading (or fact-checking through other sites) has been shown to increase students’ ability to correctly identify the reliability of different sources (Brodsky et al. 2021; McGrew et al. 2019; Wineburg and McGrew 2019). One specific method for teaching lateral reading includes The Four Moves, also known by the mnemonic SIFT (STOP, Investigate the Source, Find better coverage, Trace claims, quotes and media back to the original context) (Caulfield 2019a).[2] When teaching students to practice these strategies, they practice lateral reading by opening separate browser tabs from their original source and investigating information from other sources before committing to reading the information (Wineburg and McGrew 2019).
Lateral Reading in Practice: Introduction to International Relations
We decided to test the results of vertical versus lateral reading in the context of current events about China in introductory international relations courses.27 The instructors introduced these concepts and the SIFT method (Caulfield 2019b) through a short video (Lateral Reading 2020) and discussion. Next, the instructors divided students into groups to complete the vertical and lateral reading in-class assignment. The instructors identified four articles: two from legitimate sources (Mastro 2020; Stevenson 2022), and two from much less reputable sites (Catenacci 2021; Zhang 2022).
Two articles (Stevenson, 2022; Zhang, 2022) examined how Hong Kong is dealing with the Covid-19 surge. Instructors chose the New York Times in part because they guessed students would find this source legitimate given that articles from this source had been discussed previously in this course. This also allowed for discussion of a potential/perceived liberal bias. The China Daily, on the other hand, was not a source that had been assigned or discussed previously in this course. The article itself did not have overt bias, and if a student read vertically, they would not necessarily realize that the China Daily is state-run (Fitfield, 2020). Two other articles came from the Council on Foreign Relations and WND, formerly WorldNetDaily (Catenacci, 2021; Mastro, 2020). The Council on Foreign Relations is a respected U.S. think tank. WND is a site known for spreading conspiracy theories (Massing, 2009) and thus had not been assigned for either class previously. Both articles related to territorial disputes in the South China Sea.
After discussing Chinese censorship in the context of comparing lateral reading to vertical reading, faculty divided students into random groups. Each group examined two articles: one using lateral reading, and another using vertical reading. In addition to reading, students completed an editable Google document as a group.
Worksheet Analysis
The results of the worksheet assessment were mixed. Even when instructed to use lateral reading, some students saw the articles from China Daily and WND as valuable. Others, however, questioned their legitimacy. Students generally found the New York Times and Council on Foreign Relations articles to be more reliable. Figure 1 captures some student comments from the activity.
Figure 1. Sampled Responses to Articles from Students
| China Daily & WND | New York Times & Council on Foreign Relations | |
| Vertical Reading | Positive:
● There isn’t a clear bias. They seek to give you solely the truth. They claim they are an independent news channel that seeks truth and justice. They also state they are open to free and open debate. ● On the website we clicked they linked other credible websites. We think it is credible because they cited more than one. Neutral or Mixed: ● From reading it we don’t see any visible bias, but there is a tab for correction, which could lead to misinformation. Negative: ● The website seemed cheap with ads everywhere on the page. It also only quoted people instead of writing their own article. ● My first impression is that the article reads as if it is click bait, trying to create drama that may not be there. The website has a lot of links to seemingly politically biased articles. One linked headline states “Democrats Think more with Their Emotions.” |
Positive:
● It seems like a credible, and well established website. There are graphs which cite research institutions, and pictures of events/interviewed people. There’s also a subscription, which means enough people trust it to pay for their service ● From looking at the website, it doesn’t seem like they are biased. Most of their headlines seem neutral/uncharged. They have a separate column for opinions Neutral or Mixed: ● The linked article does not appear to have any bias but the articles linked under “More on China” all have negative titles concerning China. Negative: ● [No purely negative assessments] |
| Lateral Reading | Positive:
● It seems somewhat credible. The story was originally published on another website. It seems professional. It has a similar layout to the other article, and they have several links to outside sources (including the Associated Press). Neutral or Mixed: ● They tweet and talk about the daily things that happen in China, almost like a news site. Negative: ● Other sources[3] say that WND is a fake news website. ● When first looking at the site, it seems to me that there are just a lot of odd things that were added into this source. At first sight I immediately knew the article was biased. This is because it was in the China daily. We know China censors it’s media so the observation is logical |
Positive: ● Wikipedia[4] says it is ranked the 18th newspaper in the world which would lead you to believe it is a credible source. ● Most sources are saying that the New York times is the most reliable News paper source in the United States. It should be able to be trusted because of its reliability and how many people read it. ● The council is a non-profit and non-partisan organization made in the 1900s. They have a high accountability and transparency rating. According to sources they have a clean fact check record as well. They have representation and members from all over the world. The members are diverse and from all different regions. Neutral or Mixed: ● These sources are saying that the NYT is a long and storied newspaper, but it has been accused of liberal bias in the past, especially their editorials. Negative: ● [No purely negative assessments] |
Note: Quotes modified by embedding links instead of listing them to save space.
In total, these comments suggest that students can critically consume information, but that the skills related to media literacy need to be built and reinforced throughout a student’s college career. As a department, political scientists could work to incorporate media literacy into all of their courses. Faculty members could partner with librarians and other media professionals to design activities and lessons for upper and lower division courses to meet student needs.
Citation Analysis
In addition, we conducted a citation analysis with a newly constructed rubric. This methodology for assessment exists in multiple disciplines (Carbery and Leahy 2015; Dobbs 2017; Flaspohler et al. 2007; Goodman et al. 2018; Tuñón and Brydges 2006). Drawing from these established rubrics (Flaspohler et al. 2007; Goodman et al. 2018; Tuñón and Brydges 2006), the new rubric (available online[5]) assesses student-found news articles through source type, relevance, and trust factor on a four-point scale for a total of 12 points per source. We analyzed two samples of student work through the rubric. The first assessment followed the in-class activity, and the next assignment came four weeks later. Across the two assignments, 37 students utilized 73 sources. On average, students scored 10.52 out of 12 points on the rubric for selecting news sources. Figure 2 summarizes the results.
Figure 2. Citation Analysis Results
| Source Type (Out of 4) | Relevance (Out of 4) | Trust Factor (Out of
4) |
Average Source
Score (Out of 12) |
Average Student
Score (Out of 12) |
|
| Quiz directly following in- class activity
(n=18) |
3.59 |
3.59 |
3.59 |
10.76 |
10.76 |
| Assignment due four weeks after in-class activity
(n=19) |
3.29 |
3.61 |
3.43 |
10.32 |
10.32 |
Conclusion
We found lateral reading to be a useful technique for teaching media literacy in international politics classrooms. The approach proves relatively easy for students to grasp, and it lessens the chance of them falling for bogus sources. We have also found the technique to be adaptable to many different classes and assignment formats. The lessons are worth reiterating in all the classes that we teach.
View more media literacy teaching resources here: 1. Jennie Sweet-Cushman - Want to Build Students’ Civic Engagement? Teach Them How to Use Social Media 2. Barbara Robertson and Tamra Ortgies-Young, Critical Thinking, Information Literacy and Democracy: An Interdisciplinary Approach to Tackle Misinformation and Prepare Students for Active Citizenship
Endnotes
[1] https://drive.google.com/drive/folders/1YIqDiOi25AbGs2u77peMqa83Yy6pi9Lm?usp=share_link
[2] Caulfield’s SIFT method is built on the work of Wineberb, McGrew, et al. with the Stanford History Education Group’s Civic Online Reasoning project (https://cor.stanford.edu/curriculum/collections/teaching-lateral-reading/). Both methods are for the purpose of teaching students to evaluate social and political information online irrespective of the media format.
[3] https://www.splcenter.org/fighting-hate/extremist-files/group/worldnetdaily
[4] https://en.wikipedia.org/wiki/The_New_York_Times
[5] https://drive.google.com/drive/folders/1YIqDiOi25AbGs2u77peMqa83Yy6pi9Lm?usp=share_link
References
Baum, M. A. 2002. “Sex, Lies, and War: How Soft News Brings Foreign Policy to the Inattentive Public.” American Political Science Review 96(1): 91-109.
Brodsky, Jessica E., Patricia J. Brooks, Donna Scimeca, Ralitsa Todorova, Peter Galati, Michael Batson, Robert Grosso, Michael Matthews, Victor Miller, and Michael Caulfield. 2021. “Improving College Students’ Fact-Checking Strategies through Lateral Reading Instruction in a General Education Civics Course.” Cognitive Research: Principles & Implications 6(1): 1–18.
Brown, Colin M. 2021. “Information Literacy Exercise – Comparative Democracy.” APSA Educate. October 18. Retrieved June 13, 2023 (https://educate.apsanet.org/resource/10-18-2021/information-literacy-exercise-comparative-democracy).
Carbery, Alan, and Sean Leahy. 2015. “Evidence-Based Instruction: Assessing Student Work Using Rubrics and Citation Analysis to Inform Instructional Design.” Journal of Information Literacy 9(1): 74–90. https://ojs.lboro.ac.uk/JIL/article/view/LLC-V9-I1
Catenacci, Thomas. 2021. “Chinese Military Forces U.S. Warship out of South China Sea: Report.” WND, July 12. Retrieved March 7, 2023 (https://www.wnd.com/2021/07/chinese-military-forces-u-s-warship-south- china-sea-report/).
Caulfield, Mike. 2018. “Recognition Is Futile: Why Checklist Approaches to Information Literacy Fail and What To Do About It.” Hapgood, February 18. Retrieved November 30, 2021 (https://hapgood.us/2018/02/18/recognition-is-futile-why-checklist- approaches-to-informa tion-literacy-fail-and-what-to-do-about-it/).
———. 2019a. “Introducing SIFT, a Four Moves Acronym.” Hapgood, May 12.
Retrieved November 30, 2021 (https://hapgood.us/2019/05/12/sift-and-a- check-please-preview/).
———. 2019b. “SIFT (The Four Moves).” Hapgood, June 19. Retrieved November 30, 2021 (https://hapgood.us/2019/06/19/sift-the-four-moves/).
Dobbs, Aaron W., ed. 2017. The Library Assessment Cookbook. Chicago: Association of College and Research Libraries, A division of the American Library Association.
Entman, Robert. M. 2004. Projections of Power: Framing News, Public Opinion, and U.S. Foreign Policy. University of Chicago Press.
Feezell, Jessica T. 2021. “An Experimental Test of Using Digital Media Literacy Education and Twitter to Promote Political Interest and Learning in American Politics Courses.” Journal of Political Science Education 17(sup1): 634–48. https://doi.org/10.1080/15512169.2019.1694531.
Fitfield, Anna. 2020, January 15. China is waging a global propaganda war to silence critics abroad, report warns. The Washington Post. Retrieved June 13, 2023 (https://www.washingtonpost.com/world/asia_pacific/china-is-waging- an-aggressive-propaganda-campaign-to-distort-media-landscape-report- warns/2020/01/15/30fd4d58-374a-11ea-a1ff-c48c1d59a4a1_story.html).
Flaspohler, Molly R., Erika M. Rux, and John A. Flaspohler. 2007. “The Annotated Bibliography and Citation Behavior: Enhancing Student Scholarship in an Undergraduate Biology Course.” CBE—Life Sciences Education 6(4): 350–60. Retrieved December 5, 2022 (https://www.lifescied.org/doi/10.1187/cbe.07-04-0022).
Goodman, Xan et al. 2018. “Applying an Information Literacy Rubric to First-Year Health Sciences Student Research Posters.” Journal of the Medical Library Association 106(1): 108–12. (September 21, 2022).
“Lateral Reading.” 2020. UofL Research Assistance & Instruction, June 26. Retrieved March 7, 2022 (https://www.youtube.com/watch?v=GZvsGKvqzDs).
Mancillas, Linda K., and Peter W. Brusoe. 2016. “Born Digital: Integrating Media Technology in the Political Science Classroom.” Journal of Political Science Education 12(4): 375–86. https://doi.org/10.1080/15512169.2015.1096792.
Massing, Michael. 2009, January. “Un-American.” Columbia Journalism Review. https://www.cjr.org/essay/unamerican_1.php
Mastro, Oriana Skylar. 2020. “Military Confrontation in the South China Sea.” Council on Foreign Relations, May 21. Retrieved March 7, 2022 (https://www.cfr.org/report/military-confrontation-south-china-sea).
McGrew, Sarah et al. 2019. “Improving University Students’ Web Savvy: An Intervention Study.” British Journal of Educational Psychology 89(3): 485–500. Retrieved November 30, 2021 (https://onlinelibrary.wiley.com/doi/abs/10.1111/bjep.12279).
Schiffer, Adam J. 2021. “Teaching Media Bias: The Case of the Trump Presidency.” Journal of Political Science Education 17(sup1): 955–72. https://doi.org/10.1080/15512169.2021.1897600.
Stevenson, Alexandra. 2022. “Zero-Covid Policy Shakes Hong Kong’s Economy and Its ‘Soul.’” The New York Times, January 21. February 15, 2022 (https://www.nytimes.com/2022/01/21/business/hong-kong-covid-supply- chain.html).
Tuñón, Johanna, and Bruce Brydges. 2006. “A Study on Using Rubrics and Citation Analysis to Measure the Quality of Doctoral Dissertation Reference Lists from Traditional and Nontraditional Institutions.” Journal of Library Administration 45(3–4): 459–81.
Wineburg, Sam, Joel Breakstone, Nadav Ziv, and Mark Smith. 2020. “Educating for Misunderstanding: How Approaches to Teaching Digital Literacy Make Students Susceptible to Scammers, Rogues, Bad Actors, and Hate Mongers.” In Working Paper A-21322, Stanford History Education Group, October 21.
Retrieved November 30, 2021 (https://purl.stanford.edu/mf412bt5333).
Wineburg, Sam, and Sarah McGrew. 2019. “Lateral Reading and the Nature of Expertise: Reading Less and Learning More When Evaluating Digital Information.” Teachers College Record 121(11): 1–40. Retrieved November 30, 2021 (https://www.tcrecord.org/content.asp?contentid=22806).
Zhang, Kathy. 2022. “Multiple Initiatives to Help Fight HK’s Worsening Virus Outbreak.” China Daily, February 15. March 7, 2022 (https://www.chinadaily.com.cn/a/202202/15/WS620b03caa310cdd39bc86a 5b.html).
Ruth Castillo, MLIS, is the Director of the Library at Emory & Henry College in Virginia. Her research and work experience includes integrated information literacy, library instruction, and managing library services for on-campus and online learning, web services, and reference & research support.
Sarah Fisher, PhD, is an Associate Professor of Politics and Department Chair at Emory & Henry College in Virginia. She enjoys working with great colleagues on creative teaching projects like this one.
Kayce Mobley, PhD, is an Associate Professor of Political Science and Department Chair at Bethany College in West Virginia. Her favorite courses she has developed in recent years include Dystopian Politics, The 9/11 Era, Women in the World, and Modern Political Thought: Bioethics and Politics.
Published since 2005, The Political Science Educator is the newsletter of the Political Science Education Section of the American Political Science Association. All issues of the The Political Science Educator can be viewed on APSA Connects Civic Education page.
Editors: Colin Brown (Northeastern University), Matt Evans (Northwest Arkansas Community College)
Submissions: editor.PSE.newsletter@gmail.com
APSA Educate has republished The Political Science Educator since 2021. Any questions or corrections to how the newsletter appears on Educate should be addressed to educate@apsanet.org
Educate’s Political Science Educator digital collection
