Directed Reading in Content Moderation

James Grimmelmann

Cornell Tech

Spring 2022

Overview

This is a directed reading course on contemporary issues in content moderation on online platforms. The readings will cover:

• The dynamics of individual and crowd behavior online that create content-moderation problems
• The substantive content-moderation standards applied by major platforms
• The procedure of content moderation: who does it, and how
• The design space within which content moderation operates
• The legal framework governing content-moderation decisions, and proposals for change
• The implications for platform regulation across various legal domains

Course Logistics and Policies

The course will meet weekly. See the course policies for information on what I will do to make this course welcoming and beneficial for you, and what I expect of you to make it welcoming and benficial for each other.

Contact

This syllabus is at http://james.grimmelmann.net/courses/content2022S.

Email: james.grimmelmann@cornell.edu
Huddle: Bloomberg 370
Desk: Bloomberg 3 NW, near the bookshelves

The best way to meet with me is by appointment. Send me an email and suggest some possible times. I am happy to meet in person or to talk by phone or Zoom.

I hold drop-in office hours Mondays 2:30 to 3:15 PM in my huddle.

It is always fine to swing by to see if I’m free. I am generally around on Mondays and Wednesdays. But be warned that when I am on campus my days are usually packed with classes and meetings.

Assignments

Your work for this class will consist of the following:

First, do the assigned readings and participate in class discussions.

Second, follow current events in content moderation. Read the following newsletters. (I will provide you with paid subscriptions to the ones that are not free.)

• Everythng in Moderation
• Garbage Day
• Money Stuff (only the parts about tech and social media)
• Platformer
• Rest of World
• Stratechery (free weekly articles web version only)

Third, you will be responsible for helping to compile an annotated bibliography in connection with a virtual workshop I will be conducting over the summer. (You are welcome to attend, but it is not part of the assgned work for this course.) In keeping with the law school’s requirements, the total written work product will be approximately 15-20 pages per particpant.

Grading

This course is graded pass/fail (technically S/U in the Cornell system). If you complete the assigned readings, particpate constructively in discussion, and do your portion of the writing, you will pass.

Resources

For additional readings and references on content moderation, you may find some of the following helpful:

• MSR Social Media Collective reading list
• TSPA Resource Library

Schedule

We meet Tuesdays 10:00 AM to 11:00 AM.

The list of readings may change as the semester progresses. The field is not standing still, and neither will we.

January 25: Moderation Basics: What and How?

• Kate Klonick, The New Governors, 131 Harvard Law Review 1598 (2018). Skim the introduction and parts I and IV, but read parts II and III.
• Facebook Community Standards. Read all of the policies listed on this page. Then click through to “Other Policies” and skim the following:
  • Advertising Policies
  • Pages, Groups, and Events
  • Branded Content Policies
  • Commerce Policies
  • Recommendation Guidelines

February 1: The Moderation Design Space

• James Grimmelmann, The Virtues of Moderation, 17 Yale Journal of Law and Technology 42 (2015). Skim part IV.
• Eric Goldman, Content Moderation Remedies.
• Jialun “Aaron” Jiang, Charles Kiene, Skyler Middler, Jed R. Brubaker, and Casey Fiesler, Moderation Challenges in Voice-based Online Communities on Discord, 3 Proceedings of the ACM on Human-Computer Interaction art. 55 (2019)
• Adrian Chen, The Laborers Who Keep Dick Pics and Beheadings Out of Your Facebook Feed, Wired (Oct. 23, 2014)
• Sarah T. Roberts, Digital Refuse: Canadian Garbage, Commercial Content Moderation and the Global Circulation of Social Media’s Waste, 10 Wi: Journal of Mobile Media, 1 (2016).

Additional resources:

• Robyn Caplan, Content or Context Moderation? Artisanal, Community-Reliant, and Industrial Approaches (Data and Society 2018)

February 8: User Participation in Moderation

• Kate Crawford and Tarleton Gillespie, What is a Flag For? Social Media Reporting Tools and the Vocabulary of Complaint, 18 New Media and Society 410 (2016)
• Dariusz Jemielniak, Common Knowledge? An Ethnography of Wikipedia (2014), chapters 2 and 3
• Sarah Myers West, Censored, Suspended, Shadowbanned: User Interpretations of Content Moderation on Social Media Platforms, 20 New Media and Society 4366 (2018)
• R. Stuart Geiger, Bot-Based Collective Blocklists in Twitter: The Counterpublic Moderation of Harassment in a Networked Public Space 19 Information and Society 787 (2016).

Additional resources:

Lucas Wright, Automated Platform Governance Through Visibility and Scale: On the Transformational Power of AutoModerator, 8 Social Media + Society 1 (2022)

February 15: Notice and Takedown

• Jennifer M. Urban, Joe Karaganis, and Brianna Schofield, Notice and Takedown in Everyday Practice (ver. 2, 2017). Skip the appendices.

February 22: Section 230

• Felix T. Wu, Collateral Censorship and the Limits of Intermediary Immunity, 87 Notre Dame Law Review 293 (2011)
• Daphne Keller, Internet Platforms: Observations on Speech, Danger, and Money (Hoover Institution 2018)
• Olivier Sylvain, Platform Realism, Informational Inequality, and Section 230 Reform (Yale ISP Digital Public Sphere 2022)

March 1: NO CLASS

March 8: Real and Fake

• Danielle K. Citron and Robert Chesney, Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security, 107 California Law Review 1753 (2019). Skim Part III, but read the rest.
• Britt Paris and Joan Donovan, Deepfakes and Cheap Fakes: The Manipulation of Audio and Visual Evidence (Data and Society 2019).
• Claire Wardle, Fake News. It’s Complicated, First Draft (Feb. 16, 2017).
• Brian X. Chen, The Reason Your Photos Are About to Get a Lot Better, New York Times, Oct. 15, 2019.
• Michael Sippey, Even If It’s Fake, It’s Real, Sippey.com (Nov. 24, 2010).

Additional resources:

• Joel Snyder, Picturing Vision, 6 Critical Inquiry 499 (1980), pages 504 to 514.

March 15: Virality and Commerce

• James Grimmelmann, The Platform is the Message, 2 Georgetown Law Technology Review 217 (218).
• Nicholas Confessore, Gabriel J.X. Dance, Richard Harris, and Mark Hansen, The Follower Factory, New York Times (Jan. 27, 2018).
• Alexis C. Madrigal, The Strange Brands in Your Instagram Feed, The Atlantic (Jan. 10, 2018). Recommended but optional: Jenny Odell, There’s No Such Thing as a Free Watch (Aug. 18, 2017).
• Nicole Nguyen, Inside Amazon’s Fake Review Economy, BuzzFeed News (May 7, 2018)
• Renee DiResta, How Amazon’s Algorithms Curated a Dystopian Bookstore, Wired (Mar. 5, 2019)

March 22: Coordinated User Action

• Rebecca Lewis, Alice Marwick, and William Clyde Partin, “We Dissect Stupidity and Respond to It”: Response Videos and Networked Harassment on YouTube, 65 American Behavioral Scientist 735 (2021)
• Sarah Myers West, Raging Against the Machine: Network Gatekeeping and Collective Action on Social Media Platforms, 5 Media and Communication 28 (2017)
• Renee DiResta, Crowds and Technology, Ribbonfarm (Sept. 15, 2016)
• Chris Peterson, “Bury Until They Change Their Ways”: The Digg Patriots As User-Generated Censorship, Civic Media Project (Apr. 8, 2016)

March 29: Influence Operations

• Adrian Chen, The Agency, New York Times, June 2, 2015.
• Paul Charon and Jean-Baptiste Jeangène Vilmer, Chinese Influence Operations: A Machiavellian Moment (2021). Read the executive summary and chapter 2 section IX (“Information Manipulation,” pages 362 to 400).
• Facebook, Threat Report: The State of Influence Operations 2017–2020 (2021).
• Stanford Internet Observatory, Reply-Guys Go Hunting: An Investigation into a U.S. Astroturfing Operation on Facebook, Twitter, and Instagram (2020).
• The Verification Handbook (Craig Silverman, ed.).

April 5: NO CLASS

April 12: Privacy

• Helen Nissenbaum, The Meaning of Anonymity in an Information Age, 15 The Information Society 141 (1999)
• Oliver L. Haimson and Anna Lauren Hoffmann, Constructing and Enforcing “Authentic” Identity Online: Facebook, Real Names, and Non-Normative Identities, First Monday (June 6, 2016)
• Evgeny Morozov, Capitalism’s New Clothes, The Baffler (Feb. 4, 2019)
• Hanna Bloch-Wehba, Content Moderation as Surveillance, Berkeley Technology Law Journal (forthcoming)
• Outside Looking In: Approaches to Content Moderation in End-to-End Encrypted Systems (CDT 2021)

April 19: Competition

• evelyn douek, The Rise of Content Cartels (Knight First Amendment Institute 2020)
• Ben Thompson, A Framework for Regulating Competition on the Internet, Stratechery (Dec. 9, 2019)
• Peter Swire, The Portability and Other Required Transfers Impact Assessment (PORT-IA): Assessing Competition, Privacy, Cybersecurity, and Other Considerations (draft 2020). Read pages 1-30.
• Mike Masnick, Protocols, Not Platforms: A Technological Approach to Free Speech (Knight First Amendment Institute 2019)
• David S. Evans, Governing Bad Behavior by Users of Multi-Sided Platforms, 27 Berkeley Technology Law Journal 1201 (2012)

Additional resources:

• Herbert Hovenkamp, Antitrust Interoperability (draft 2022)

April 26: Due Process and Transparency

• Kate Klonick, Inside the Making of Facebook’s Supreme Court, New Yorker, Feb. 12, 2021
• Caitlin Vogus and Emma Llansó, Making Transparency Meaningful: A Framework for Policymakers (CDT 2021)
• Rory van Loo, Federal Rules of Platform Procedure, 88 University of Chicago Law Review 829 (2022)
• Daphne Keller, User Privacy vs. Platform Transparency: The Conflicts are Real and We Need to Talk About Them, Stanford Center for Internet and Society Blog (April 6, 2022)
• evelyn douek, Content Moderation as Administration, Harvard Law Review (forthcoming)

May 3: Must-Carry and Must-Remove

• Danielle Keats Citron, How to Fix Section 230, Boston University Law Review (forthcoming)
• Dawn Carla Nunziato, Protecting Free Speech and Due Process Values on Dominant Social Media Platforms, Hastings Law Journal (forthcoming)
• Eugene Volokh, Treating Social Media Platforms Like Common Carriers?, 1 Journal of Free Speech Law 377 (2021)
• evelyn douek, Australia’s ‘Abhorrent Violent Material’ Law: Shouting ‘Nerd Harder’ and Drowning Out Speech, 94 Australian Law Journal 41 (2020)

May 10: Everything Is Content Moderation

• Matthew Prince, Why We Terminated Daily Stormer, Cloudflare Blog (Aug. 16, 2017).
• Carrie Battan, How Politics Tested Ravelry and the Crating Community, New Yorker, Mar. 22, 2021.
• Andrew Marantz, Reddit and the Struggle to Detoxify the Internet, The Yorker, Mar. 12, 2018.
• Thorsten Busch, Kelly Boudreau, and Mia Consalvo, Toxic Gamer Culture, Corporate Regulation, and Standards of Behavior among Players of Online Games, in Video Game Policy (2015)
• Nora A. Draper, Distributed Intervention: Networked Content Moderation in Anonymous Mobile Spaces, 19 Feminist Media Studies 667 (2018)
• Tiffany Hsu, Pinterest Bans Climate Misinformation from Posts and Ads., New York Times, April 6, 2022.
• Karen Levy and Solon Barocas, Designing Against Discrimination in Online Markets, 32 Berkeley Technology Law Journal 1183 (2017)
• Katherine Rosman, Ben Sisario, Mike Isaac, and Adam Santorino, Spotify Bet Big on Joe Rogan. It Got More Than It Counted On., New York Times, Feb. 17, 2022.