In an era of mass misinformation, how should we teach students meaningful online research skills? Recent groundbreaking legislation mandating media literacy education in K-12 schools has tasked New Jersey educators with finding a way. But now, new technology is changing the conversation. In the span of just a few months, artificial intelligence chatbots such as ChatGPT have become integrated with popular search engines and they are changing how internet users write, research, and perhaps even think. As the New Jersey Board of Education works to establish statewide media literacy standards, we urge policymakers and educators to consider the ethical implications of this technology. When thinking about media literacy, we must think about AI literacy, too.   

As university educators who teach college writing, we know our students face real hurdles in accessing quality information. Their difficulty cannot be attributed to novice research skills alone. Thanks to intentional design decisions made by powerful corporations, students often encounter Google search results littered with ads, and paywalls shaped by algorithmic biases and search engine optimization hacks. These corporations are the same ones driving decisions about AI chatbots’ role in our classrooms, and their rapid integration into search engines like Microsoft’s Bing and Google will likely make teachers’ jobs harder.  

The future internet landscape is poised to be drastically different from the one we know today: an internet where content produced by generative AI is ubiquitous, and where engagement with an AI chatbot is a customary, or even required, mediator of users’ access to information. Effective media literacy education in the era of AI will therefore require teaching students not only how to use these tools but also how to question, critique, and at times, even resist them.  

School districts across the country have been grappling with what to do about ChatGPT. Stoked by fears of widespread cheating, some districts, like Clifton Public Schools, have decided to ban ChatGPT until further review. Others have embraced it, dismissing plagiarism concerns and instead encouraging educators to see AI chatbots as an opportunity to “play,” However, there are meaningful reasons, beyond student cheating, to resist the hype surrounding AI chatbots and to critically question the role of these tools in our classrooms.   

Ethical dangers 

To better understand AI chatbots’ implications, we turned to computer scientists and other AI experts in the field, many of whom, it is worth emphasizing, are women and people of color. They found significant ethical dangers, which include:  

• Fabrication and misinformation: Due to the design of its language processing model, ChatGPT — as well as other large language model chatbots — generate inaccurate and completely fabricated outputs. Their untrustworthiness ranges from wrong answers to basic questions, to errors in equations and code, to fully fabricated quotes and citations for works of academic prose. These fabrications are indistinguishable to the reader unless they are fact-checked; 
• Plagiarism and sourcing: Despite reporting that suggests advanced chatbots only produce unique and original phrasing, recent investigations show that they will directly copy passages of existing text. ChatGPT-generated text does not provide attribution or credit sources that form the basis of its content, and OpenAI does not receive permission or consent to use the data scraped from the internet to train the model;
• Racism and bias: Because ChatGPT and other large language model chatbots are trained on a huge corpus of text with deeply entrenched human biases, their responses can reflect and reproduce bias, stereotyping, racism,  and even hate speech. Filters and counterprogramming intended to block biased output are easily bypassed; 
• Exploited labor: Tech companies leading AI chatbot development “depend on heavily surveilled gig workers,” many of whom are recruited from developing nations and paid below the poverty line to complete menial yet often traumatic AI training tasks. One investigation recently revealed that OpenAI paid data labelers in Kenya $1-2 per hour to review violent, graphic and racist depictions in order to improve ChatGPT’s text processing;  
• Environmental cost: Along with labor, training large language models requires tremendous amounts of energy derived primarily from non-renewable sources. Owned by and serving the interests of powerful and well-resourced tech companies, these tools are least likely to benefit the marginalized communities who face the most immediate negative consequences of climate change.  

We hesitate to ask students to use a tool that requires so many ethical compromises. Nor should teachers be expected to radically change their teaching practice to accommodate the product of a secretive company with questionable ethics.  

While AI undoubtedly poses a challenge for New Jersey education, it also offers an opportunity. One of the goals of the legislation is the “ethical production of information.” As New Jersey takes the lead on media literacy education, it should also lead on ethical AI instruction. AI literacy starts with resisting the assumption that these tools are inevitable and must be embraced. We should instead embrace our expertise as teachers, not surrender it to powerful technology companies whose investment in our students’ education remains dubious. Generative AI and large language models may indeed change how we teach. But it is up to us to decide how.