SAN FRANCISCO—The Electronic Frontier Foundation (EFF) today unveiled the Red Flag Machine: an interactive quiz and report demonstrating the absurd inefficiency—and potential dangers—of student surveillance software that schools across the country use and that routinely invades the privacy of millions of children. 

The Red Flag Machine is the result of EFF’s investigation of GoGuardian, a tool used to surveil about 27 million students—mostly in middle and high school—in about 11,500 schools across the United States, according to the company. Like similar tools such as Gaggle and Bark, GoGuardian gives schools and the company access to an enormous amount of sensitive student data, while at the same time mis-flagging massive amounts of useful material as harmful.  

The investigation identified categories of non-explicit content that are regularly marked as harmful or dangerous, including college application sites and college websites; counseling and therapy sites; sites with information about drug abuse; sites with information about LGBTQ+ issues; sexual health sites; sites with information about gun violence; sites about historical topics; sites about political parties and figures; medical and health sites; news sites; and general educational sites. Interfering with students’ access to such sites deprives them of information they need to excel in their studies, remain healthy, or improve their lives. 

To illustrate the shocking absurdity of GoGuardian’s flagging algorithm, EFF built the Red Flag Machine. Derived from real GoGuardian data, users are presented with websites that were flagged and are asked to guess what keywords triggered the alert.   

“This project reveals the very real and very specific failures of student monitoring technology,“ said EFF Director of Investigations Dave Maass. „It’s one thing to be concerned broadly about student surveillance, but it’s an entirely different experience to see that students are ‚flagged’ for researching Black authors, the Holocaust, and the LGBTQ+ rights movement. It’s shocking, but it’s also absurd: Students have even been flagged for visiting the official Marine Corps’ fitness guide and looking at the bios of the cast of Shark Tank.“ 

By design, GoGuardian is also a mass surveillance system. It lets school officials track trends in student search histories, it identifies supposedly “at risk” students, and it gathers location data on where and when a device is being used, letting anyone with access to the data create a comprehensive profile of students who use devices with the software installed. When students try to access information that the software deems inappropriate, it flags students and sends alerts to school officials, parents, and potentially, police. 

As a result, school administrators and teachers using GoGuardian have nearly unfettered access to huge amounts of data about students, including browsing histories, documents, videos, app and extension data, and—in some circumstances—live views of students’ screens. This adds up to a stunning invasion of privacy.  

„School safety concerns have sparked unprecedented growth in use of ‘edtech’ software in recent years, but with surveillance-specific apps like GoGuardian and Gaggle, students are being taught that they have no right to privacy and that they can always be monitored,” said EFF Activism Director Jason Kelley. “Schools should be safe places for students, but they must also be places where students feel safe exploring ideas. Student monitoring software not only hinders that exploration but also endangers the most vulnerable, such as LGBTQ+ students and students of color.”   

EFF filed public records requests to obtain tens of thousands of results from GoGuardian, and analyzed data from schools in both liberal- and conservative-leaning states. The investigation found the tool’s “false positives” exponentially outweigh its ability to accurately determine whether a website’s content is harmful—leaving millions of students flagged for viewing content that is not only benign, but often educational or informative.  

Although this investigation focused on GoGuardian, it should raise questions about other, similar student-monitoring tools currently in use. 

“Knowing how you’re being surveilled—and how accurate that surveillance is—must be the first step to fighting back and protecting your privacy,” Kelley said. “We hope this research will help government officials, school administrators, parents, and students to force companies making this software to improve, or to abandon such software’s use entirely.”