SIO Trust and Safety Project

120521 7824

Platform Trust and Safety

The global technology industry is facing a major challenge in building trustworthy technology that lives up to the expectations of citizens, governments and the media. At the same time, academics, policy makers and concerned citizens are struggling to understand the options available to deal with problems inherent in billion-user networks and the interventions that could make a positive impact. Trust and Safety is the term used in Silicon Valley for the teams that research, find and stop “abuse” of their systems. Abuse is the technically correct use of a product to cause harm, in contrast to the field of information security, which is generally focused on preventing exploits that defeat technical protections around data and systems.

This moment is not completely unprecedented, as the software industry faced a similar crisis around information security in the late 1990s. One of the root causes of the failure of the technology industry was a lack of intellectual scaffolding and educational content aimed at the leaders of large companies and new startups alike. At the time, only a handful of computer science departments offered information security courses to undergraduates, and secure development techniques were never integrated into collegiate or post-collegiate software engineering curricula. Through this project, SIO leverages the experience of its team members and collaborators to fill that gap.


CS 152: Trust and Safety Engineering

An introduction to the ways consumer internet services are abused to cause real human harm and the potential operational, product and engineering responses. Students will learn about spam, fraud, account takeovers, the use of social media by terrorists, misinformation, child exploitation, harassment, bullying and self-harm. This will include studying both the technical and sociological roots of these harms and the ways various online providers have responded.