SIO Trust and Safety Project

The global technology industry is facing a major challenge in building trustworthy technology that lives up to the expectations of citizens, governments and the media. At the same time, academics, policy makers and concerned citizens are struggling to understand the options available to deal with problems inherent in billion-user networks and the interventions that could make a positive impact. Trust and Safety is the term used in Silicon Valley for the teams that research, find and stop “abuse” of their systems. Abuse is the technically correct use of a product to cause harm, in contrast to the field of information security, which is generally focused on preventing exploits that defeat technical protections around data and systems.

This moment is not completely unprecedented, as the software industry faced a similar crisis around information security in the late 1990s. One of the root causes of the failure of the technology industry was a lack of intellectual scaffolding and educational content aimed at the leaders of large companies and new startups alike. At the time, only a handful of computer science departments offered information security courses to undergraduates, and secure development techniques were never integrated into collegiate or post-collegiate software engineering curricula. Through this project, SIO leverages the experience of its team members and collaborators to fill that gap.

Outputs

• The first undergraduate Trust and Safety Engineering course
• An open source textbook on Trust and Safety Engineering
• Blog posts on evolving trust and safety issues observed on platforms