How Location Technology Can Make Trust & Safety More Cost Effective

The world’s biggest tech companies are dedicating significant portions of their budgets to trust and safety initiatives, but is that money working to its fullest potential?

Trust and safety teams are specialist units dedicated to targeting pain points like digital trust deficits, user and content moderation, fraud detection and prevention, and more. Less than 50% of the users of some of the most significant tech giants, including Facebook, Tiktok, and Google, trust them to manage personal data, and that’s a problem that platforms are committing serious money toward solving. 

Twitter, Twitch, AirBnB, and Facebook are just a few examples of the major companies with some variation of a trust and safety initiative. T&S is a vital part of digital consumer relations in the present day, but today’s predominant trust and safety strategies may not be the most efficient use of the allotted budget.

Prioritizing moderation over identity: the flawed approach driving up trust and safety costs

Building a Trust & Safety team is expensive. In 2018 alone, AirBnB spent $150 million¹ on new initiatives, including a 24/7 hotline for renters to contact safety agents. Facebook has spent more than $13 billion² on safety and security since the 2016 elections, with about 40,000 employees working in this area. As of 2022, Facebook has committed 5% of revenue or $3.7 billion³ to content moderation alone. 

User and content moderation are some of the most resource-intensive parts of a trust and safety initiative. Even for a relatively small user base, moderators can never know who the bad actors are until after they’ve already broken community guidelines, making for a constant struggle to keep up with inappropriate content. Content filtration and moderation software, as well as user reporting systems, are meant to help with this massive undertaking, but this approach may be flawed from the very start. 

A bad actor banned for a singular post often has no trouble creating a brand new account and creating even more abusive content, requiring even more of the trust and safety team’s resources. 

Instead of focusing on the content layer, trust and safety teams should target bad actors at the identity layer - who is this person, and how can the platform ensure bans against this person stay permanent? 

By re-identifying the user, despite a new account, moderators can stop bad actors from reoffending or ban evading. When platforms focus on content moderation over identity, it’s all too easy for banned or suspended users to return to the platform under a different name and recommit abuse. 

Moderation on a post-by-post basis is only the tip of the iceberg–identifying bad actors that return despite being banned helps moderation teams go deeper and stop the problem at the root. 

Bad actors evade bans by purchasing new devices with new phone numbers–sometimes even hundreds in operations called fraud farms–or creating new email addresses. For many social media platforms, user onboarding is a streamlined process to promote a frictionless user experience, but unfortunately, this makes things easy for ban evaders as well. 

Fortunately, there are ways that these platforms can target bad actors at the identity layer without increasing onboarding friction for their good users. A powerful combination of location and device intelligence is the key. 

Location technology for a more cost-effective trust and safety approach

Two main signals can support trust and safety initiatives in a cost-effective manner: location intelligence and device intelligence. While other aspects of a person’s digital identity–email addresses, for example–are easily replaced, location and device intelligence aren’t so easily circumvented. 

Mobile devices and the locations people use them in tend to be very personal. It’s rare to see two different people who share the same mobile device. If location signals are able to identify that the device behind a new account creation was previously used to create or manage a banned account, that’s a strong indicator that the new account holder might be intending to again publish bad content on the platform. 

Location is a powerful identifying signal for the same reasons. Even if a bad actor uses a new phone, or several, they aren’t likely to relocate as well. By linking different devices to the same location, it’s possible to identify the same individual regardless of new devices. By combining these two signals—device and location—platforms are able to accomplish much better identity affirmation at the beginning of the user journey without the friction of a full KYC check.

It’s important to note that not all location signals are created equally: IP-based geolocation and GPS alone are not accurate or spoof-resistant enough for trust and safety purposes.

It’s very easy for fraudsters and other bad actors to spoof both their IP address (using VPNs or proxy servers) and their GPS location (using an emulator or a GPS spoofing app that anyone can download).  Additionally, the accuracy shortcomings of GPS means that administrators would have to block entire buildings to target the apartment of one individual. 

However, when GPS is combined with other powerful indicators such as nearby Bluetooth and WiFi signals and mobile device intelligence, it can create a spoof-resistant solution that’s accurate down to the apartment level. Using this location technology in combination with device location, administrators uncover bad actors’ hiding places. 

Spoof-resistant location identity can be used to block or watchlist certain users and locations. Even if an offending user accesses a new device to circumvent a ban, their location will give them away every time. 

Using this solution, platforms can target fraud farms, ban evaders, bots, and bad actors using multiple burner accounts. 

Location targets bad actors at the identity layer and ensures they can’t come back, reducing the amount of money and resources trust and safety teams must spend moderating repeat offenders. 

¹https://www.bloomberg.com/news/features/2021-06-15/airbnb-spends-millions-making-nightmares-at-live-anywhere-rentals-go-away

²https://www.theverge.com/2021/9/21/22685863/facebook-safety-security-staff-spending-misinformation-abuse-wall-street-journal-reports 

³https://knowledge.wharton.upenn.edu/article/social-media-firms-moderate-content/#:~:text=Indeed%2C%20content%20moderation%20efforts%20eat,than%20Twitter's%20entire%20annual%20revenue