Orwellian Japanese social media platform “Dystopia” uses AI surveillance and censoring to prevent cyberbullying 

Japanese web service developer Soudanbako released the beta version of their new social media platform called Dystopia on September 24 for iOS and Android. Dystopia uses automatic AI surveillance to censor content and was developed under the concept of “a social media platform where no one gets hurt.” The AI seems to have a rather heavy-handed approach in its censoring. 

Dystopia is described by its developers as an AI-controlled social media service in which AI detects any offensive phrases, including slander and harassment, and converts them into “appropriate alternatives.” The app’s purpose is to prevent users from getting hurt and to eliminate both victims and perpetrators of social media harassment. 

In response to the growing issue of online public shaming, frequent internet pitchfork mobs and smaller, everyday exchanges which can be hurtful and influence peoples’ lives, the developers of Dystopia decided to implement AI as a “truly neutral and fair” monitor of their platform. 

Similarly to other social media, Dystopia has a timeline where posts are published, but before any post goes online, it is screened and censored by the AI. The developers provide some “before and after” examples of how the AI executes this, but the changes are a bit dramatic, to put things lightly. One example is “Die, a*shole!” being converted into “My heart is full of boisterous revelry!” and another is “The next time I see you you’re so dead” being processed into “The next time we meet, I believe we will be able to spend a delightful time together!” 

The company also reveals being inspired by George Orwell’s novel “Nineteen Eighty-Four” and partially treating the project as a social experiment to see how the complete control of an AI “Big Brother” will affect people’s communication. While simply seeing the words “we’re using AI to prevent cyberbullying” gives a naïve impression, the project this time around seems to be self-aware with its cheeky naming sense and all.  

The idea of going onto a social media network knowing you’re about to be censored beyond recognition is definitely novel. Dystopia has been downloaded over 5000 times on the Play Store at the time of writing, and is currently at the very top of the social media app ranking on the App Store in Japan. I assume the app is going to see a lot of people posting the most putrid offenses they can think of to see how the AI chooses to fix them. 

The Dystopia beta version is available now on the Google Play Store and Apple App Store for free. The App Store description states that the app is available in English as well as Japanese.