TW: This is going to be a fairly, long detailed article aimed at laymen about anti-cheat from the perspective of a game programmer and systems analyst. I hope I can remove some common misconceptions and put everyone on the same page. By the end of this word salad, I hope that everyone can understand why this was a poorly-researched decision by VRChat’s management.
I am what most people would consider to be a typical programmer: Some guy who lives out of a deliberately darkened room for 16 hours a day pounding gibberish into a keyboard and making computers do strange things. During the COVID-19 shutdowns, I decided to pick up a Valve Index VR Headset and try getting some socialization in via an online social network called VRChat. While VRChat is far from perfect, it’s a great way to decompress, sit down with a bunch of your friends, and just bullshit the night away. I generally try to force myself to get online once every evening.
I’ve been away from civilization for a a few days to visit my folks, have a birthday, and get my car fixed. While I was out of town and on a crappy 768kbps rural ADSL connection, VRChat made the decision to suddenly – and with little warning – add Easy Anti-Cheat to their game.
Context: Clients, Mods, and Rippers, Oh My!
VRChat is a popular social game, and one of the reasons for that popularity is its considerable array of user-generated content and diverse arsenal of compatible VR headsets. Everything from the Valve Index (the most expensive consumer-grade headset currently available) to the Oculus Quest (basically a head-mounted Android cellphone hardwired for Facebook) are supported by VRChat through SteamVR and Oculus SDK.
Unfortunately, VRChat was rapidly developed, with proper security and content protection as an afterthought. VRChat uses Unity3d as its game engine, which isn’t a bad decision, but many of the features that it offers are merely placeholders designed for rapidly prototyping a game, getting swapped out with in-house or commercially-licensed replacements. For this reason, certain systems are vulnerable or just plain broken.
One painful example that continually haunts VRChat is the cache: Assets (avatars and maps) are streamed over HTTPS (what you’re using to view this website), and then stored in a cache. The cache is implemented using Unity’s built-in streaming asset caching system (as VRChat’s own attempts tended to overflow and fill up users’ disk space). While it works very well, it causes another problem: Unity’s caching system stores each file as is, with no encryption and no chunking. The only protection they rely on is that the name of each file on disk is seemingly gibberish, but which users have already figured out. This means that malicious users can easily steal someone’s content by loading the file in the Unity editor using a single function. This is called ripping.
Another feature of the engine is that Unity runs on something called Mono, which is an open-source variety of .NET. Mono permits the game to be written in simple languages like C# or Visual Basic. A feature of Mono and .NET is something called reflection, where you can pull up data about the code itself, load code into the game from disk, or even modify existing code while the game is running. Reflection is a powerful tool with all kinds of honorable applications, but one of the most common uses in Unity is modding.
Modding in Unity consists of somehow convincing the game to load and execute code that the user provides, usually in the form of a DLL file. This code is called a mod or a plugin, depending on the game or mod framework used. This mod, when executed, will then change aspects of the game and even load other code from disk or the web (the latter of which is stupidly dangerous, stop it emm/requi).
The problem with modding is that, since the code is often provided by a third party, there’s usually no vetting process to weed out malicious code, and there’s no guarantee of safety. Many companies who have never heard of a liability waiver consider this a problem. VRChat is one of these companies.
Many mods are also malicious in nature, allowing users to crash others through malformed packets, broken avatars, or invalid API requests, and some mods also include code that is dangerous to the person using them, including things like keyloggers, trojans, and ransomware. These can generally be avoided by not engaging in malicious activity, and using well-known safe distribution points who vet content.
“Clients” in VRChat are just mods with an ego.
Modding in VRChat
VRChat has gone through many changes over the years, and the modding community has had to keep pace by making their own changes.
The most recent popular modding community uses a system called MelonLoader to trick Unity into loading a DLL that then injects Mono mods into the game (after checking them for obvious bad things like scrambled/obfuscated code). MelonLoader’s community then offers a curated list of open-source, code-reviewed mods that users can install with a package manager called MelonAssistant.
Other communities exist, but tend to build themselves around malicious activity or ripping, as MelonLoader tries to identify malicious mods and refuse to start up if detected.
Why Mod A Social Game?
One of the most common misconceptions is that modders only mod VRChat to crash users or rip content. In fact, the most popular mods for VRChat are safety mods, so called because they try to eliminate known security flaws (reported to VRChat ages ago) such as NaN floating point vectors or rotations, and malformed audio packets for an audio system so old it was pulled from the Unity Asset Store years ago.
Other mods provide useful tools, such as the ability to adjust audio falloff so it is easier to focus on the people you are talking to, the ability to remove annoying or seizure-inducing shaders, a system to provide text-to-speech or speech-to-text, et cetera.
There’s even quality-of-life mods that make it easier to organize avatars, view a list of avatar pedestals in a world with potentially hundreds of them, block or hide avatars by author, and even just remove post-processing effects and chairs on worlds without toggles.
EAC Has Joined the Lobby
EAC (Easy Anti-Cheat) is a commercial plug-in anti-cheat solution. I am not an expert in how it works, as I don’t have a setup yet for kernel-level serial debugging, but from analysis of other anti-cheat systems and experience in trying to play other games with it enabled, as well as VRChat’s public announcements, it scans the game to ensure only approved system DLLs are loaded, and then scans all processes to ensure one isn’t trying to run anything it doesn’t recognize.
However, EAC and VRChat have a problem, and it’s very, very simple. Let’s leave EAC’s capabilities alone, ignoring any accusations of kernel injection, drivers, issues with one program or another or anything else, and let’s focus on the root of the problem, from a big-picture view:
Anti-Cheats are not a solution. They are a tool that are meant to be used with other tools, such as security reviews, content protection, and decent reporting apparatus.
EAC and VRChat
EAC will likely detect half-assed malicious mods that don’t have bypasses. This is all well and good, but the problem is that VRChat has user-generated content. UGC can be (and is) deliberately and accidentally corrupted in such ways to produce crashes, poor performance, or even provoke medical conditions, and EAC does not inspect UGC.
Unfortunately, due to an ancient design decision by VRChat, UGC is stored as assetbundles, which are a proprietary, undocumented format used by Unity and are impossible to inspect reliably from code, neither internally (by game code) nor externally (by other programs). Because of this, the most advanced third-party anti-crash mod just fired up a small Unity program and tried to load that assetbundle. If the program crashed for any reason, it would mark that assetbundle as a possible crasher and block it. While clever, this did not address the root problem.
VRChat could solve this by moving to a format of their own design which they could validate before loading, but as millions of assets already use assetbundles, they may decide not to pursue this route.
Another problem is that EAC is merely a band-aid over a deeper problem: Poor network security design.
Never Trust the Client
Clients, in programming parlance, are the programs that connect over a network connection to a central program called a server, which receives messages from clients, processes them, and then sends messages to other clients. A protocol is the collection of rules by which those messages are sent, and the format they use to convey data in those messages. In order to have a working multiplayer game, you generally need to have a client (which is what the user interacts with), and a server (which is running in a datacenter somewhere).
VRChat uses Photon for their game networking protocol, which is a proprietary system that offers cloud-based server hosting. For Photon, you can write client and server code, but the more complex the server code is, the more resources it uses and the more expensive it is to run a given number of server instances.
VRChat decided to save money by using the server as basically a dumb repeater. So, as long as data sent to the client follows a few simple rules, the server will happily repeat it to everyone else in the same instance. The client would then process the data and reflect the commands sent to it by the server in the rendered game world.
Throughout my college and university education, and then professional development career, I have had one simple directive continually hammered into my head, and you will find it anywhere there’s a tutorial on client/server programming or website development: Never Trust the Client. The reason? Clients can be modified or bypassed by any sufficiently motivated threat actor. Servers are under your direct control. (You don’t trust other servers, either, just in case.) This golden rule wasn’t followed by VRChat.
The result is that maliciously modified clients can tell the server something like:
Client 1: Hello, I am on a barstool!
Server: Hey, everyone, Client 1 is on a barstool!
Client 1: I am now on the moon!
Server: Hey, everyone, Client 1 is on the moon!
This is why teleporting, flight, no-clip, and ESP (the server sends the location of EVERYONE at ALL TIMES, even when not actually visible or blocked) are all possible with minimal effort. It’s even possible to have the server happily repeat packets that will crash clients due to how little checking is done on the server. For instance, before I left home for my trip this week, bots are still able to send malformed audio packets at a rate that crashes the client.
What should happen is:
Client 1: Hello, I am on a barstool!
Server: Okay, that makes sense. Hey, everyone, Client 1 is on a barstool!
Client 1: I am now on the moon!
Server: Uh, the moon is 384,000km from your position, and my simulation of you moving in that direction using vanilla game code shows you moving 1m. That is statistically significant. Hey, everyone, I disconnected Client 1 for acting suspiciously!
So What Can VRChat Do Instead?
VRChat will need to step back and re-evaluate a lot of their design choices in order to improve their security. Right now, many of their systems are still acting like lab prototypes that were just scaled up and forgotten about. VRChat needs to have a full security review of their code, infrastructure, and processes, and need to collaborate and brainstorm about what they could do better before jumping to conclusions and making a precipitous, heavy-handed decision like this one. While EAC can help in some areas, right now it is causing more harm than good by eliminating accessibility, disabling reputable protection mods, and throwing false flags on accounts for currently unknown reasons.
