Keeping the Pirates at Bay
So you’ve worked 10- to 12-hour days for the past two years, trying to make your latest game the best ever. You even added copy protection to try to stop the pirates, but within a few days of release there are already crack patches flying around the Internet. Now anyone can help themselves to your hard work, without so much as a “please” or “thank you.”
This is what happened to Insomniac’s 1999 Playstation release, Spyro 2: Ripto’s Rage. Even though it had good copy protection, it was cracked in a little over a week. So when we moved on to Spyro: Year of the Dragon (YOTD), we decided that something more had to be done to try to reduce piracy. The effort was largely successful. Though a cracked version of YOTD has become available, it took over two months for the working patch to appear, after numerous false starts on the part of the pirates (the patch for the European version took another month on top of that). The release of patches that didn’t work caused a great deal of confusion among casual pirates and plenty of wasted time and disks among the commercial ones.
Two months may not seem like a long time, but between 30 and 50 percent of most games’ total sales occur in that time. Approximately 50 percent of the total sales of Spyro 2, up to December 2000, were in the first two months. Even games released in the middle of the year rather than the holiday season, such as Eidetic’s Syphon Filter, make 30 percent of their total sales in the first two months. If YOTD follows the same trend, as it almost certainly will, those two to three months when pirated versions were unavailable must have reduced the overall level and impact of piracy. On top of this, since YOTD was released in Europe one month after the U.S., those two months protected early European sales from pirated copies of the U.S. version.
So why did it take so long to crack YOTD when a patch was available for Spyro 2 so quickly? The difference was that Spyro 2 only had copy protection, while YOTD added crack protection. The crack protection complemented the copy protection by checking for alterations to the game, rather than just making sure the game was run from an original disk. This extra layer of protection slowed down the crackers significantly, because removing the copy protection had to be done without triggering the crack protection. Basically, YOTD is booby-trapped — one wrong bit and it will blow up in your face. This article will explain the techniques that we used in YOTD, what we learned from using them, and some ideas about how to take our techniques even farther. However, I will not go into explicit detail, as most of the coding involved is relatively simple. Crack protection is more about out-thinking the crackers than out-coding them. A great advantage of any method of protection is novelty. Even a new implementation will give an advantage over simply reusing code, regardless of whether it was successful in previously delaying a crack.
Defining the Problem
From the very beginning we recognized that nothing is uncrackable. Many different software and hardware techniques have been used in an attempt to stop piracy; as far as I know every one of them has been bypassed or cracked. Our goal was to try to slow the pirates down for as long as possible.
First we looked at the copy protection: was there any way to reduce its vulnerability to cracking? We could call the copy protection multiple times throughout the game, making it harder to bypass. Unfortunately, the copy protection requires exclusive access to the CD for about 10 seconds, which is an eternity when you are waiting for a level to load. So that was out of the question.
Then we looked at how a typical crack is made. Most cracks for Playstation games replace the boot executable with an “intro” that proclaims the prowess of the crackers and allows cheats to be activated. This intro is concatenated with a copy of the patched executable and compressed so that the total file size is no larger than the original file. The new file bears little resemblance to the original boot program. This difference gives us the opportunity to reload the boot executable sometime after startup, causing severe corruption if it has been altered in this way. As with the previous option, this solution suffers from the problem of adding to the load time. It is also vulnerable to the pirates finding some other space on the disk to hide their crack, leaving the original boot file untouched. While this solution might have slowed the pirates down for a few more days, it didn’t seem like it was the answer we were looking for.
We decided that we needed a thorough way of detecting at run time that the game had been cracked. When we could reliably determine whether the game had been modified, we could stop the game anytime we found a discrepancy. We also needed a method that didn’t require access to the disk. It should just check the code in memory, unlike the standard copy protection or our option of reloading the boot program. This would allow us to place the check anywhere in the game, making it much harder to remove.
So now we had a definite goal, an approach that should significantly improve the protection for YOTD.
Checksumming
Finding out if a block of data has changed in any way is actually pretty easy. Techniques have been used for error detection for years and are well documented. Just search for “checksum” on the Internet. For YOTD, we decided to use a CRC checksum: it’s robust, simple, and fast.
The checksum was calculated bitwise rather than using tables, as tables would be an easy point for a cracker to attack. We took care to hide and protect the checksum values as well. If these could be found and altered easily, a cracker would simply replace the checksum with a new value that matched the cracked data, which is far easier than removing the code. For the same reason we didn’t use functions to calculate checksums, we inlined the code as much as possible. If the code was in a function, it would only have to be removed once. The inline code would have to be removed as often as it was used.
We used a few slightly different implementations to stop simple pattern searches from being used to find the checksum code. To what degree this survived compiler optimization we don’t know. To make our lives easier we made macros. These could be sprinkled around the code and mixed in with other tasks, which would make it much more difficult to spot where the checksum was being calculated.
Unfortunately, because checksums are designed to detect errors and not modification, they cannot offer full protection against modification. The checksum value for any block of data can be made to add up to any value by modifying the same number of bits that are used for the checksum. In other words, if the checksum is 16 bits, altering 16 bits in the data can make the checksum match any value.
To deal with this, multiple checksums were applied to the same data. Each checksum used a different start offset into the data, and stepped through the data by different amounts. This meant that overlapping and interleaved sections of data were checksummed at different points, making it almost impossible to alter anything and still have all the checksums add up. I could have used different checksum algorithms for the same effect, but in this case I didn’t have time to implement more than one method.
We used the fact that altering a small number of bits can give you any checksum value to our advantage. By inserting the correct value into the middle of the data, the checksum could be made to equal any predetermined value. This meant the checksum value could be hard-coded and therefore become part of the data being checksummed. This is bewildering to even think about, let alone try to crack.
Since the game used multiple code overlays (or DLLs), they cross-checked each other as much as possible. This further reduced the chance that any section could be altered without being spotted. If any overlay noticed a discrepancy, it altered data in the core such that no subsequent checksum would be valid. This meant that if an alteration was detected in one overlay, then other overlays loaded later would know about it. This made it difficult for the cracker to spot what actually triggered the protection, as I’ll explain later.
Obfuscation
Now that the meat of YOTD‘s crack protection had been implemented, it was time to move on to the second stage — slowing down the crackers as much as possible. We had already tried to make the protection as difficult as possible to understand, mixing in the checksum code with regular game code, and using different implementations so that it would be hard to understand. We thought that if the crackers couldn’t understand what we had done, it would be a lot harder for them to crack the game. We wanted to make it hard enough to reduce the pool of people capable of cracking it. If there are only a couple of pirates with enough skill to crack a game’s protection, it might take them a week or two to get around to it. Unfortunately, with YOTD being such a high-profile game, this was probably wishful thinking.
Thus we wanted to make the job of cracking YOTD time-consuming and tedious. If we could just keep the crackers busy at finding the protection, that’s time taken away from them working out how to remove it. Again, we were trying to reduce the pool of people available who could crack the game. Not every cracker would have enough time available to make the crack; it probably isn’t anyone’s day job. On this note, it’s worth pointing out that for most crackers this is a hobby. If they get bored, they may well give up. We tried to make the crackers have to wade through plenty of chaff before finding the protection. There were a couple of techniques we tried to achieve this.
First of all, if the crackers know what they are looking for, they often don’t even have to boot up the game to find the protection. They can simply search the disk and sometimes even edit the protection right there and then. Simply doing an XOR of as much of YOTD’s code as possible before burning it to the CD means that this technique will not work. It also makes it difficult to match up data in memory with its source on the disk. We worked under the assumption that code can’t be modified if it can’t be found.
Second, we made it as difficult as possible to debug. If you’ve ever had to debug code that behaves differently when you trace through it, you know how much of a pain debugging can be. We used trace traps to make the code behave differently if breakpoints had been placed. The checksum helped with this, as any software breakpoints used would alter the checksum. Rebooting the debugger and the game takes time, and the more often we could force the cracker to do this, the more of their time we were wasting.
Perversely, though, the harder a crack is to make, the more fun it is for the cracker to make it. It’s a challenge, and therefore fun. Paradox, the cracking group who produced the working patch for YOTD, even thanked the “Sony coders” who added such interesting protection to the game. The more difficult the crack, the more effort they will put into making it.
It’s the same with the length of time it takes to produce. The longer that the game has been out without a crack, the more prestige there is in being the first to produce one. Again, this means more effort will be put into producing it.
Taking Action
Of course, all of this effort is worth nothing if the game doesn’t do anything once a crack is detected, but this needs to be handled carefully. If the game just halts as soon as any modification is detected, the cracker would soon find and remove all the protection. However, if we wait too long to react, too much of the game would be playable even if an incomplete crack was used. To balance this, we used as many layers of protection as possible, which occurred at different points during the game. In YOTD we had four layers, including the copy protection.
The copy protection stopped the game very early. When this was removed, the game appeared to work for some time. We assumed that the crackers generally don’t play the games they crack very much, they just play until the point where the protection they know about kicks in. Then they release a crack, believing it to be complete.
To play on this, we designed the game to break in ways that weren’t immediately obvious. Most of the protection is “off-camera,” affecting levels other than the one currently being played. In YOTD the object of the game is to collect eggs and gems, which are then used to open later parts of the game. The protection removed eggs and gems, so that the player could not make progress. We tried to make the game unplayable for any length of time, while at the same time making it difficult to determine exactly where things had gone wrong. If errors accumulated slowly until the game broke, the cracker would not notice such behavior so easily.
Other, more obvious protection was done less frequently. Callbacks were corrupted, which made the game crash in odd ways. The European version changed languages randomly. Some of these actions break the game and others are just an annoyance to the player, but if the game is difficult or frustrating to play because of the failed crack, this can be more effective than breaking completely.
By making the game behave in as many odd ways as possible, we hoped to cause a lot of confusion. The pirates wouldn’t know if the crack didn’t work, whether they had just failed to apply the crack correctly, or if the disk had failed to burn correctly. The people who didn’t play a lot of the game wouldn’t notice that anything was wrong and claim that the crack worked. This happens more than you would think. A lot of people pirate more out of habit than anything else, booting up the game to have a look before moving on quickly. All of this would help to delay a complete crack from being made, because no one would be sure that it was required.
The Costs
Implementing all of this protection takes time and resources away from actually developing the game. For YOTD those costs were as follows:
Programmer time. One programmer was required for three to four weeks. The programmer spent this time adding the copy protection, integrating the anticrack protection into the game, and writing tools to mask the data and generate checksums. For about six months prior to actually writing any code, some time was spent thinking of methods for protecting the game and what to do when a crack was detected. This was slightly less than two percent of the total programmer time budgeted for the game.
Game data preparation. The game data needed additional preparation before a disk could be burned. The game’s WAD file had to be run through tools to generate checksums and mask data. This added about an hour to the burn cycle, making it about three hours long. The extra steps involved also made this process more prone to error, though this diminished over time as we became used to it and automated what we could.
Debugging. Any version of the game with protection included was very difficult to debug, as any software breakpoints would trigger the protection. Beyond a certain point, hardware breakpoints were turned off by the copy protection. This effectively meant that any debugging had to be done by the programmer who implemented the protection (me) on production versions of the game.
Testing. The protection was designed to produce effects almost indistinguishable from bugs, so testing was also affected. If any false positives occurred in the protection, they could be reported incorrectly. For this reason a very thorough debugging plan was produced just for the protection. Every location that could trigger protection was listed, along with how long it would take to trigger, what the exact effect would be, and where you had to look to see the effect. Testers had to visit the locations, wait the required amount of time, and then look to see if the protection had been triggered. Having any of the protection give a false positive was obviously our biggest worry. Therefore all the protection was set up on a compile-time switch so that it could be turned off at any time if we weren’t absolutely sure that the protection was reliable (and believe me, there were a few moments when it didn’t seem to be).
After the Crack
In the end, rather than trying to remove all the checksum code, the crackers simply found a way to bypass it. I’m not exactly sure how, but I know YOTD was vulnerable because the copy protection was only run once, at boot time. I assume the crack bypassed the copy protection and then restored the data to its original state. Any checksums performed after this point would not find any alterations (and any checksums before this were removed by the crackers).
While the protection on YOTD was reasonably effective, there were definitely things that we could have done better. If we had been able to check the data on the disk and run multiple copy protection checks, then it would have been a lot more difficult for the crackers. As I mentioned at the beginning of this article, there were practical reasons why these approaches could not be applied to YOTD. Maybe if the protection had been integrated into the game earlier, these difficulties could have been overcome.
Also, too much of the game could be played with a partial crack. This was a balancing act, though. If the protection had kicked in faster, perhaps the crackers would have realized sooner that they hadn’t been successful with the first crack. But in the end, we were perhaps a little too cautious. We could have reduced the amount of the game that could be played with an incomplete crack.
What We Learned
Were all our efforts worth it? Yes. While the effects of crack protection against piracy are extremely difficult to measure, we certainly caused a great deal of confusion. Until the crack came out, YOTD was the most talked about game on the copying forums. People wasted disks, blamed the cracking teams, and claimed that the cracks that didn’t work were O.K., just because they hadn’t seen anything go wrong. People were saying nasty things about Insomniac and Sony because they couldn’t “back up” the game. Some people even thought it was funny when the fairy character, who normally offers players helpful advice, instead told them they were playing a modified game. There is also an effect on future piracy to consider: at the very least we made a few people think twice about buying a cheap copy of a game.
We’ve gained valuable knowledge about what works and what doesn’t. Layering protection that doesn’t kick in immediately is definitely a very effective protection. If nobody thinks a crack is required, they won’t be working on one. Even when they do work on the crack, it takes them longer. The crackers apparently spent quite some time play-testing YOTD before they released the final crack, just to make sure they didn’t get burned twice.
Unfortunately, the crack protection is weaker once the copy protection has been run. The cracker only needs to remove the code that runs the copy protection. Once it has been run, the original code can be restored, and the checksum will be correct. If this is only in one place, it is easier to attack. To combat this, the copy protection needs to be run as often as practical from independent copies of the code.
If there is space, put multiple copies of the game data on the disk. The cracker will have to find out which one is used or alter them all. Either way, you’ve slowed them down. An extension to this would be to actually use multiple copies of the data, either loading a random selection or loading using a pattern based on when the data is being loaded. If some of the copies are masked differently and some are never used, the cracker will have to find and alter them all to ensure that the crack is complete.
Even better than masking the data is compressing it, which offers many advantages over simple masking. The relationship between compressed and uncompressed data is much less obvious, the file sizes are different, and any cracked data has to be compressed or else it won’t fit back on the disk. This means the cracker has to find out what compression was used, and if you customize the algorithm for your data, they may have to write a compression program just to be able to make the crack.
Looking back at the choices we made, we could have implemented multiple copy-protection checks throughout the course of the game. Unfortunately, this isn’t always possible or practical, depending on the method of protection used (especially if minimizing load times is a primary concern). An alternative is to check the source data on the disk. Of course you can’t check the entire disk, but all the executables can be checked, along with the table of contents and boot information. This is something YOTD failed to do and is probably how it was cracked.
Reality Check
We may not be able to stop the pirates, but we can have enough of an impact to make pirating a much less attractive option. Given the choice of buying a game or waiting two to three months for a pirated version, a lot of pirates are going to start buying games. Or at least they’ll buy their favorite ones.
There is also an advantage in numbers; the more games that add effective protection, the greater the benefit is for all games. Crackers have limited resources, and the longer that they’re tied up on each game, even if it’s only for a few weeks, the fewer cracks they can produce.
Games that implement just a standard copy protection scheme can be cracked in less than a day. Sometimes a tool is even available which does it in seconds. Any game that takes longer than this because of added protection will be put in line until the cracker has time to deal with it. The longer that line is, the longer it will take for any given game to be cracked. The trick is to keep your game from reaching the front of that line for as long as possible.
