SDMI challenge FAQ

Contents

  1. Introduction and background
  2. What we did
  3. How we did it
    • Details pending
  4. Why we did it: questions about our motivation
  5. The future

Answers

Introduction and background

Q. Who are you?

We are a group of researchers studying computer security and digital watermarking. The members of our group include:

Scott Craver, Patrick McGregor, Min Wu, Bede Liu, (Dept. of Electrical Engineering, Princeton University)
Adam Stubblefield, Ben Swartzlander, Dan S. Wallach (Dept. of Computer Science, Rice University)
Edward W. Felten (Dept. of Computer Science, Princeton University)

Q. Who/what is SDMI?

SDMI is the Secure Digital Music Initiative, a group whose goal is to "protect the playing, storing, and distributing of digital music" (from their website at www.sdmi.org.)

The recording industry is naturally concerned about the ease with which digital music can be copied, and thus pirated. SDMI is attempting to curb piracy using technological means.

Q. What are they building?

SDMI is developing specifications for a system, to be enforced by future music players/recorders, to hinder unauthorized copying by screening music. In a nutshell, music will be protected with various technologies (such as digital watermarking---see the next question.) Devices that play or record music will first screen it, and protected music clips will only be playable under certain conditions.

For instance, with this system, you could buy a CD at a record store that contains protected music. You would be able to play the CD in an SDMI-compliant CD player. However, if you take a song from the CD, compress it into an MP3, and make it available on the Internet, those who download the MP3 will have trouble playing it on an SDMI-compliant device.

Q. What is watermarking? What is a digital watermark?

A digital watermark is an imperceptible signal hidden in an audio clip, or an image, or any other object of value. The hidden signal is intended to communicate information about the marked object.

There are many applications of watermarking. In the context of SDMI, the application is screening and piracy prevention: an audio clip with a watermark is recognized as copyrighted, warning a portable device that it should not be recorded (or possibly even played) except under specific conditions.

Watermarking is apparently one of the central technologies behind SDMI's music protection system.

Q. What is the SDMI challenge?

On September 6, 2000, SDMI issued "An Open Letter to the Digital Community," inviting people to attempt to crack certain technologies they are considering for use in their system. They set up a web site where music samples and some other information could be downloaded to aid in analyzing the technologies.

In the first round of the challenge, SDMI provided four "watermark" challenges and two "non-watermark" challenges, as described below. For each watermark challenge, three audio streams were presented: a reference stream in its original form, the same stream with a watermark, and a challenge stream, watermarked, with no corresponding reference stream. The challenge was to submit to the SDMI "oracle" (a Web site), a version of the challenge stream with the watermark removed but without degrading the perceived sound quality of the original stream. The oracle would respond by email, after several hours, with an "ACCEPT" message (if the watermark was removed without degrading the sound quality too much) or a "REJECT" message.

In the second round of the challenge, SDMI offered additional "challenge" tracks to participants who succeeded in defeating the original challenges. No oracle was offered. The SDMI requested that participants send the results of their watermark removal tools along with technical details of how the watermarks were removed. Following this, the SDMI would then offer participants the chance to sign a non-disclosure agreement in return for receiving a fraction of the prize money.


What we did

What did you do? What were the results?

We analyzed the clips watermarked with the four technologies, and successfully modified them so that the watermarks could no longer be detected, while maintaining a level of audio quality satisfactory to SDMI.

Q. How do you know if you succeeded?

SDMI provided "oracles" on their web site. People could download watermarked music, attempt to damage the watermarks, and upload the music to an oracle that would inform them of the results. The oracle would email the submitter if the attack appeared to have rendered the mark undetectable, without significantly damaging the audio quality in the process. SDMI's oracles told us that our attacks have succeeded on all four watermarking technologies.

Q. How do you know that you did not ruin the audio quality of the music when you modified it to defeat the watermark?

We know this for two reasons. First, and most important, we listened to the music ourselves, to satisfy ourselves that the audio quality was satisfactory.

Second, contrary to the statements of the SDMI, their oracles did check the audio quality of the submitted music. We know this because they rejected some of our early submissions due to their poor sound quality, but they accepted our later submissions.

Q. How high does the audio quality of the unwatermarked music have to be in order to defeat a watermarking technology?

Remember that the goal of a watermarking system is to prevent piracy. If there is a way to remove the watermark (or render it undetectable) while retaining high enough audio quality that a pirate would be able to distribute the music, then the watermarking technology will not be effective. In short, if the modified music still sounds good to an ordinary person, its quality is high enough.

There is no need for the kind of elaborate testing involving "golden-ear" experts that SDMI now wants to establish as the standard of success. It doesn't matter what a few exceptional individuals can hear in a piece of music, what matters is whether it sounds okay to a regular person.

Q. Is it true that you admitted that some of your successful challenges had poor sound quality?

That is false. We did not say this, because is it not true. We don't know why the RIAA is telling people we said this.

Q. Are you sure you successfully defeated their system?

Due to the constraints imposed on us by the SDMI challenge, we have no ironclad proof, only evidence. Our evidence consists of output from oracles based on the watermarked samples they provided. One never knows, for instance, if an attack only works on the specific watermarked samples available on the SDMI web site, or if the oracles were faulty. However, such possibilities seem very unlikely, especially since we obtained a number successful results on all watermark technologies, using various techniques.

Q. How did they judge quality? How did you judge quality?

We don't know how SDMI judged the quality of a modified audio clip. As for our standard of audio quality, we have reason to believe that some modifications we performed were no more damaging than the watermarking methods themselves. If consumers consider those modifications too damaging to music, then they might feel the same way about the watermarks.

Q. Did anybody else defeat any or all of the SDMI challenges?

We don't know. We have not heard of anybody else defeating the challenges, but we would not be surprised at all to hear that others had done so. As we get confirmed reports of cracks by other teams, we will place any links to those teams here.

Q. You talk about four watermarking challenges issued by the SDMI. Weren't there really six challenges?

Yes, there were six challenges. Four (challenges A, B, C, and F) involved watermarking and two (challenges D and E) involved other parts of the SDMI system. As we will explain at length in our upcoming technical report, the success of SDMI's music protection technology depends critically on watermarking, so defeating the four watermarking technologies is enough to show that the overall SDMI system as currently designed would not work.

Q. What were the two non-watermarking challenges?

We will discuss these in detail in our upcoming technical report. Briefly, challenge D concerned a method for preventing people from ripping tracks from a CD, unless that CD has a signature on it marking the CD as original. Thus, as far as we can tell, this is intended to allow you to make mix CDs and MP3s from your original CDs, but not permit you to make mixes and MP3s from other mix CDs or pirated CDs.

We think that we understand how this technology works and how it could be defeated in practice, but we were unable to test our understanding because the SDMI oracle for challenge D did not behave as documented. (The oracle always returned "INVALID", indicating that the submission was improperly formatted, even when the submission was formatted as specified in the documentation. Our attempts to contact SDMI about the misbehaving oracle were fruitless.) Without a working oracle, we were unable to confirm our analysis of challenge D.

Challenge E concerned a similar method, which appeared to prevent people from performing a clever attack by splicing together snippets of songs. The documentation and data that SDMI provided for challenge E was so vague and incomplete that there was no way to even begin the process of analyzing it.

Q. Are your modifications too complicated, difficult, or time-consuming to be within the capabilities of the average pirate?

A pirate does not need to be clever to perform clever hacks. If one malicious individual was capable of writing a computer program that thwarted SDMI, then any other malicious individuals would need only to be willing and able to download the program and click an "OK" button.

People commonly underestimate the capabilities of pirates, in part for this reason. A better question might be: are these modifications within the capabilities of a talented pirate, who could then write and distribute a piracy tool? We believe so.


How we did it

The specific details of our analysis will be addressed in a comprehensive technical paper separate from this FAQ. The paper has been accepted for publication in the Proceedings of the Fourth International Information Hiding Workshop. The workshop will be held in Pittsburgh, April 25-27, 2001. When the paper is released, it will be available at http://www.cs.princeton.edu/sip/sdmi.


Why we did it

Q. Wasn't there a boycott of the SDMI challenge? Why didn't you participate in the boycott?

Several groups announced a boycott, and we do not know how many people participated in it. We decided not to boycott the challenge, because we felt we could help both the public and the SDMI by participating in the challenge on our own terms. We believe that public discussion of the drawbacks of SDMI's technologies will be beneficial in the long run.

We also believe some of the arguments for the boycott are due to misconceptions about the challenge and the technology. These particular arguments are addressed in the next few questions.

Please keep in mind that our team consists of people who study computer security, watermarking systems and attacks on watermarking systems; our decision to not boycott was made in this context, based on our expertise and familiarity with these kinds of systems.

Q. By participating in the challenge, weren't you helping the record companies impose restrictive technology on music lovers?

and...

Q. By participating in the challenge, weren't you helping pirates steal copyrighted music, impoverishing musicians and songwriters?

We believe our success against all four watermarking technologies, and our sharing of those results with other researchers, will not help anyone impose or steal anything.

On the one hand, this information cannot be used to make restrictive technology. If anything, it suggests that all of the proposed technology is incapable of being restrictive.

On the other hand, this information cannot be used by pirates if the technologies are never deployed. This is why it is best to perform analysis on a security system before it is released.

Q. Still, wouldn't it have been better for SDMI had you not analyzed their system?

SDMI invited the public to analyze their technologies (to "crack them" said their invitation,) setting up a web site and hiring people to assist. Also, any weaknesses in SDMI's technology would have existed even if we hadn't looked for them---analysts do not create flaws, but merely detect them---and if the SDMI system had been deployed as is, pirates would have found and exploited those weaknesses, regardless of our actions.

The study of information security is based on two equally important components: the design of security systems, and the analysis of (attempts to break) those security systems. One occasionally encounters the misconception that analysis is destructive and evil, and that people performing analysis are attackers who wish to exploit those systems. Rather, analysis is a critical component of the development process. Without it, one would never know if systems were well-designed, and one would never learn how to design better systems.

Q. Still, wouldn't it have been better for opponents of SDMI if you let SDMI go ahead and deploy a flawed technology, so music lovers could teach them a lesson by copying music despite the technology?

Of course not. This is scientific research: it is not our goal to engage in tactics such as tricking the industry into choosing a flawed system. Our goal is simply to analyze security systems and share our results openly with the scientific community.

Again, researchers who crack cryptosystems and security systems are not motivated by a desire to exploit these flaws later. They are merely subjecting systems to analysis, motivated instead by a desire to increase the existing body of knowledge about security systems.

Secondly, if the technology is cracked in deployment, rather than on the drawing board, everyone loses to some extent. The recording industry obviously, device manufacturers most certainly, but even opponents of SDMI. Even pirates! To an opponent of SDMI, even a broken, circumventable SDMI system is worse than no SDMI system at all.

Finally, see below. The DMCA may have prohibited analysis outside the challenge deadline.

Q. Why did you choose not to participate in the second round of the SDMI challenge?

As academic researchers, we felt the second round of the challenge was unscientific and offered us no further information. Our goal is to understand, document, and study the technologies being used by SDMI. Since the second round provided no oracle access and no further unwatermarked content, there was nothing we could learn from it.

In addition, we feel that the second round as designed by SDMI is not a valid test of whether a first-round success is repeatable, since it gives the participant much less information than was available in the first round.

Q. Did you share your results with SDMI?

As scientists, we are sharing our results with everyone, including SDMI.

Q. What about the cash prize offered by SDMI?

SDMI did offer a small cash prize to be split among everybody who defeated at least one of the six technologies. However, to be eligible for the prize, researchers had to sign a confidentiality agreement that prohibited any discussion of their findings with the public. The terms of the challenge also allowed researchers to publish their findings if they decided to forgo the cash prize. We decided from the beginning that we were more interested in publishing our results than accepting any share of the cash prize.

Q. Didn't the Digital Millennium Copyright Act (DMCA) criminalize the study of these kinds of technologies in the United States?

Fortunately, the DMCA did not apply to this challenge, since SDMI granted explicit permission to study their technologies. We are not sure whether it would have been legal to study these technologies outside the context of this challenge. We think the DMCA, by criminalizing some kinds of study of important technologies, represents an "ignorance is bliss" approach to technological copyright enforcement, which will not work in the long run. We lobbied against certain aspects of the DMCA while it was before Congress, and we still consider it to be a seriously flawed law.

Above, we mentioned the important role of analysis in the design of security systems. The main problem with the DMCA is that it hinders this analysis, restricting it in order to provide an extra layer of legal protection for existing copyright systems. But this causes the scientific process to stagnate. Imagine a federal law making it illegal for anyone (including Consumer Reports) to purposefully cause an automobile collision. While this may be a well-intentioned attempt to stop road-rage, it also bans automobile crash-testing, ultimately leading to unsafe vehicles and the inability to learn how to make vehicles safe in general. The situation with the DMCA is analogous.


The future

Q. I heard people complain that the challenge period was too short and the information on the site too meager for the challenge to be taken seriously. Were they right?

For cryptographic challenges, it is expected for researchers to be given a long time, often indefinitely, to crack a cipher. It is also expected for the cipher algorithms to be provided (the security of a cryptosystem must not rest on the obscurity of the algorithm). SDMI only provided about 3 weeks, and did not provide any details on how the watermarking technologies worked. They did not even provide programs to detect or embed marks, handling detection themselves via oracles.

The SDMI challenge seemed to be designed as much to hide the design of the watermarking schemes as to test whether those schemes could be broken in practice. In practice, once SDMI-enabled players were deployed, the algorithms they used would eventually be reverse engineered and analyzed. Even before the algorithms were reverse engineered, any consumer with an SDMI-enabled player would have more information than SDMI provided in the challenge. For example, a consumer could use his player as an oracle; such an oracle would be faster than SDMI's Web oracle, and it would provide more information. Thus the SDMI challenge was unrealistically difficult.

Fortunately, analyzing watermarking technologies is easier than analyzing ciphers, because the watermarking problem is much more difficult than the problem of encryption. In cryptography, a successful attack often requires deciphering an enciphered message. In steganography (information hiding) merely destroying the hidden message (e.g. the watermark), usually by slightly distorting the medium containing it, is a successful attack, even if one cannot decipher or detect any hidden message contained in the medium.

We do believe, however, that in any future challenges SDMI should provide more information than they did this time around. Researchers were provided with less information than ordinary people would obtain by using SDMI devices! For instance, the oracles, when reporting that an attack did not succeed, would not tell us if the failure was due to the watermark surviving, or to quality being degraded beyond SDMI's quality standards.

Q. Does this mean watermarking, as a technology, is infeasible?

No! Watermarking has a lot of different applications, and a lot of potential. Any successful hack of SDMI's watermarking technologies is due to the particular application of watermarking they had in mind, and the way they intended to integrate it into a security system.

Q. What if SDMI uses your results, and those of others, to develop a more secure or unbeatable system?

We believe their general security model is inherently vulnerable to a number of attacks no matter how sophisticated their watermarking technologies become. We can never say for certain, but we are confident that we can continue to develop attacks like we have if SDMI updates their technologies.

This is essentially the situation of the "trusted" client in a hostile environment, a common problem in piracy prevention. Basically, an anti-piracy measure is enforced by a device or computer program belonging to an adversary who wishes to circumvent it, and who can take apart and analyze it. Such measures are usually quickly circumvented, and many attacks exist that involve the exploitation of the device itself.

The watermarking technologies are similarly designed. They are what we call public watermarking technologies, in which no secret key is needed to detect the mark; all devices know where to look for it. Because the secret embedding method is implemented in so many devices, the odds of an adversary learning how to perform and reverse it are very high. Also, these watermarks must be robust to all modifications a listener considers slight, and the number of possible slight modifications to an audio clip is constrained only by one's creativity.

Q. What if SDMI completely overhauls their system so that these inherent problems no longer exist?

The underlying problem that SDMI is trying to solve, that of protecting content from a hostile platform while allowing the platform to "play" the content, is inherent very difficult, both in theory and in practice. To overhaul their system, SDMI may well have to overhaul their business model.

We would be deeply impressed if SDMI or anyone else developed a secure system for piracy prevention given the requirements of music listeners. We would be happy to examine any system they have, assuming they offer a fair challenge.

Q. What if SDMI has more watermarking schemes than what they put on their website, and just uses one of these unbroken technologies for their system?

Then they will be using a system that has not been subjected to any open scrutiny, a sure recipe for disaster. We encourage SDMI to let the scientific community review their systems before committing them to actual devices.

Q. Will you participate in any future challenges?

Sure, as long as they are fair. In this challenge a bare minimum of information was given to researchers, and we hope any future challenges will be more open.

Q. Where can I get more information?

If the information you need is not in this FAQ, then try our Web site at http://www.cs.princeton.edu/sip/sdmi. If you still can't find the information you need, then contact Edward Felten at felten@cs.princeton.edu or (609) 258-5906.


Copyright (C) 2000, Princeton University. All rights reserved.