Do Search Engines Influence Media Piracy? Evidence from a Randomized Field Study

There is an ongoing public policy debate regarding the role search engines can play in the fight against intellectual property theft. However, for any sensible policy discussion it must first be the case that search engine results influence consumer piracy choices, and there is very little empirical evidence one way or the other on this question. The goal of this research is to use a randomized field study to analyze whether search results can influence consumers choices for infringing versus legal consumption channels. To do this, we design a customized search engine that allows us to experimentally manipulate pirated and legal links in users’ search results. We then conduct separate experiments on a general population of users and on college-aged users where we randomly assign users to a control condition or to separate treatment conditions where infringing sites or legal sites are artificially promoted in the search results. Our data show that relative to the non-manipulated (control) condition, the presence of pirate or legal links in search results strongly influences the behavior of both the general and college-aged populations: users are more likely to choose a legal option to acquire the movie when legal sites are promoted, and users are more likely to choose a pirate option when piracy links are promoted. By analyzing users’ initial search terms, we also find evidence that users who initially intend to pirate are more likely to purchase legally when legal links are promoted and that users who initially intend to consume legally are more likely to pirate when pirate links are promoted. Together our results suggest that reducing the prominence of pirated links can be a viable policy option in the fight against intellectual property theft.


Introduction
While the Internet has created many new opportunities for media companies to reach consumers through digital channels, it has also given rise to widespread media piracy. The potential impact of piracy on sales and overall social welfare has given rise to a variety of antipiracy efforts initiated by governments and industry participants. However, these efforts are complicated by the fact that piracy is a multifaceted phenomenon with multiple stakeholders.
Specifically, Internet intermediaries such as Internet Service Providers (ISPs), web-hosting services, and search engines all play a role in delivering information and content to the end user.
While end users are ultimately responsible for their content consumption, many in the creative industries argue that Internet intermediaries should play a more active role in reducing piracy.
One active policy question is whether search engines can, and should, play a role in the fight against piracy by reducing the prominence of pirated links in search results displayed to users.
Not surprisingly, this issue has strong views on both sides. Search engines argue that they are already doing a great deal to stop piracy. For example, in September 2013, Google released a report titled "How Google Fights Piracy" 1 which detailed Google's efforts to reduce piracy by removing infringing links from their search results based on notices sent by copyright owners. This report also notes that in August 2012 Google started taking valid copyright removal notices into account when ranking sites in search results. 2 In fact, Google's "Transparency Report" identifies more than 25 million copyright removal notices that they responded to in June 2012, affecting nearly 40,000 unique Internet domains. 3 At the same time, many content owners argue that search engines' efforts have been ineffective and that more is needed. For example, a recent study funded by the MPAA 4 analyzed the role that search results play in the discovery of television and movie content, arguing that current efforts by search engines to reduce the availability of pirated content are insufficient and that most of the search queries that result in viewing infringing links do not contain keywords that indicate specific intent to view this content illegally.
The key question though is whether pirate and legal results are sufficiently close substitutes in the eyes of consumers that making one harder to find in search results will increase consumption of the other. In an editorial commenting on the RIAA's desire for Google to make piracy harder to find, the technology blog Techdirt summarizes the view of many in the technology industries on this question: "The RIAA might not like it, but the simple fact is that when people are searching for [ In this view if a user is looking for infringing content, they have already made up their mind about whether to consume legally, and minor changes in search results will not change that behavior.
Thus, while it is well known that when users are choosing between close subsitutes changing the order of search results can change what sites users select, in our setting pirate and legal links may be sufficiently differentiated that users looking for pirate content will not substitute toward legal links and vice versa. In the words of the technology blog, ExtremeTech: "In general, if someone wants to pirate something, they're going to pirate it, even if they have to click through a couple of pages of search results." 6 Thus, while many copyright holders have called for Google and other major search engines to reduce the prominence of pirate links in search results, a fundamental question remains: Are users' media consumption decisions for infringing versus legitimate sources influenced by search results? In particular, are users more or less likely to consume content from an infringing source if search results increase or decrease their exposure to infringing content? If anything, these 5  However, despite the importance of these questions for both managers and policymakers, we are aware of no studies that empirically analyze the role of search results on the consumption of legal versus infringing content. One reason for the few studies in this area may be that assigning causation is tricky in this context. This is because, by design, the top results are also likely to be the most "relevant" results to the user. If a user is searching for infringing content and sees infringing results listed first, it is impossible to disentangle whether she clicks on that content because of her interest or because of its placement. In short, a user's desire to look for particular content shapes their search terms and subsequent behavior, making it impossible to use observational or archival data alone to show whether search results influence user choices.
In this paper, we address this methodological challenge through randomized field experiments, using the experimental design to avoid contamination due to user intent. We then implement this design in two experiments. The first experiment uses participants drawn from a representative panel of the U.S. population, and the second uses a panel of college-aged participants. In both cases, the participants are recruited through an independent company maintaining large survey panels. We expose these users to a potential task of finding a movie through online channels, and encourage users to use our custom-built search engine in place of the search engine they would normally use (e.g., Bing, Google).
In the first experiment (representative panel), our search engine displays results to users in one of three randomly assigned conditions: in the control condition users see the same results that would be displayed from a major search engine, in the first treatment condition infringing links are artificially promoted in the search results, and in the second treatment condition legal links are artificially promoted in the search results. The second experiment (younger users) adds two additional treatment conditions to test milder legal and milder infringing search manipulations.
We avoid the user intent fallacy by randomly assigning users to one of these treatment conditions.
We also record and examine their search choices through our interface, providing additional detail on their behavior within the experiment. Finally, we ask them to complete a questionnaire at the end of the experiment to measure their attitudes regarding piracy. Our results suggest that the prominence of search results can play an important role in users' subsequent choices about whether to pirate content or consume through legal channels. In the control condition about 80% of users choose to buy the product legally versus 94% who choose to purchase legally in the legal treatment condition and 56% in the infringing treatment condition. 8 We see similar results in a second experiment with a college-aged population (18-24 year old users): 62% of users in the control condition choose to purchase the product legally, versus 92% in the legal treatment condition and 39% in the infringing treatment condition. Our second experiment also shows that stronger treatments (in terms of number of pirated or legal links) lead to stronger outcomes: In the "mild" legal treatment the number of purchases drops to 76% (versus 92% in the regular legal treatment), and in the "mild" infringing treatment the number of purchases rises to 48% (from 39%).
Our data also allow us to infer user intentions from their initial search terms. When we classify user intentions based on the content of their initial search terms we find that users who initially express an intention to consume legally are less likely to do so in the infringing treatment condition than in other conditions, and conversely that users who initially express an intention to consume pirated content are less likely to do so in the legal treatment condition than in the other conditions. Together, our results suggest that user behavor is elastic to search results when it comes to media consumption decisions, and that reducing the prominence of pirated links in search results can be a viable strategy for fighting intellectual property theft for both a general population of users and for younger (college-aged) users and for both "undecided" users and users with a pre-existing preference for legal/pirated content.

Related Literature
Our research primarily relates to studies in the economics and information systems literatures analyzing the impact of piracy on sales and analyzing the effectiveness of anti-piracy measures.
Within these literatures, the vast majority of papers find that piracy harms sales. For example, Rob and Waldfogel (2006) show that overall piracy reduced per capita expenditures on music by about 20% and Zentner (2006) shows that piracy reduced music sales by about 7.8% in 2002 (for reviews of the literature, see Liebowitz (2008), Oberholzer-Gee and Strumpf (2010), and ).
With piracy's impact on sales well established in the literature, several recent papers have analyzed the effectiveness of regulatory and industry-led efforts to reverse the impact of piracy.
These papers generally find that making content available in legal digital channels reduces the incidence of piracy (see, for example Danaher et al. (2010) and ) and that anti-piracy policies targeting the demand-side of piracy (e.g., the HADOPI anti-piracy law in France as studied by ) and the supply-side of piracy (e.g., the shutdown of Megaupload studied by . Beyond these legislative interventions, Reimers (2014) shows that industry-led notice and takedown strategies for eBooks can be effective at increasing legal sales, and Bhattacharjee et al. (2006) find that RIAA lawsuits lowered levels of file sharing, with a much greater impact on high-level sharers.
Another important question is whether piracy impacts social welfare. Here the results are less conclusive. On one hand, Abhishek, Telang, and Zhang (2013)) use analytical models to show that firms' incentives to invest decrease in the presence of piracy, especially when users do not readily migrate back to legal channels. Further, using the Indian motion Picture Industry, Telang and Waldfogel (2014) find that media piracy had a strong negative impact on movie production. However, Waldfogel (2012) finds that the overall impact of technological change (including piracy) from 1980 to 2010 did not reduce the quantity or the overall quality of music released.
Our paper extends these results by studying anti-piracy efforts in the context of search results, and specifically whether reducing the prominence of pirated links in search results can be effective at changing user behavior and user satisfaction with search results.
In addition to the piracy literature, our research relates to the academic literature on consumer behavior at search engines and overall consumer search costs online. Previous studies of consumer behavior at search engines examined how the prominence of search results influence user behavior, showing that the position of search results can have a significant impact on clickthrough rates and conversion rates for both sponsored search results (e.g., Yang and Ghose 2010, Agarwal et al. 2011) and organic search results (e.g., Baye et al. 2012).
Specifically in the context of organic search results, Brooks (2004) showed that searchers are more likely to click links that are placed higher in the results listing. This is because the top results are perceived to be more relevant and because the search costs of exploring results with lower rank is relatively high. Similarly, an eye tracking experiment performed by Pan et al. (2007) revealed that college students have substantial trust in Google's ability to rank results by their true relevance, such that users' decisions are strongly biased towards links higher in position even if the abstracts themselves are less relevant. Further, using click-through observational data, Chesnes, Dai, and Jin (2014) study how the ban of non-NABP-certified pharmacies from Google's sponsored search listings affects consumer search on the Internet, finding that non-NABP-certified pharmacies receive fewer clicks after the ban than they did before the ban.
An important difference between these studies and our study is that our setting analyzes how search costs affect users' substitution pattern between legal content with infringing content. Thus our study relates to an active policy question, while also addressing a setting where the tradeoffs between different (legal and pirate) links could be quite high. Unlike most other studies which focus on organic search results or sponsored links, the perceived difference between legal content and infringing content could be large, particularly for consumers who may have a stated preference for one type of content or the other.

Experiment 1 -General Population, 2 Treatment Conditions Experimental Design
In this experiment we utilize a custom-built search engine to test the impact of search results on media consumption choices among the general population of Internet users. The participants for the experiment were recruited from the general population by an independent company maintaining a large survey panel. The study was performed online and consisted of three parts: (1) a screening phase in which the participants were asked to choose a movie they wish to acquire; (2) a search phase in which the participants were asked to use our custom search engine to search for a source to acquire the movie; and (3) a post-experiment questionnaire. During the screening phase the participants were asked to choose a movie they wish to watch from a list of 50 alternatives (see Figure 1). We asked users to select a movie from a specific list of movies in advance to make sure that our manipulation conditions could be implemented. We asked users to select a specific movie they were interested in (as opposed to assigning them a movie) to ensure that the users were motivated to find a movie they wished to view. Consistent with our goal of observing user behavior in search for movies, we excluded from the experiment any participants who stated that they are not interested in watching any of the movies in our list.
Participants who chose a movie were given a $20 prepaid virtual Visa card as compensation for their time in completing the experiment. The payment details were given to the participants before they were presented with the task in order for them to think of the money as if it was their own. The money could be spent however the participant chose and could be retained after the experiment (i.e., the money was not limited for use in purchasing the movie they selected, and could be used for purchases in any store that accepts Visa cards).
The participants were then given a task to search for a source from which they would like to acquire (download, stream, purchase, or rent) the movie. The participants were not told that we were examining their piracy behavior. Instead, the experimental instructions stated that the user was participating in an experiment to test the effectiveness of our search engine. We highlighted that our search engine was designed to efficiently search for movie related content (see Figure 2). Therefore, they were instructed to use our search engine in place of Google, Bing or other search engines when they searched for content. The instructions also stated that if the participant already had a specific website in mind to obtain the movie, they could go directly to that site after initially trying to search for it using our search engine. The participants were able to keep the movie they acquired, and keep any money left over after the task was completed.
We also stated that their identity is unknown to us and their behavior cannot be traced back to them. Before the experiment started, each participant was randomly assigned to one of three search engine conditions: no manipulation, non-infringing (legal) content manipulation, or infringing (piracy) content manipulation.  After the participants completed the experiment task, they were required to complete a postexperiment survey. In the questionnaire, they were asked about the source from which they acquired the movie, its price ($0 if the movie was acquired from an infringing source), and they were asked several demographic questions and questions about their media consumption preferences (see Figure 4 for sample post-experiment questions). It is important to note that we can only directly observe user behavior while they are on our search engine: we do not observe their behavior outside of the search engine. Thus, our observation of where the user obtained their content is based on their survey answers. While this may introduce some recall bias, the degree of recall bias should not vary across the control and treatment conditions and the survey occurs immediately after the participant obtains the content, significantly reducing the overall possibility of recall bias. It is also possible, based on prior studies that found that survey respondents are likely to under-report socially undesirable activities, that participants who choose a pirated option could be less likely to reveal that in a survey than other users are (Cannell et al. 1965;Warner 1978;Wyner 1980;and Means et al. 1992). However, again the propensity to misreport should not vary across the control and treatment conditions and any bias would lead to an underestimate of the degree that infringing links induce more piracy. We also note that we can use observed search and clicking behavior to validate a user's survey answers (including in some cases verifying price). In the results section we show that the users' survey responses were very consistent with their observed behavior.
In order to make sure that participants understood the task correctly, we included in our analysis only those users who claimed the $20 virtual Visa card, used our custom-built search engine, and reported the source from which they acquired the movie and its price.
Approximately 1,000 participants were invited to participate in the study. Of these participants, 770 met the initial qualifications for participating in the study by expressing an interesting in watching one of the movies on our list. These participants were invited to participate in our study and 632 of them logged into the system and were presented with the task details. Out of these participants, 235 completed the task as instructed, and 196 (66 men, 130 women) qualified for our study by also completing the post-experiment questionnaire. The dropout rates across each of the different stages described above are not statistically different across experimental conditions.
Prior to analyzing the results of the experiment we confirmed that the distribution of user To further verify similarity in pre-existing attitudes toward piracy, we tested whether the participants' initial intent for pirate or legal content is similar across the three experimental conditions. We did this by classifying their initial search terms reflected neutral, legal, or infringing intent. We did this classification by analyzing the degree to which pirate or legal links were present in the (unmodified) search results for queries commonly issued by our users. We discovered that search terms using only the movie's name contained almost exclusively "neutral" results (i.e., results that neither promote legal or pirate sources), and thus we classify these searches as "neutral." However, when search terms included the words "buy," "rent," or "purchase" the search results contained 38% more legal links than pirate links, and when the search term contained a legal domain name (e.g., amazon), the search results contained 78% more legal links than pirate links. Thus, we classify these search terms as representing "legal" intent.
Conversely, when search terms included the words "download," "stream," or "full movie" there were 33% more pirate links in the search results than legal links, and including the domain name of an infringing site (e.g., piratebay) resulted in search results that included 65% more pirate links than legal links. Because of this, we classify these search terms as representing "infringing" intent.
We then classify intent based on the initial intent reflected in each user's search terms (or "neutral" if the user did not express intent in their searches). As above, reassuringly, the distributions are not statistically different across the control and treatment conditions (see Table 2 for frequencies across conditions).  Finally, we note that while the characteristics of users who participated in the experiment were similar across the control and treatment conditions, it is possible that our participant pool skews toward more media or tech savviness than the general population. As such, in the results below one should focus on the relative levels of sales/piracy between the control and treatment conditions as opposed to the absolute levels within any particular condition. Table 3 compares the proportion of legal purchases (and the average price of a legal purchase)

Results
across the different treatment conditions. This table shows that in the control condition, where the search results are not manipulated, 80% of our participants chose to acquire the movie through a legal channel (and the remaining 20% consumed through a pirated channel). Relative to this baseline, participants in the legal content treatment condition were significantly more likely to acquire the movie from a legal channel (P = 94.37%, SD = 23.22%) than were participants in the baseline condition (P = 80%, SD = 40.34%). A t-test (t(121) = 2.85, p < .01) confirms that these differences are statistically significant.
Conversely, the results show that participants in the infringing content treatment condition were significantly less likely to acquire content from a legal channel (P = 56.92%, SD = 49.90%) relative to the baseline condition (P = 80%, SD = 40.34%). Again, a t-test confirms that the differences are statistically significant (t(91) = 2.43, p < .01).  These results, and the statistically significant differences across groups, strongly suggest that reducing the prominence of pirated content in search results can have a significant impact on user behavior, and thus is a viable anti-piracy strategy. This result may be initially surprising given that our treatment doesn't remove all pirated or legal links, just the ones appearing on the first page and on the first three positions on subsequent pages. However, the fact that our users are willing to substitute between legal and infringing options with even minor changes in the rank of these options suggests that users in our sample are relatively willing to substitute between legal and infringing consumption channels.
To further explore user's willingness to substitute between channels, we analyze whether changing the prominence of pirated links can affect the behavior of users who state an initial preference for legal or pirated content. This is important because these users might be considered among the most "committed" to consume through legal or pirated channels, and thus among the least likely to change their behavior.
We do this by following the methodology described above (see Table 2 and surrounding text) to classify each user's "intent" based on whether their initial searches express a preference for legal or illegal content. In Table 4, we compare how the purchase likelihood for users with pirated or legal intent varies across the different treatment groups.
Each cell in the table reports the number of users observed in a particular experimental and "initial intent" condition, the number of those users who made a legal purchase, and the proportion of legal purchases. For example, Table 4 shows that 33% (4 of the 12) "control group" users who initially expressed intent to consume through infringing channels, ultimately purchased through a legal channel.

First search term Control group Legal content manipulation Infringing content manipulation
Legal intent 31/31 (100%)

Table 4: Legal purchase rates across treatment conditions and initial intent
This table shows that users who initially express intent to consume legally are significantly less likely to do so when placed in the infringing content manipulation (73%) than in either the legal (96%) or control (100%) conditions (t-tests: t(48) = 2.55, p < 0.01; t(32) = 3.46, p < 0.01, respectively). Likewise, users who initially express intent to consume infringing (pirated) content are significantly more likely to purchase legally in the legal content manipulation (88%) than in the infringing (45%) or control (33%) conditions (t-tests: t(15) = 2.42, p < 0.05; t(18) = -3.36, p < 0.01, respectively). These results suggest that even users with an initial preference for pirate or legal channels have a relatively high substitutability between channels based on the ranking of search results.
While the tests reported above are sufficient to determine differences between the control and treatment groups based on our experimental manipulations, we can also use a logistic regression model to control for and analyze differences between groups based on observed characteristics. Specifically, we use the following logistic regression model to control for observable participant characteristics: (3) term that the user entered indicate that his/her intent is to acquire a legal copy of the). We present the results of this model in Table 5 below.
These results are consistent with our means comparison results above in that the treatment variables are statistically different from the control condition and that they have the expected sign (the legal treatment condition increases the likelihood of purchasing legally and the infringing treatment condition decreases the likelihood of purchasing legally). These results also confirm, as one would expect, that participants who consumed infringing content in the past are less likely to purchase the movie legally and that those who use a search term that implies legal intent are more likely to purchase legally.
In the next section, we report the results of a similar experiment that uses college-aged participants and varies the intensity of the search manipulations. (1) (

Experiment 2 -Younger Audience, 4 Treatment Conditions Experimental Design
We obtained participants for the second experiment from the same independent survey panel company used in Experiment 1. However, in experiment 2, we limited participation to users who were 18-25 years old and who were college students at the time of the experiment or who had at least 2 years of college education. We did this to analyze the degree to which our results extend to a college-aged population, a population that tends to be disproportionately more likely to pirate than other demographic segments are (e.g., Waldfogel 2007, Vandiver et al 2012).
The second difference between our first and second experiments is that our second experiment adds two additional treatment conditions. We retain our original "legal" and "infringing" manipulations that, respectively, remove all pirated or legal options from the first page of results and from the first three results displayed on subsequent pages. However, in experiment 2 we add two additional manipulations that soften the impact of these manipulations by only changing the top three results on the first page and making no other changes to the displayed results. Specifically, the two additional treatment conditions are:  Condition 2a -Mild legal content manipulation: The first 100 search results were retrieved from a major search engine and were displayed to the searcher such that the first 3 results on the first page (and only the first page) were replaced (if necessary) with results that offer legal options to rent/purchase the movie.
 Condition 3a -Mild piracy content manipulation: The first 100 search results were retrieved from the major search engine and were displayed to the searcher such that the first 3 results on the first page (and only the first page) were replaced (if necessary) with results that offer infringing (piracy) options to download/stream the movie.

Data and Results
Following our approach in experiment 1, in our second experiment we included in our analysis only those users who claimed the $20 virtual Visa card, used our custom-built search engine for at least one search, and reported the source from which they acquired the movie and how much they paid for it. A total of 234 participants (86 men, 148 women) qualified for our study. 9 In Table 6 we report average statistics for these participants across the manipulation conditions. 9 Approximately 650 participants that were invited to participate in the study and expressed an interest in watching one of the movies in the study. A total of 550 participants logged into the system and were presented with the task details. Out of these participants 270 completed the experimental task as instructed, and of these participants 234 (86 men, 148 women) qualified for our study by also completing the post-experiment questionnaire. The dropout rates in each of the different stages described above are not statistically different across experimental conditions. These statistics show that the distribution of demographic characteristics and attitudes towards piracy are similar across the control and treatment groups, as one would expect given the experimental assignment.  As in experiment 1, we also compare the initial search intent expressed by users across the different treatment conditions (Table 7), finding no significant differences in expressed intent across conditions.  consumed through a pirated channel). Note that in the general population sample, we found that 80% of the participants in the control condition acquired the movie through a legal channel.
Comparing this figure to the 61.9% figure above suggests that, as expected, younger users are less likely to acquire content through legal channels than are users in the general population.
Relative to the proportion of legal purchases in the control condition, Table 8 shows that participants who were assigned to the mild legal content treatment condition were more likely to acquire the movie from a legal channel (P = 75.5%, SD = 43.4%) than were participants in the baseline condition (P = 61.9%, SD = 43.4%), a difference that is statistically significant (t(83) = -1.39, p < 0.1). We also see that the effect of the more intense legal content manipulation (which is the same the legal content manipulation used in study 1) resulted in a statistically higher proportion of legal purchases compared to the milder legal manipulation condition (P = 91.7%, SD = 27.9%, ttest: t(63) = -2.18, p < .05).
Conversely, the results show that participants in the mild infringing condition were significantly less likely to acquire content from a legal channel (P = 47.7%, SD = 50.5%) relative to the baseline condition (P = 61.9%, SD = 43.4%; t-test: t(84) = 1.32, p < 0.1). Likewise, the effect of the more intense infringing content manipulation (which is the same as the infringing content manipulation used in study 1) is stronger than that of the mild treatment, resulting in a directionally lower proportion of legal purchases compared to the baseline condition (P = 38.2%, SD = 49%  These results not only suggest that increased search costs associated with finding pirated content can have a significant impact on user behavior even for younger users; they also suggest that the effect varies with the intensity of the treatment, and is present even for relatively minor reductions in search costs.  Our analysis of the initial intent expressed by participants is also consistent with that seen in experiment 1. Table 9 suggests that users who initially express intent to consume legally are significantly less likely to do so when placed in the infringing content manipulation (63%) than in either the other treatment conditions (p < 0.05 in all four t-tests performed). Likewise, users who initially express intent to consume infringing (pirated) content are significantly more likely to consume legally in the legal content manipulation (71%) than in the other treatment conditions (p < 0.1 in the t-test that compares the mean of this condition with the mean of the intense legal manipulation, and p < 0.05 in the other three t-tests performed).

Legal content manipulation
Finally, we use a logistic regression model to control for and analyze differences between groups based on observed characteristics. This model is similar to the model from Experiment 1, except that it includes dummy variables for the additional treatment conditions. Specifically, we use the following logistic regression model to control for observable participant characteristics: where, is an indicator variable denoting whether participant was treated with the mild non-infringing search condition; is an indicator variable denoting whether participant was treated with the infringing search condition; and the other variables are the same as before. We present the results of this model in Table 10 below.
The results in Table 10 are consistent with our means comparison results above in that the intense treatment variables are statistically different from the control condition and that all treatment variables have the expected signs (the non-infringing treatment condition increases the likelihood of purchasing legally and the infringing treatment condition decreases the likelihood of purchasing legally). As with experiment 1, the results show that participants who consumed infringing content in the past are less likely to purchase the movie legally, and that those who use a search term that reveals a clear non-infringing intent are more likely to purchase legally.
In summary, the results from experiment 2 are consistent with those in experiment 1 in showing that search results can have a significant impact on whether consumers choose to acquire legal or infringing media content. In addition, experiment 2 further shows that these results hold for younger uses and that stronger treatments yield stronger responses. (1) (

Discussion
As search engines become an increasingly important source of public information, they are being asked to play a stronger role in regulating what information is made available to the public. This role is seen in a variety of contexts including court ordered enforcement of an individual's "right to be forgotten" in a variety of countries, the United States Federal Drug Administration and Department of Justice requirements that search engines ban sponsored search listings from non-FDA approved pharmacies, and efforts by Google to inform its consumers about hacked sites, 10 insecure sites, 11 or sites suspected of malware and phishing.
Recently, many in the content industries have added pirated content to this list by asking whether changing the availability of pirated content in search results will change consumers' choices for piracy versus legal consumption channels.
The goal of this study is to assess whether search results can influence consumers choices for piracy versus legal consumption channels. To address this question, we used a unique experimental design to simulate consumers' online movie search and consumption processes.
Our custom search engine allowed us to experimentally manipulate the rank and positioning of pirate and legal links in the search results.
We ran our experiment in two stages. In the first stage, we ran the experiment on a representative sample of the population recruited through an independent company maintaining large survey panels. We then exposed these users to three randomly assigned search conditions: a control condition which displayed search results from a major search engine, an infringing content treatment condition which artificially promoted infringing sites in the search results, and a legal content treatment condition which artificially promoted legal sites in the search results. In the second stage of the experiment we recruited a separate sample of collegeaged students (18-25 year olds) as participants, and added two additional treatment conditions to the experiment: a mild legal treatment condition and a mild infringing treatment condition.
Our results in the first experiment with the general population show that 80% of users in the control condition choose to purchase content through legal channels. Relative to this baseline, we find that 94% of users in the legal treatment and 57% of users in the infringing treatment purchase content through legal channels. We see similar results for the second experiment with a college-aged audience: 62% of users in the control condition purchase content versus 92% in the legal treatment and 39% in the infringing treatment. The second experiment also shows that stronger treatments lead to stronger outcomes: 76% of users purchase in the mild legal treatment (versus 92% in the legal treatment) and 48% of users purchase in the mild infringing treatment (versus 39% in the infringing treatment).
We also find evidence that search results can even affect the behavior of users with a stated preference for legal or infringing content. When we classify user's intentions based on their initial search terms, we found that users who initially express an intent to consume legally are less likely to purchase legally in the infringing treatment condition than in other conditions, and that users who initially express an intention to consume through pirate channels are more likely to consume legally when they are placed in the legal treatment condition.
Together our results suggest that reducing the prominence of piracy links in search results can have a significant impact on consumer behavior, and may be a viable policy option in the fight against intellectual property theft. However we also note that more research is needed to shape such a policy. Identifying infringing content in real-time search results is different, and potentially more challenging, than identifying malware or phishing content. There also could be significant social costs from false positives, and opportunities for strategic manipulation of which sites and links are designated "infringing." Any policy should take these issues into account, and may benefit from the lessons learned from similar issues in the context of the enforcement of DMCA notices.
Our experiment is, of course, not without limitations. One possible limitation of our study is that while our custom search engine closely replicates the functionality of standard search engines, it still could be perceived differently by the users relative to a non-experimental setting. A second possible limitation is that the participants received a $20 Visa virtual card and were asked to follow a specific experimental task. While this is standard practice in the experimental economics literature, it may not necessarily reflect their normal movie consumption environment. For these two reasons we emphasize that, in interpreting our results, one should focus on the relative levels of sales/piracy between the control and treatment conditions as opposed to the absolute levels within any particular condition. Finally, we note that our experiment does not allow us to test potential longer-term changes in user behavior from manipulating search results.