AN EXPERIMENT IN HIRING DISCRIMINATION VIA ONLINE SOCIAL NETWORKS

. We investigate whether personal information posted by job candidates on social media sites is sought and used by prospective employers. We create profiles for job candidates on popular social networks, manipulating information protected under U.S. laws, and submit job applications on their behalf to over 4,000 employers. We find evidence of employers searching online for the candidates. After comparing interview invitations for a Muslim versus a Christian candidate, and a gay versus a straight candidate, we find no difference in callback rates for the gay candidate compared to the straight candidate, but a 13% lower callback rate for the Muslim candidate compared to the Christian candidate. While the difference is not significant at the national level, it exhibits significant and robust heterogeneity in bias at the local level, compatible with existing theories of discrimination. In particular, employers in Republican areas exhibit significant bias both against the Muslim candidate, and in favor of the Christian candidate. This bias is significantly larger than the bias in Democratic areas. The results are robust to using state- and county-level data, to controlling for firm, job, and geographical characteristics, and to several model specifications. The results suggest that 1) the online disclosure of certain personal traits can influence the hiring decisions of U.S. firms and 2) the likelihood of hiring discrimination via online searches varies across employers. The findings also highlight the surprisingly lasting behavioral influence of traditional, offline networks in processes and scenarios where online interactions are becoming increasingly common.


Introduction
The rise of Internet and social media services like online social networks has created new channels through which employers and job candidates can find information about each other. Those channels can facilitate and improve the matching between firms and workers. However, job seekers also reveal, online, information that would not be easily discovered during the interview process, and which may be even illegal for employers to request or use in the hiring process. Thus, although new online tools can facilitate labor market matching, they may also create a new arena for labor market discrimination.
To date, no field data has demonstrated how online information affects the hiring behavior of U.S. firms. In surveys, employers admit to using various online services to research job candidates. 1 However, the Equal Employment Opportunity Commission (EEOC) has cautioned firms about risks associated with searching online for protected characteristics, 2 and thus may have dissuaded some firms from using social media in the hiring process. Some states have even drafted bills limiting employers' ability to access candidates' online information. 3 Thus, whether hiring bias results from personal information posted online remains an open question.
Furthermore, in surveys, employers claim to use social media merely to seek job-relevant information about candidates. 4 However, much more private information can be gleaned from the online presences of prospective hires. On a social media profile, a status update can reveal a candidate's place of worship, a comment can suggest sexual orientation, and a personal photo can reveal ethnic origins. In a country known for its cultural assimilation of immigrants (Vigdor 2008), private differences that have traditionally been scrubbed out for work or education might now be more visible online. Whether U.S. employers react to such online personal information, rather than merely to the online professional information they may seek, is not known.
We present a randomized field experiment testing the joint hypothesis that (i) firms search online for information about job applicants and (ii) change their hiring behavior according to manipulated online personal information. The experiment relies on a methodology consisting of the creation and careful design 1 Various surveys suggest that U.S. employers search job candidates online, but the reported frequency of searches varies considerably across surveys. See "Survey Shows 48% of Employers Conduct Social Media Background Checks." EmployeeScreenIQ. Accessed February 26, 2016. http://www.employeescreen.com/iqblog/social-media-2/48-of-employers-conductsocial-media-background-checks/; "Employers Using Social Networks for Screening Applicants." Wikibin. Accessed February 26, 2016.
http://wikibin.org/articles/employers-using-social-networks-for-screening-applicants.html; "Ponemon Institute/Littler Mendelson Study." International Association of Privacy Professionals. Accessed February 26, 2016. https://www.privacyassociation.org/publications/2007_12_ponemon_institute_littler_mendelson_study; Johnston, Stuart J. 2010. "Microsoft Survey: Online 'Reputation' Counts." Internet News, January 27. Accessed February 26, 2016. http://www.internetnews.com/webcontent/article.php/3861241/Microsoft+Survey+Online+Reputation+Counts.htm. 2 See Theodore Claypoole, "EEOC Regulations Spotlight Social Media," Womble Carlyle, last modified May 24, 2011, accessed February 26, 2016, http://www.wcsr.com/Insights/Alerts/2011/May/EEOC-Regulations-Spotlight-Social-Media. On the potential risks of using social media in hiring for firms, see Harpe (2009). 3 For instance, Texas S.B. 118 aims to prohibit an employer from "requiring or requesting access to the personal accounts of employees and job applicants through electronic communication devices; establishing an unlawful employment practice." See S.B. 118, 83rd Leg., Reg. Sess. (Tex. 2013). 4 See surveys listed in footnote 1. of online presences of fictional individuals. We design social media profiles for four job candidates -a Muslim versus a Christian candidate, and a gay versus a straight candidate -to manipulate personal information that may be hard to discern from résumés and interviews, and that may be protected either under federal or some state laws (henceforth referred to as "protected information"). 5 We manipulate the candidates' personal information exclusively via their online profiles, using material revealed online by actual members of popular social networking sites and job seeking sites. Candidates' professional background and résumés are kept constant across conditions. After vetting the realism and quality of candidates' online profiles in a randomized online pilot experiment (henceforth the "online pilot"), we submit résumés and cover letters on behalf of those four candidates to over 4,000 U.S. job openings, with a single application sent to each employer (henceforth the "field experiment"). The résumés and letters contain no references or links to the candidates' manipulated personal information: To be treated by our manipulation of online profiles, the employer must independently choose to search online for information about the candidate using the name indicated on the submitted résumé. The main dependent variable in the field experiment is the number of interview invitations each candidate receives (i.e., callbacks). We compare the callback rate for the Christian candidate to that for the Muslim candidate, and the callback rate for the straight candidate to that for the gay candidate. We control for independent variables used in related literature (e.g., Bertrand and Mullainathan 2004;Tilcsik 2011), including firm characteristics, job characteristics, and geographical characteristics of the job's location. We also use Google and LinkedIn data to estimate the frequency with which employers search the candidates online. We test for a stronger callback bias in states and counties that have a higher prevalence of demographic traits associated with bias in prior research. More negative attitudes to both Muslims and gay people have been found among survey respondents who are Republican, older, and who do not personally know Muslims or gay people (Arab American Institute 2012;Pew Research Center 2012;Pew Research Center 2013). Thus, we test for stronger bias in firms located in states and counties that have a high fraction of Republican voters, a high median age, and low fractions of Muslims (in the Muslim-Christian manipulation) or gay people (in the gay-straight manipulation).
Nationwide, we detect no difference in callback rates for the gay candidate compared to the straight candidate, but we find 13% fewer callbacks for the Muslim candidate compared to the Christian candidate.
While the difference is not significant at the national level, it exhibits significant and robust heterogeneity in bias at the local level. We find that discrimination a) against the Muslim candidate and b) in favor of the Christian candidate varies significantly and robustly with employer characteristics in manners predicted by 5 Different types of personal information enjoy different levels of protection across U.S. states. Some personal traits cannot even be inquired about in interviews, while others cannot be used in the hiring decision. Some are protected across all states, and others only in some states. For simplicity, we refer to information about these traits collectively as "protected" information, but investigate statelevel differences in the degree of their protection through our empirical analysis. both theoretical and empirical previous work. In counties with a high fraction of Republican voters, the callback rates for the Muslim and Christian candidates are, respectively, 6.25% and 22.58%. In contrast, in the Democratic counties, the callback rates for the Muslim and Christian candidates are, respectively, 12.37% and 12.13%. The results are robust at the state level.
Using the callback rates for the gay and straight candidates as a benchmark, we find that our results are driven by both a negative bias against the Muslim candidate, and a positive bias toward the Christian candidate. Furthermore, we find that the findings are robust to the inclusion of firm characteristics (including firm size and ownership status) and a host of additional controls, as well as to different categorizations of Republican, mixed, and Democratic states or counties based on electoral results and Gallup Organization surveys. These results are consistent with those of the online pilot, where we find significant bias against the Muslim candidate, relative to the Christian candidate, among subjects with hiring experience who selfidentified as Republican and Christian.
These findings make a number of contributions to the literature. First, an emerging stream of work in information systems research investigates the impact of online information systems on offline behaviors (for instance, Chan and Ghose 2014 investigate the potential role of online platforms such as Craigslist in the transmission of STDs). The findings of our experiment suggest that, while hiring discrimination via online searches of candidates may not yet be widespread, online disclosures of personal traits can significantly influence the hiring decisions of a selected set of employers. At the same time, the results suggest an intriguing phenomenon for scholars of online transactions: to the extent that group-beneficial cooperation plays a role in explaining the findings, the experiment highlights the lasting behavioral influence of traditional offline networks of people physically close to each other. Even as online networks and interactions become more common, they may sometimes facilitate parochial cooperation in local, physical networks. Second, the findings highlight an emerging tension between modern information systems and institutional regulation written for a pre-Internet world. The latter aims to preclude certain information from being used in the hiring process; the former can effectively bypass legislation by allowing individuals to make their information openly available to others online. A similar tension is being studied in the growing empirical (Miller and Tucker 2009;Goldfarb and Tucker 2011) and theoretical (Acquisti, Taylor, and Wagman 2015) information systems literature on privacy and its economic aspects (Stigler 1980;Posner 1981). Third, the paper introduces a methodology -the manipulation of Internet profiles -for field experiments on digital discrimination. One advantage of this method, which we exploit here, is its suitability for investigating discrimination associated with traits that may be protected under a country's laws or that may be difficult to realistically manipulate in résumé-only or audit studies (as most job candidates may refrain from revealing certain types of personal information on résumés). In fact, this methodology (creating online presences to test dynamics in the offline world) may be used not only in studies of discrimination in the job market, but also in comparable studies of bias in access to credit, housing, or educational DRAFT 5 opportunities. Fourth and finally, this paper illustrates a wrinkle in the literature investigating the role of information in economic outcomes. Economists have long been interested in the role of information (Stigler 1962) and signaling (Spence 1973) in job market matching. Recent work has highlighted how hiring mechanisms that reduce candidates' information available to employers (for example, by making the candidate anonymous) may increase interview opportunities for certain categories of applicants (Goldin and Rouse 2000;Aslund and Skans 2012) and may raise social welfare (Taylor and Yildirim 2011). In this manuscript, conversely, we find that Internet and social media platforms can affect interview opportunities for some categories of applicants at the expense or benefit of others, by making more information about candidates available to employers. 6 The extent to which this new information channel will improve labor search efficiency (Kroft and Pope 2014) and reduce labor market frictions (by allowing better matching of employers and employees), or will in fact lead to more discrimination, is likely to be an issue of increasing public policy relevance.

Public Disclosures, Social Media, and Social Networking Sites
The rise of social media has both fueled and been fueled by an arguably unprecedented amount of public sharing of personal information. The shared information frequently includes disclosures and revelations of a personal, and sometimes surprisingly candid, nature. In certain cases, personal information is revealed through fully identified profiles. 7 Other times, users provide personal information under pseudonyms that may still be identified. 8 Some social media users take advantage of privacy settings to manage and restrict their online audiences (Stutzman, Gross, and Acquisti 2012). Others think they do, but actually fail to protect their information. 9 Employers can access shared personal information in many ways. Some job candidates make their online profiles (on social media or blogging platforms) openly accessible to strangers. 10 Others are more 6 In competitive markets, firms may overinvest and collect an "excessive" amount of information in equilibrium, because of a contrast between social incentives and firms' data-gathering incentives (see Burke, Taylor, and Wagman 2012). In a labor market search model, Wagman (2014) finds that firms may search for negative news about applicants and end up collecting too much information, resulting in applicants inefficiently matched with firms of lower productivity. Seabright and Sen (2014) examine how reductions in the cost of applying for jobs may increase the number of applications to a degree that adversely affects firms. 7 For instance, an overwhelming majority of Facebook users in a sample of North-American college students surveyed by Acquisti, Gross, and Stutzman (2014) used real first and last names on their profiles. 8 Numerous examples exist of techniques through which seemingly pseudonymous online profiles can be re-identified across a variety of platforms and scenarios. See, for instance, Narayanan and Shmatikov (2009). 9 Consider the gap between stated and actual privacy settings of online social network users reported by Acquisti and Gross (2006). Similar results have been subsequently found by Madejski, Johnson, and Bellovin (2012). 10 For instance, using data collected for this study (see Section 4.2), we estimate that, in 2011, 42% of all profiles members of the Facebook network of a major North-American college shared "likes" publicly (where a like could be an interest, book, movie, music, or a TV program). Data reported in Acquisti, Gross, and Stutzman (2014) also shows that a majority of Facebook members in a North American city used facial images in their primary profile photos (which are public by default). Facebook data analyzed by Johnson, Egelman, and Bellovin (2012) indicates that around 54% of Facebook members made available to strangers at least some of their profile information.
selective, but sensitive information such as religious affiliation, sexual orientation, or family status may still be indirectly inferable from seemingly more mundane data. 11 Finally, some employers engage in social engineering (such as using friends of friends connections to view a candidate's profile), or even ask for the candidates' passwords to access their profiles. 12 Although the legal consensus seems to suggest that seeking information online about job candidates (or employees) may not violate U.S. law, such searches do raise privacy and legal issues, such as the discriminatory behavior that may result from discovering protected information (Sanders 2011;Sprague 2011;Sprague 2012).
Above-cited surveys suggest that U.S. employers have been using social media to screen prospective job candidates. However, no prior controlled experiment has measured the frequency of firms' usage of online profiles in hiring decisions, and how profile information actually affects those decisions. The study that comes closest to our experiment is Manant, Pajak, and Soulié (2015), who investigate the role of social media in the French labor market. Other than differences in the country of study (USA vs. France) and sample size (about 4,000 versus about 800 employers), Manant et al (2015) focus on a different research question from our study: this manuscript focuses on testing the joint hypothesis that firms search online for information about job applicants, and change their hiring activities based on the personal information they find; Manant et al (2015) focus on investigating the impact of different search costs for finding candidates' information via social media. In addition, Bohnert and Ross (2010) use a survey-based experiment to investigate how the content of social networking profiles can influence evaluations of job candidates. Garg and Telang (2012) analyze how job applicants can use LinkedIn to find connections that may lead to a job offer. Kluemper, Rosen, and Mossholder (2012) assess the relationship between the content of a person's Facebook profile and her future job performance.

Discrimination and Résumé Studies
Following Bertrand and Mullainathan (2004), experiments using written applications to real employers have found evidence of discrimination against people with various traits. Pertinent prior literature has found discrimination against job candidates with Muslim rather than Swedish names in Sweden (Carlsson and Rooth 2007), candidates who openly signal Muslim beliefs (relative to other religious beliefs) on their résumés in New England , and candidates who explicitly or implicitly signal same-sex sexual orientation (Weichselbaumer 2003;Ahmed and Hammarstedt 2009;Tilcsik 2011), 13 but no evidence of discrimination against Muslims (and little systematic discrimination by caste in new economy sectors) in India (Banerjee et al 2009). 11 For instance, Jernigan and Mistree (2009) show that a Facebook member's sexual orientation may be inferable from knowledge of her friends' network. 12 Cases in which employers asked job candidates to provide passwords to their online profiles have been reported in the media. See Stern, Joanna. 2012. "Demanding Facebook Passwords May Break Law, Say Senators." ABC News. Last modified March 26. Accessed February 26, 2016. http://abcnews.go.com/Technology/facebook-passwords-employers-schools-demand-access-facebooksenators/story?id=16005565#.T7-fVMWQMQo. 13 See also Drydakis (2009) and Hebl et al (2002). One crucial difference between existing résumé studies and our approach is that we focus on employers' volitional choices to search online for candidates' information. A second important difference is that we focus on candidates revealing information in personal online social profiles, rather than volunteering personal traits in a professional context. This approach facilitates the investigation of discrimination based on protected information that candidates do not frequently provide in their résumés or during interviews.

Specification
Prior literature on job market search strategies has highlighted employers' use of formal or informal information networks (Rees 1966), and their reliance on intensive or extensive searches (Barron, Bishop, and Dunkelberg 1985). A central privacy concern in online job market search is that Internet information exchanges may bundle job relevant information and personal information to an extent not seen in traditional labor market practices. This raises the question we investigate in this manuscript: Is there evidence that employers search for and discriminate on the basis of publicly posted yet personal information? We test for the existence of hiring discrimination that stems jointly from two, possibly dependent, actions: first, each employer decides whether to search online for a candidate's information; and, second, each employer who finds the candidate's profile is unknowingly treated to the experimental manipulation and chooses whether or not to interview the candidate.
It is important to note the difference between our question and more frequently asked questions in the economics literature on discrimination: Does discrimination exist, to what extent, and in what form? That literature has developed sophisticated methods for testing whether or not discrimination is actually present and for empirically separating taste-based from statistical discrimination (Mobius and Rosenblat 2006;Neumark 2012;Ewens, Tomlin, and Wang 2014). In this paper, we do not attempt to separate bias from search, let alone separate taste-based discrimination from statistical discrimination or other forms of bias. We instead focus on the more basic question of whether or not social media profiles can impact hiring via the combined effects of online search and bias.
Using a between-subjects design, we test for an effect of random assignment to treatment conditions on callbacks: where A i is an indicator of random assignment of employer i to either the Muslim condition (A i = 1) compared to the Christian condition (A i = 0) or to the gay condition (A i = 1) compared to the straight condition (A i = 0), ß 0 and ß 1 are unknown parameters, x i and z i are vectors of, respectively, observed and unobserved regressors capturing employer i's traits, γ and δ are vectors of unknown parameters, and ɛ i is an error term. Callback i equals one if employer i contacts the candidate for an interview and zero if there is no response or an explicit rejection. Assuming successful randomization of A i , the estimate of ß 1 will be an unbiased estimate of the effect of manipulated online profiles on callbacks. It is analogous to an intent-to-DRAFT 8 treat effect, but we refer to it as an assignment effect to emphasize that the main goal of our design is testing the joint effect of (i) choosing to search (and thus unknowingly self-selecting into our experiment's treatment) and (ii) being treated, whereas researchers in the intent-to-treat literature are primarily interested in isolating the treatment effects.
We do not expect search rates to differ across condition assignments, because employers are treated by our manipulated profiles after they choose to search. However, it is likely that both search and discrimination probabilities will vary with employer characteristics, leading to heterogeneous assignment effects. 14 We test for heterogeneous assignment effects in our regression analysis by including interactions between the random assignment and regressors that prior literature predicts may interact significantly with our manipulations.

Dependent and Independent Variables
Our experimental design manipulates personal information that is unlikely to be volunteered by candidates in résumés and that may be risky under federal or state law for employers to inquire about during interviews, but which may be obtained online. Building on résumé studies by Weichselbaumer (2003), Carlsson and Rooth (2007), and Tilcsik (2011), we focus on sexual orientation and religious affiliation.
The dependent measure is interview invitations that the candidate receives from actual employers, either by email, phone call, or letter. Based on prior evidence, we expect callbacks to be more biased in certain areas than in others. Independent surveys consistently find three variables that are associated with more negative attitudes to both Muslims and gay people. Surveys on attitudes to Muslims and Arabs find more bias against Muslims among respondents who are Republican, older, and who know fewer Muslims (Arab American Institute 2012). Separately, surveys on attitudes to gay people find more bias against gay people among respondents who are Republican, older, and who know fewer gay people (Pew Research Center 2013). In communities with a high prevalence of types of people who show bias in these surveys, employers themselves may be more likely to share the attitudes expressed by community members, or they may face incentives to hire candidates whose traits match those of existing employees and community members. Thus, we test for stronger bias in firms located in states and counties that have: (i) a higher median age than the U.S. median age, (ii) a lower fraction of the population that is Muslim than the U.S. fraction of Muslims (in the Muslim-Christian manipulation), or a lower fraction of households that are male-partner households than the U.S. fraction of male-partner households (in the gay-straight manipulation), and (iii) a high fraction of voters in the 2012 Presidential election who voted for the Republican candidate (see Section

5.2.2).
In addition, we control for a host of other variables used in comparable résumé studies, particularly Bertrand and Mullainathan (2004) and Tilcsik (2011), including employer characteristics (such as industry or number of employees), job characteristics (such as job-specific requirements), and regional characteristics based on location of the job (for instance, variations in state and local policies to protect gay people). Finally, we include controls that are specific to our experimental design's online dimension, such as a measure of Facebook penetration across states (a proxy for the likelihood that an individual in a particular state may be inclined to search online for the candidate's social media profile).

Interpreting a Null Result
Given that our design tests jointly for online searching and for discrimination on the basis of manipulated online profiles, a null effect in the field experiment may signal low levels of either of these behaviors. In addition, the experiment could fail to reject the null hypothesis in a number of additional ways.
First, employers may search the candidates online but fail to find our profiles. Our design addresses this concern by using name-selection criteria that produce high-ranked search results, and by checking, for the duration of the experiment and across a variety of platforms, that candidate name searches produce the desired results (see Section 4.2). Another possibility is that the manipulations may fail to signal the traits we intended to manipulate. To address this possibility, prior to conducting the field experiment, we use an online pilot to test whether we successfully manipulated beliefs about the candidates' personal traits. A further concern is that the candidates may be either under-or over-qualified, creating floor or ceiling effects in callback rates. We address this possibility by designing the résumés and professional backgrounds of the candidates in concert with human resources professionals, and testing their quality during the online pilot. An additional possibility is that employers may not pursue the candidates because of suspicions that they are fictional. Consequently, we design résumés and profiles using information that real individuals (including members of job search sites and members of the same social network the candidates were purported to belong to) had also made publicly available on their profiles or on job search sites. Furthermore, we choose privacy and visibility settings that were common among the profiles of real members. The online pilot tests the success of our efforts at creating realistic profiles by asking questions aimed at discerning whether online subjects had doubts about the veracity of the candidates (see Section 5.1). Finally, a null effect may be consistent with scenarios in which employers may search, but at a later stage in the hiring process (for instance, after interviews with the candidates). Our design would not capture this behavior, and would not be able to distinguish this scenario from one with little or no discrimination.

Design
We implemented a between-subjects design with four treatment conditions consisting of social media profiles for a single job candidate. We manipulated religious affiliation (a Christian versus a Muslim male) and sexual orientation (a gay versus a straight male). 15 Thus, the experimental conditions represent a range of traits that include federally protected information (religious affiliation) 16 and information protected only in certain states (sexual orientation). 17 While these traits enjoy different levels of protection under U.S. law, job candidates can signal this information online both in an explicit manner (e.g., self-descriptions on a profile indicating one's sexual orientation) and an implicit one (e.g., a primary profile photo suggesting the individual's religious affiliation).
For each of the four candidates, we designed: (i) a résumé; (ii) a profile on LinkedIn (a popular online professional network commonly used by human resource professionals and job seekers, henceforth referred to as "PN," as in "Professional Network"); and (iii) a profile on Facebook (a popular social networking site commonly used for socializing and communicating, henceforth referred to as "SN," as in "Social Network"). 18 We designed candidates' résumés and PN profiles to show identical information across conditions, except for the candidates' names. In contrast, the candidates' SN profiles manipulated information about the candidates' personal traits using various profile fields, including those that some employers admit to using during the hiring process in semi-structured interviews (Llama et al 2012). Each applicant's name corresponds to an experimental condition; the names link together a candidate's résumé, PN profile, and SN profile. 19 Because a single name links all of these materials, we can submit just one application to each employer in our sample. 20 Submitting multiple candidates' applications to the same employer would increase the risk of an employer detecting social media profiles which have identical photos and generic information, yet come from candidates with different names and condition-specific information.

Design Approach and Priorities
We designed profiles that (i) were realistic and representative of the population of SN users with the traits we were manipulating, and (ii) held constant direct signals of productivity outside of beliefs stemming from the trait itself. As described in the rest of this section, we populated the résumés and online profiles using existing information posted online by actual SN members demographically similar to the candidates, and by individuals who listed their résumés on job searching sites. The SN profiles are the vehicles for the manipulations, and responses to these profiles comprise our study's core. The remainder of this section discusses our design in more detail.

Candidates' Information
Names. We designed first and last names representing a U.S. male for each of the conditions used in the experiments. We chose first names common among U.S. males in the same age group as our candidates, and assigned identical first names to candidates in matching pairwise conditions (the gay and straight candidates were both named Mike, and the Muslim and Christian candidates were both named Adam). 21 We then designed last names by altering letters of existing but similarly low-frequency U.S. last names. The last names were chosen to have the same number of syllables, identical lexical stress, and similar sounds across matching pairwise conditions (see Appendix Table A1 for more detail). A potential concern is that Christian and Muslim names often differ. However, roughly 35% of the about 2.35 million American Muslims are, like our candidate, US-born, and many have Anglo-Saxon names. 22 We iteratively tested several combinations of first and last names until we found a subset that satisfied three criteria. Criterion 1 was that the exact first and last name combination would be unique on SN and PN: no other profile with that name should exist on the network. Criterion 2 was that SN and PN profiles designed under each name would appear among those names' top search results conducted with the most popular search engines (Google, Bing, and Yahoo), as well as searches conducted from within the SN and the PN social networks. We continuously monitored the fulfillment of these criteria for the experiment's duration. 23 Criterion 3 was the most critical one: names alone should not elicit statistically significant differences in perceptions of the manipulated traits. We conducted two checks of this criterion. First, we tested that names, by themselves, did not influence perceptions of the traits manipulated in the profiles. We recruited 496 subjects from Amazon Mechanical Turk (or, MTurk) -a popular platform for behavioral experiments 24 -and presented each with one first and last name combination, randomly chosen from the list of possible names. Each subject then responded to a questionnaire on associations between that name and the traits manipulated in field experiment. We selected names for the field experiment that did not elicit statistically significant differences in perceptions of traits (see Appendix Table A5), and then randomly assigned them to candidates. The second check of Criterion 3 combined names, résumés, and professional online profiles, and checked that names and professional information did not elicit differential propensity to invite a candidate to an interview (see Section 5.1).

Email addresses and telephone numbers.
For inclusion in the résumé and cover letters sent to companies, we designed email addresses using a consistent format across the profiles and registered a telephone number for the applicants. We used automated message recordings for the number's voice messaging service. This message was identical across conditions and recorded with the voice messaging service's standard greeting.
We also used the same postal address for all candidates, corresponding to a residential area of a mid-size North-American city.
Résumés. The résumés contained information depicting a professional and currently employed candidate.
Each résumé contained the candidate's contact information, educational background, and work experience, as well as technical skills, certifications, and activities. The résumés were held constant across conditions, except for the names of the applicants and their email addresses. 25 Hence, professional and educational backgrounds did not vary across experimental conditions, thus holding constant the candidates' job market competitiveness. The information included in the résumé was modeled after résumés found on websites such as Monster.com and Career.com for job seekers demographically similar (in terms of age and educational background) to the candidates. The résumé represented a candidate with a bachelor's degree in computer science and a master's degree in information systems. 26 Creating a single SN and PN profile per candidate constrained the types of jobs to which we could apply. Hence, all résumés needed to be consistent with the constant information provided in the candidates' social media profiles (for instance, all candidates' profiles exhibited the same master's degree in information systems). Therefore, the openings (and respective résumés) tended to be technical, managerial, or analytic in nature. Two human resource recruiters vetted the résumés for realism, professionalism, and competitiveness before they were tested in the online pilot. A design objective (also tested during the online pilot) was to create a sufficiently competitive candidate for stimulating the level of interest necessary to generate an online search, but not so competitive as to outweigh any potential effect arising from an employer's perusal of the candidate's online profile.
The résumés did not include links to the candidates' personal profiles or references to the personal traits we manipulated on the SN. The experimental design relied entirely on the possibility that employers 25 Research assistants blind to the experimental conditions were allowed to add up to two technical skills (for instance, Java) or certifications (for instance, CISSP certification) to a résumé if the job description required those skills or certifications as preconditions. This fine-tuning always occurred before a candidate's name was randomly added to the résumé, and therefore before the candidate was assigned to a job application. 26 We prepared 10 versions of the same résumés, focusing on slightly different sets of expertise: web development, software development, quality assurance, project or product management, medical/healthcare information, information systems, information security, business intelligence, business development, and analytics. A sample résumé is presented in Appendix Table A2. would autonomously decide to seek information about applicants online, searching for their names either on popular search engines or directly on popular social networks.

PN ("Professional Network") profiles.
We designed PN profiles for each name, maintaining identical profile information across conditions. The content of the profiles (see Appendix Table A3) reflected the information provided in the résumés. To increase realism, we also designed additional PN profiles for other fictional individuals and connected them to the candidates' profiles, so that they would become "contacts" for our candidates. We took great care to avoid any possible linkage between the actual candidates' profiles. 27 The profiles with data extracted from actual profiles of social network users who were demographically similar to the candidates; we made sure that the overall amount of public self-disclosure in our profiles would be equivalent to the amount of self-disclosure in actual social network profiles of users demographically similar to our candidates; we designed profiles to include a combination of information that changed based on the experimental condition and information that was held constant across all conditions; and finally, we posted the same amount of manipulated and constant information for each candidate (therefore, the Christian and Muslim candidates' profiles, and the gay and straight candidates' profiles, presented equivalent information about the strength of their religion, or sexual orientation).
First, we downloaded the public profiles of 15,065 members of the same Facebook college network from which the candidates purportedly graduated. The vast majority of those profiles belonged to individuals with a similar age range (their 20s), current location (a North-American mid-size city), and educational background as our candidates. As further detailed below, among that set of profiles we then focused on those with the same gender (male) as our candidates, and who self-disclosed traits matching the ones manipulated in the experimental conditions (that is, Christian, Muslim, straight, or gay). Then, we designed several components of each SN profile based on the information we had mined from real online profiles: personal 27 We also created websites, email accounts, and PN presence for some of the companies reported in the candidates' résumés, as well as other workers at those companies and potential letter-writers, in order to have a complete, believable background should anyone search. The candidates were also registered as alumni from the institution that granted their degrees. As noted, we took down these materials after the completion of the experiment. 28  and therefore visible to a human resources professional. To avoid potential confounding effects from overdisclosure, we refrained from showing certain fields that only a minority of users of the network actually publicly show, such as posts, status updates, and friends list.
Appendix Table A3 presents the resulting information included in each of the four SN profiles. By design, some of these profile components (the non-condition-specific traits) were constant across the different profiles and thus across the different experimental conditions. Other components (the condition specific traits) were manipulated across conditions. We chose a mix of condition-specific and non-conditionspecific traits that replicated the combination and balance we observed in the actual profiles we mined and analyzed (as described above). The manipulation of fields representing different types of personal information was meant to increase the realism and ecological validity of our experimental design -which we tested in our pilot experiment. We discuss each type of component in the following sub-sections.

Information kept constant across SN profiles.
By design, we made the candidates' primary profile photo, secondary photos, current location, hometown, age, education, employment history, and friend list constant across conditions. Basic personal information (such as the SN member's current location and city of birth, educational background, employment history, hometown, and age) was made publicly visible on the profile and kept constant across the conditions. That information was made consistent with the content of the résumé. The candidate was presented as U.S. born and English speaking. 30 The photo on the candidates' social media profiles was of a Caucasian male with dark hair and brown eyes. We picked the photo after recruiting non-professional models on Craiglist and asking them to submit a portfolio of personal photos. About forty subjects (recruited via MTurk) rated all models who submitted their photos along two 7-point Likert scales, indicating their perceived attractiveness and professionalism. We selected one male model that received median perceived attractiveness and professionalism ratings. 29 Johnson, Egelman, and Bellovin (2012) report that only 14.2% of the users of a popular social network whose profile information they mined had a public wall (that is, visible status updates and comments). That noted, we reiterate that the profiles used in the experiment contained equivalent amounts of information across conditions, so that any difference in callback rates could not be merely imputed to a higher amount of disclosure by one type of candidate over another. Furthermore, we note that several fields (such as name, gender, primary photo, profile photo, and networks) are mandatorily public on the network we used for the experiment. 30 The Muslim candidate's social network profile also presented him as speaking both Arabic.
We designed numerous "friends" profiles to connect to the candidates' profiles. Again, the number and identities of friends were identical across conditions. 31 The set of friends showcased a mix of names, backgrounds, and profile visibility settings. We set the friends list to "private" -that is, not visible to an employer -because of extant research suggesting that the overwhelming majority of members of the network do not publicly disclose their friend lists. 32 Information manipulated across SN profiles. Some fields in the profiles were manipulated across conditions: close-ended text fields (e.g., interests), open-ended text fields (e.g., quotations), and the background image. Furthermore, we manipulated the candidates' sexual orientation by filling out the field "interested in" (either male interested in females or interested in males), and we manipulated religious affiliation through the "religion" field (either Christian or Muslim, with no specific denomination).
We abided by a number of principles in designing the manipulated information. First, we constructed "baseline" profile data, which was constant across all conditions, including information such as specific interests and activities statistically common among the SN profiles we had previously mined ("baseline information"). We then augmented the profiles with additional data (such as additional interests or activities) specific to the traits that we wanted to manipulate ("treatment information"). Both "baseline" and "treatment" information were extracted from real, existing SN profiles. The profiles therefore represented realistic pastiches of actual information retrieved, combined, and remixed from existing social media accounts. 33 We took care to avoid confounding the trait we were manipulating with signals of worker quality beyond signals inherent in the manipulated trait itself. Furthermore, the different candidates' profiles were designed to disclose the same amount of personal information, including the same quantity and types of data revealing their sexual orientation or religious affiliation, so as not to create profiles with unbalanced condition-specific disclosures. For similar reasons, we took great care to ensure that the information revealed by the candidates would not be construed as over-sharing, or as a forced caricature of what a Muslim (or Christian, or gay, or straight) profile should look like. The combination of baseline and treatment information of true existing profiles was one of the strategies we used to meet this goal; using only information existing in other SN profiles, and making certain fields publicly inaccessible, were two others. The online pilot 31 Also in the case of the SN, we attempted to create a number of friends that would match the median number of connections of U.S.-based SN profiles at the time the experiment started (around 100). However, we were only able to create 49 friends, due to technical barriers such as (as noted earlier) the need to provide unique mobile phone numbers to create new profiles on the network. As noted, we prevented cross-linkages of the candidates via their common networks of friends by appropriately selecting privacy and visibility settings for the social network profiles. 32 See Johnson, Egelman, and Bellovin (2012). Although we made the list of friends private, we did not delete the friend profiles because 1) some of those friends' comments appeared on the candidates' profile photos, adding realism to the profiles; 2) the presence of a network of friends decreased the probability that the candidates profiles could be identified as fake and deactivated by the network (none of the candidates profiles was deactivated during the duration of the experiment). 33 Technical limitations inherent in designing an experiment via online social networks precluded us from the possibility of randomizing the entire content of the candidates' profiles by creating thousands of different profiles for each candidate. On the other hand, our approach achieves ecological realism by relying on existing profile data. experiment, with its open-ended questions about the profiles, tested whether our goal was attained (see Section 5.1. and Appendix A).
Close-ended text fields such as interests and activities were extracted using a combination of manual and statistical analysis from real SN profiles of people who shared demographic characteristics with the candidate (for the "baseline information") and who exhibited the same characteristics we manipulated (for "treatment information"). For example, when text involved countable objects such as one's favorite books or movies, we calculated the most popular items listed by the overall population of SN profiles and used that as the "baseline information" for the candidate. Then, we repeated the operation, focusing on the subset of the public disclosures of over 15,000 SN profiles that displayed the same traits we manipulated, in order to create the "treatment information." For instance, we constructed a portion of the profile text indicating activities and interests for the Christian male profile using statistical analysis of the entire sample of over 15,000 profiles that also included non-Christian ones; the remaining portion of the profile text concerning his activities and interests was constructed using statistical analysis of the sub-sample of profiles of Christian males at his university. If our sample did not provide enough information (for instance, movies) for individuals with a given trait, we complemented the statistical analysis with a manual analysis of the same SN profiles.
We also extracted open-ended fields (such as personal self-descriptions, personal quotations, or non-textual fields such as profile background images) through manual analysis of existing profiles, since the extreme variety of styles and contents across open-ended texts made a statistical approach unfeasible.

Online Pilot Experiment
Before submitting résumés to actual job openings at U.S. firms, we conducted a pilot experiment, consisting of a randomly assigned questionnaire with online participants and a between-subjects design. The online pilot was designed to test whether the treatment conditions successfully manipulate relevant beliefs, such as the religious affiliation or sexual orientations of the candidates. Furthermore, its open-ended questions were designed to test the perceived realism of the profiles. Finally, the pilot experiment was designed to test whether, in absence of the manipulated personal profiles (SN), the candidates' names, résumés, and professional profiles elicited differential propensities to invite the candidate for an interview. Details of the design, analysis, and findings of the online pilot are presented in Appendix A and are summarized here.
We recruited over 1,750 U.S. residents as participants using Amazon Mechanical Turk. Participants were presented with the candidates' social profiles and résumés prepared for the actual field experiment.
Participants included individuals with previous hiring experience -something we exploited in the analysis of the results. Participants were randomly assigned to one of four conditions (gay, straight, Christian, or Muslim candidate), and were provided links to one candidate's résumé, PN profile, and SN profile. The survey instrument had four elements: (i) introduction and randomized manipulation; (ii) measurement of perceived employability; (iii) measurement of beliefs for the purpose of manipulation check; and (iv) open-ended questions and demographic characteristics. Participants were asked to evaluate the candidate. The main dependent variables were hypothetical willingness to call the candidate for an interview and perceptions of the candidate's suitability for the job.
The online pilot found, first, that the treatment conditions successfully manipulated relevant beliefs, such as the religious affiliation and sexual orientation of the candidates. Second, open-ended questions checked for perceived realism of the profiles; they provided no evidence of doubt, among the participants, that the candidates were real. Third, the online pilot tested whether, in absence of links to the manipulated personal profiles (SN), the candidates' names, résumés, and professional profiles elicited differential propensities to invite the candidate for an interview; we found no evidence that the candidates' names and professional materials elicited different responses in the absence of the manipulated online social media profiles. Finally, responses of hypothetical hiring behavior and judgments of employability provided evidence to complement the findings of our field experiment. Consistent with our field findings (presented further below in this section), manipulated profiles in the online pilot elicited no bias against the gay candidate, relative to the straight candidate. However, among subjects with hiring experience, we found highly significant bias against the Muslim candidate relative to the Christian candidate, especially among those who self-identify as Republican and Christian.

Field Experiment
The field experiment consisted of a between-subjects design in which each employer was randomly assigned to receive one job application from either the Muslim, Christian, gay, or straight candidate. The job application period lasted from early 2013 through the summer of 2013. Samples used in résumé studies (see Section 2.2) have ranged from a few hundred to a few thousand employers. We aimed for the higher end of the spectrum, ultimately applying to every U.S. job opening that fit our pre-defined criteria of a reasonably suitable position for our candidates. This amounted to 4,183 U.S. job applications (or roughly 1,045 employers per experimental condition). Ten applications failed to meet criteria we had defined in our procedures for acceptable and complete applications, leaving us with 4,173 usable applications.
We extracted job openings from Indeed.com, an online job search site that aggregates jobs from several other sites. 34 We selected positions that fit the candidates' backgrounds -namely, positions that required either a graduate degree or some years of work experience. For each position, we sent the most appropriate version of our résumé. (Recall from Section 4.2 that we designed different versions of the résumé to fit ten different job types covering a combination of technical, managerial, and analytic positions.) We defined several criteria that jobs and companies had to pass for us to apply to them. We focused on private sector firms. Primarily, the job had to be related to the candidates' background and level of 34 Appendix Table A4 lists the search terms used to find different type of jobs.
experience, although we also included (and controlled for) positions for which the candidates could be considered slightly over-or under-qualified. In addition, we carefully avoided sending two applications to the same company, or to companies that were likely to share HR resources such as databases of applicants (for instance, parent companies of firms to which we already applied). 35 We also excluded staffing companies, companies located in the same geographic region as the candidates' reported current location, and companies with 15 or fewer employees (to limit the costs imposed on them by the process of searching and vetting fictional job candidates). 36 All applications (résumés and cover letters) were submitted online, either by email or through employer-provided web forms.
We recorded the city and state listed on job postings, and, when possible, we recorded the city and state where the job would be located. When job location was not provided, we used the location of the company's headquarters. We obtained this measure for all observations and employed it for our state level analysis. From the company name, we were able find the street address of the company headquarters for all but a few hundred observations. We used ArcGIS 37 to match this street address to its county. We then merged our data with county level data from the American Community Survey 38 based on the county where the company headquarters is located. We used a combination of two data sources to estimate the frequency with which employers searched for the candidates online. One data source consisted of Google AdWords "Keyword Tool" statistics. These publicly accessible statistics capture the number of times a certain term is searched on Google from various locations. We used this tool to estimate the number of times the exact names of the candidates were searched from U.S. IP addresses. The second data source consisted of statistics provided by the PN network (LinkedIn) via so-called "Premium" accounts. If a user subscribes to a Premium account, that user will be able to get information such as the count of visits to its PN profiles, and in some cases the actual identity of the visitors. We subscribed to Premium accounts for each of our candidates' profiles in order to track visits to those profiles.

Search Trends
Each of these sources of data is imperfect. Google Keyword Tool statistics provide aggregate monthly means for searches of a given term, rather than raw data. Furthermore, if the mean is higher than zero but below 10 searches per month, no exact count is provided. Similarly, "Premium" PN accounts do not actually track all visits to that account (for instance, in our tests, visits from subjects that had not logged onto the network went undetected and uncounted; visitors who only viewed the summary profile of the candidate, rather than his full profile, also were not detected or counted; in addition, certain Premium accounts may allow "invisible" views of other LinkedIn profiles). Nevertheless, considered together, these data sources do offer a rough estimate of the proportions of employers searching the candidates online.
We tracked Google Keyword Tool statistics and LinkedIn data over a period of several months.
Based on the documented searches we detected on Google and on the PN, we can estimate a minimum lower threshold of employers who searched for the profiles at 10.33%, and the likely proportion of employers who searched at 28.82% (see Appendix B for details). These estimates appear consistent with the results of a 2013 CareerBuilder survey of 2,291 hiring managers and HR managers, according to which 24% claimed to "occasionally" use social media sites to search for candidates' information; 8% answered "frequently;" and only 5% answered "always." 40 The search rates highlighted in this section are sample-wide, aggregate estimates, because (as noted) in most cases we could not identify the specific employers visiting the profiles. The sub-sample of cases where we could directly capture the identity of an employer searching for our profiles is small (N = 121), but, unsurprisingly, represents employers who are strongly interested in our candidates. The overall callback rate for the four candidates in this sub-sample was 39.67%. The callback rates for the straight and gay candidates were 31.25% and 40.00%, respectively, and were not significantly different from each other.
However, the callback rates for the Christian and Muslim candidates were 54.84% and 32.14%, respectively and were significantly different at the ten-percent level (using a chi-squared test). This is consistent with the main results presented in the remainder of this section, where we do find evidence of callback bias in favor of the Christian candidate compared to the Muslim candidate among certain types of employers, but no evidence of bias against the gay candidate compared to the straight candidate.

Employer Callbacks
Our primary dependent variable is a callback dummy that equals one if the candidate was contacted for an interview and zero if the candidate received a rejection, no response, or a scripted request for more information. Of the 4,173 observations from both manipulations, 11.20% were callbacks, 15.86% were rejections, 69.29% had no response, and 3.32% were scripted requests for more information.
Timing of Callbacks. Figure A1 (in the Appendix) shows the geographical distribution of applications across the United States. Figure A2 ( Muslims or gay people). We thus test for differences in bias according to geographical differences in the prevalence of these traits in the population.
Although we cannot identify the individual making the callback decision, and thus cannot observe his or her traits, following the literature we use demographic measures in the firm's geographical area.
Specifically, we indicate states and counties with a: (i) higher median age than the U.S. median age, (ii) lower fraction of the population that is Muslim (or gay) than the U.S. fraction, and (iii) a high fraction of voters who voted for Mitt Romney in the 2012 Presidential election. Our measure of political party support in (iii) follows Tilcsik (2011), who used Presidential election data in his audit study, and the Gallup Organization, which regularly produces lists of the ten most Republican and ten most Democratic states based on survey data. Voting results are available at the county level and are based on real behavior. We thus indicate, respectively, the ten states with the highest fractions of 2012 Romney voters and the ten states with 41 We did not apply to job openings that had been posted for over 30 days. It is possible that some job openings may have been filled or were otherwise inactive. We expect these cases to be uncorrelated with the condition assignments. the lowest. We refer to the remaining states as politically mixed. 42   interaction is less robust than the political party support interaction: it is not present at the state level (we show in Section 5.4 that theories of underlying mechanisms can fully explain the age interaction, so it may be a statistical artifact). Finally, we find no significant differences in bias between high and low Muslim counties or states. In the gay-straight manipulation, we find no differences in bias across any of the geographical areas. For instance, in Republican states, 15.38% of the straight candidate applications and 42 We use Presidential election data from Leip (2013), which is compiled, where possible, from final election results from official sources (e.g., certified election results posted by state election boards). We found that election data from news organizations is often not updated to include final election results. In Section 5.2.3 we show that the results are robust with respect to using the Gallup Organization's classification of Republican and Democratic states for the year of 2012, according to daily tracking of self-reported political party identification. 43 Consider an employer search rate of 30% (see our search rate estimates in Section 5.2.1). Suppose there are three types of employers: Type 1 rejects the application and does not search the candidate online; Type 2 considers the candidate further and searches online; and Type 3 considers the candidate further but does not search online. Suppose that 65% of the employers are Type 1, 30% are Type 2, and 5% are Type 3. In this example, the overall search rate is just the prevalence of Type 1 employers, i.e., 30%. If we assume that the callback rate by Type 3 employers (who do not search) equals the average of the callback rates for the two candidates by Type 2 employers (who do search), then we have two equations with two unknowns: P1*0+P2*M+P3(M+C)/2=.02 and P1*0+P2C+P3(M+C)/2=.17, where P1, P2, and P3 are the probabilities of employers being types 1, 2, and 3; and M and C are the callback rates for the Muslim and Christian candidates among employers who search. Sample-wide callback rates of 2% for the Muslim candidate and 17% for the Christian candidate can be explained by callback rates among employers who search of 2% for the Muslim candidate and 52% for the Christian candidate, and a callback rate when employers consider the candidate but do not search of 27%. It is worth noting that if, in Section 5.2.1, we underestimated actual employers' search rates, or if employers in Republican states have a higher than average search rate, then this example implies a smaller callback bias for employers who search.
14.29% of the gay candidate applications received callbacks (see Table 2, Columns (4) and (5)). In Democratic states the callback percentages for the straight and gay candidates were 11.24 and 11.71, respectively. 44 We also test whether the biases found in Table 2 stem from both the Muslim and Christian conditions or only from one. We compare callbacks in each of these conditions to callbacks in the pooled gay-straight conditions -using the latter as benchmarks. Consider columns (6), (7), and (8) of Table 2: both the Muslim and the Christian conditions contribute to the biases shown in Panel A (i.e., in the bias-prevalent areas). Columns (7) and (8), respectively, test for differences in the number of callbacks in (i) the Muslim condition to the pooled gay/straight conditions and (ii) the Christian condition compared to the pooled gay/straight condition. The first row shows that in Republican states, the Muslim callback rate of 2.27% is significantly lower than the pooled gay/straight callback rate of 14.77%. The remaining rows of Panel A show that in Republican counties, older counties, and states and counties with a lower fraction of Muslims, the callback rate in the Christian condition is significantly higher than the callback rate for the pooled gay/straight condition. Thus, both the Muslim and the Christian conditions play significant roles in generating callback bias in areas with a high prevalence of types who show bias in surveys.

The result that both the Muslim and the Christian conditions contribute significantly to the bias in
Republican areas is not sensitive to which sample is chosen as the comparison sample. We find similar results when we separate the pooled gay/straight candidate sample into just the gay candidate sample or the straight candidate sample. Regardless of whether we use the sample with the gay candidate or the sample with the straight candidate, we find that the callback rate for the Muslim candidate in Republican states is significantly lower and the callback rate for the Christian candidate in Republican counties is significantly higher.
In the next sub-section, we present regression analysis using both the state-level and county-level measures of Republican, politically mixed, and Democratic areas. We present the state-level analysis first. However, the state-level analysis has two drawbacks. First, taking all four conditions together, there are 1,745 employers in 10 Democratic states or districts (including Washington DC), 2,244 in 31 mixed states, and just 184 in 10 Republican states. Second, we cannot fully control for state-level covariates with statelevel definitions of political areas. Thus, we address these issues by also using county-level regressions with state-fixed effects. The county-level measures, in addition to allowing state controls (and thus focusing on within-state variation), have a larger sample size in the Republican areas. The sample sizes in the Democratic and Republican counties are 2,475 and 228, respectively. Finally, in Section 5.2.3, we test for robustness with respect to changes in estimators and types of standard errors, and with respect to additional specifications and measures of political areas.  (1) the interaction between the Muslim assignment effect and the Democratic states dummy is positive, 0.145, and significant at the five-percent level. The interaction between politically mixed states and the Muslim assignment dummy is also positive, 0.120, and significant at the ten-percent level. Column (2) is the same as column (1), except that it includes additional state-level control variables (see Table 3 notes). The Muslim assignment effect and the interaction effects are similar to column (1). Column (3) is the same as Column (2) except that is adds firm-and job-level controls to the regressions (see Table 3 notes). 46 These results are, again, consistent with the results of Columns (1) and (2).

Regression Analysis.
Column (4) is identical to Column (1), except that it uses county-level measures with state fixedeffects. 47 The effect of the Muslim assignment dummy in the default category -namely, Republican counties where the median age is higher and the fraction Muslim lower -is negative and significant at the one-percent level. The interactions between the Muslim assignment dummy and the politically mixed and Democratic counties are positive and significant at the ten-and five-percent levels, respectively. The interaction between a low median age and the Muslim assignment dummy is positive and significant at the five-percent level.
Column (5) is the same as Column (4), except that it includes additional county-level control variables analogous to the state-level controls that were included in Column (2) (see Table 3 notes). Column (6) is the same as Column (5), except that it adds the firm level controls that were included in Column (3). The results in Columns (5) and (6) are consistent with the Column (4) results just summarized. Furthermore, looking 45 Table 3 presents robust standard errors with no clustering. The results are robust with respect to different specifications of the model, as discussed in Section 5.2.3 (see Appendix Table A16 for probit estimates and Appendix Table A17 for OLS estimates with clustered standard errors). 46 Fewer than 500 employees is a widely used cutoff standard for identifying small firms in U.S. Small Business Administration programs. 47 The county-level sample is smaller because we were unable to identify counties for all of our observations. We lose a few additional observations in the county-level regressions because of missing county-level demographic data.
across Columns (4)-(6), one can see, in rows 2 and 3, that the callback rate for the Christian candidate is slightly lower in politically mixed and Democratic counties than it is in Republican counties. Table 4 presents regression results for our sexual orientation manipulation. In contrast to the results of the religion manipulation, there are no significant interactions between the treatment assignment and interacted geographical areas.
In short, the regression analysis highlights results that are robust to a variety of specifications. The results in Tables 2, 3, and 4 are consistent with prior results that were suggestive of similar patterns but not definitive. First, they are consistent with our online pilot experiment (see Section 5.1): the online pilot showed significant bias against the Muslim candidate relative to the Christian candidates among subjects with hiring experience, but no bias against the gay candidate relative to the straight candidate (see Appendix A). Second, in a Republican area, Gift and Gift (2014) find evidence (significant at the one-percent level in a one-tailed test) of discrimination against job candidates who signal Democratic Party affiliation on résumés compared to those who signal Republican Party affiliation. In a Democratic area, there is a directionally consistent bias against job candidates who signal Republican Party affiliation, but it is not significant at the five-percent level in a one-tailed test. This is consistent with our finding of stronger discrimination in Republican areas. In Section 5.2.4, we consider a theory that explains why bias is stronger in some places, and derive testable predictions about where to expect stronger bias.
The lack of evidence of bias against the gay candidate differs from the results reported by Tilcsik (2011), who found a significant effect of sexual orientation on callbacks. Rather, our finding is consistent with more recent results presented by Bailey, Wallace, and Wright (2013), who do not find evidence of discrimination against gay men or lesbians. Evolving attitudes toward gay people may explain the differences between our results and Tilcsik's (2011), whose experiment was conducted in 2005. According to our analysis of General Social Survey data, 48 acceptance of gay marriage increased among self-identified Republican, Independent, and Democratic respondents from 2006 to 2012.

Robustness checks
Despite having a binary dependent variable, we used OLS in our analysis above because of concerns about interaction effects in probit regressions (Ai and Norton 2003). Appendix Table A16 is identical to  Appendix Table A19 presents regression results using the Gallup list (Column 1) and Combined list (Column 2) definitions of Republican, Democratic, and politically mixed states. Columns (1) and (2) (3) and (4) show the smallest and largest interactions with the Democratic states that we could find by deleting each of the 50 states (including Washington, DC), one regression at a time. Column (3) removes California from the sample and column (4) removes Idaho. The results remain robust with respect to dropping one state at a time. Even the smallest interaction with the Democratic states dummy is significant at the five-percent level. Columns (5) and (6) are the same as column (2) except that they add, respectively, the additional controls included in columns (2) and (3) of Table 3.
Again, the results are robust with respect to the inclusion of these additional controls.

Limitations
A number of limitations are inherent to the experiment's design. First, a null effect in the Gay-Straight manipulation may not necessarily be interpreted as an absence of discrimination, because a number of potential explanations may exist. Second, heterogeneity in the assignment effect according to firm characteristics cannot be separated into firms' differences in search rates and differences in the treatment effect. Third, the results stem from candidates with certain characteristics, applying to certain types of jobs, 50 Gallup produced this list of states from a sample of 321,233 respondents surveyed by Gallup Daily tracking over the course of the year at the rate of 1,000 respondents per day.
with certain types of employers; they may not apply to other types of candidates, positions, or organizations.
Fourth, we measure employers' traits with state or county level data; we do not capture the traits of the actual employers or human resource decision makers. Finally, discrimination based on information posted online may stem partly from the signal that the posted information sends about a candidate's judgment in choosing to post that information. However, the disclosures we investigated in this paper took place in personal profiles that employers must actively seek, rather than in résumés in which candidates intentionally reveal potentially sensitive information to employers, and the consequences of the fact that people use online social networks to disclose this information in this way, is part of what we sought to understand. Our experimental approach -namely, designing profiles to replicate information found in real social media profiles of individuals demographically comparable to our candidates (see Section 4.2) -increased the realism and ecological validity of our experimental design, and decreased the chances that employers may interpret the profiles as "over-sharing" beyond the levels that we observe in real online social network profiles.

Conclusion
We set out to answer the question: do hiring decision makers seek online information which should not be used in the hiring process and are they affected by what they find? We focused on the effects of religious affiliation and sexual orientation. We used randomized experiments to answer those questions, capturing discrimination both in behaviors of real employers and attitudes of survey subjects. Our results were broadly consistent across the online pilot and the field experiment. They suggest that while hiring discrimination via Internet searches and social media may not be widespread for the companies and jobs we considered in this experiment, revealing certain traits online may have a significant effect on the hiring behavior of selfselected employers who search for the candidates online.
The findings provide further evidence of a phenomenon increasingly studied in the information systems literature: the impact of online IT services on offline outcomes and behaviors. They also highlight the novel tensions arising between regulatory interventions designed in a non-digital world (that attempt to protect personal information), and technological innovations that bypass those protections (by making said information otherwise available).
This work suggests various directions for future research. There is a tension between a potentially efficiency-enhancing effect of online information on labor market search and a potential for that same information to increase labor market discrimination. To the extent that online markets facilitate not just matching but also discrimination, people in disadvantaged categories may face a dilemma: if they censor their online activity, they may be able to protect their privacy, but a limited online presence might, by itself, signal that there is something to hide, or that something is missing. Imagine a charismatic Muslim candidate with both a strong social network in his religious community and many professional contacts. If he were to reduce his online presence to hide his friendships in his religious community, at least three problems arise. .1284 [257] There are no significant differences in callback rates shown in Columns (4) and (5). 0.004 0.007 0.042 0.030 0.038 0.073 * p < 0.10, ** p < 0.05, *** p < 0.01. Dep. Var. =1 if the candidate is contacted for an interview, zero otherwise. Numbers reported are OLS coefficients (robust standard errors in parentheses). Additional statelevel geographical controls included in column 2 and 3 are: 2012 unemployment rate, fraction foreign-born, natural log of median income, fraction non-white, fraction college educated or more, fraction evangelical Christian, fraction urban, Facebook penetration, and legal protection from religious discrimination. Additional county-level geographical controls in col. 5 and 6 are: 2012 U.E. rate, fraction foreign-born, natural log of median income, fraction non-white, fraction college educated or more, fraction evangelical Christian, and rural-urban continuum code dummies. Firm-level controls included in columns 3 and 6 are: dummies for women and minority owned, public firm, large firm (500 employees or more), federal contractor, and included application-level characteristics are dummies for entry level position, references required, preferred salary required, master's degree required, one or more years of experience required, multiple fields of experience required, and 9 dummies for field of employment. The continuous state and county variables -namely, 2012 unemployment rate, fraction foreign-born, natural log of median income, fraction non-white, fraction college educated or more, fraction evangelical Christian, fraction urban, and Facebook penetration -are centered on their means. The omitted categories for dummies capturing variables with more than two categories are: Republican states or counties; counties with less than 20,000 people; and jobs in information systems. The remaining variables are binary. Full regression results are available in the Appendix.  Table 2, with the exception that columns 2 and 3 control for legal protection from sexual orientation discrimination, instead of legal protection from religious discrimination. Full regression results are available in the Appendix.