Transcription

Revised 07-12-2016Ambiguity in Privacy Policiesand the Impact of Regulation45 Journal of Legal Studies (forthcoming) Joel R. Reidenberg, Jaspreet Bhatia, Travis D. Breaux, Thomas B. NortonAbstractWebsite privacy policies often contain ambiguous language that undermines the purpose andvalue of privacy notices for site users. This paper compares the impact of different regulatorymodels on the ambiguity of privacy policies in multiple online sectors. First, the paper developsa theory of vague and ambiguous terms. Next, the paper develops a scoring method to comparethe relative vagueness of different privacy policies. Then, the theory and scoring are appliedusing natural language processing to rate a set of policies. The ratings are compared againsttwo benchmarks to show whether government-mandated privacy disclosures result in noticesless ambiguous than those emerging from the market. The methodology and technical tools canprovide companies with mechanisms to improve drafting, enable regulators to easily identifypoor privacy policies and empower regulators to more effectively target enforcement actions.* Respectively,Stanley D. and Nikki Waxberg Professor of Law, Fordham University andVisiting Research Collaborator, CITP, Princeton; Ph.D. candidate, Carnegie MellonUniversity; Assistant Professor of Computer Science, Carnegie Mellon University; PrivacyFellow, Fordham CLIP. Work on this project was supported in part by NSF Awards#1330596 and # 1330214, the Carnegie Mellon Requirements Engineering Laboratory, anda Fordham Faculty Fellowship. The authors would like to thank Stephen Broomell for hisadvice on testing validity and Stephanie Tallering for her assistance with coding. Theauthors benefited from very helpful comments on earlier drafts by participants at PLSC2015 and the University of Chicago Coase-Sandor Institute Conference on “Contracting overPrivacy.”1

Revised 07-12-2016TABLE OF CONTENTS1.Introduction . 22.Defining and Measuring Ambiguity . 32.1 Taxonomy of Vague and Ambiguous Terms through Grounded Theory . 32.2 Vague Terms . 82.3 Comparative Levels of Ambiguity . 92.4 Ambiguity through Incompleteness . 123.Scoring Vagueness . 123.1 The Landscape of Vagueness in Privacy Policies. 133.2 The Scoring Model . 134.Comparative Scores and the Impact of Regulation . 154.1 Company Scores for Unregulated Disclosures . 154.2 Scores for Regulated Disclosures . 174.3 Normative Role of Privacy Notice Regulation . 205. Public Policy Considerations: Technological Tools, Linguistic Guidelines andReporting . 215.1 Technical Tools . 215.2 Linguistic Guidelines . 225.2 Reporting Framework . 236.CONCLUSION . 231. INTRODUCTIONPrivacy policies often contain ambiguous language describing website practices fordata processing activities such as collection, use, sharing, and retention. Whilescholars have shown weaknesses in the readability of privacy policies (McDonald andCranor 2008; Pollach 2007; Jenson and Potts 2004) and weaknesses in thesubstantive protections (Marrotta-Wugler 2015; Pollach 2007), they have notfocused carefully on policy ambiguity. Ambiguity regarding these practicesundermines the purpose and value of a privacy policy for website users. Withoutclear affirmative statements, privacy policies are, in effect, meaningless. They wouldconvey no true indication to users of the website’s actual practices and they wouldprovide declarations that would be unenforceable. On a practical level, ambiguity2

Revised 07-12-2016also challenges the usability of privacy technologies for user empowerment: clarity inprivacy practices is a necessary prerequisite to empowering users to make informeddecisions.1This paper will explore the problem of ambiguity in policy language. In Part II, wedevelop a theory for the definition of ambiguous terms and for the measurement ofsuch terms. In Part III, we develop a scoring method to compare the relativevagueness of different privacy policies. In Part IV, we apply the theory and methodusing natural language processing (NLP) techniques to score a set of privacy policiesfor clarity and comparison. We then use these comparative rankings to examinewhether regulation improves the clarity of privacy policies. To test the impact ofregulation, we compare the ambiguity of policy language under three conditions: 1)no privacy regulation; 2) regulation under Gramm-Leach-Bliley Act; and 3) regulationunder the US-EU Safe Harbor inter-governmental agreement. The results providenormative insight on the role of privacy notice regulation. In Part V, we address anumber of practical public policy considerations resulting from our scoring. Thetechniques and corresponding technical tools can provide companies with a usefulmechanism to improve the drafting of their policies. At the same time, automatedtools embodying our theory and scoring method will enable regulators to easily scanindustries and companies for poor language in their privacy policies. Suchinexpensive scans revealing problems with privacy policy language then empowersregulators to more effectively target defective privacy policies for remedial action.2. DEFINING AND MEASURING AMBIGUITY2.1 Taxonomy of Vague and Ambiguous Terms through GroundedTheoryAmbiguity arise when a statement is incomplete and missing relevant information, orwhen a word or phrase has more than one possible interpretation and the reader isuncertain about which interpretation the author intended. Linguists often addressvagueness as a form of ambiguity. (Massey 2014) In contract theory, vaguenessconnotes a distribution around a norm without a clear delineation while ambiguityrefers to situations where a word may have at least two meanings. (Farnsworth 1999§ 7.8 ) In each case, multiple interpretations can arise when a statement is incomplete,or when a generic word or phrase is used in place of a more specific word or phrase.When a website privacy policy uses vague or ambiguous terms, the language choices1For example, the joint Carnegie Mellon University, Fordham University and StanfordUniversity usable privacy project seeks to combine crowd sourcing, natural languageprocessing and machine learning to develop browser plug-in technologies that willautomatically interpret privacy policies for users. (Usable Privacy Project, 2016) If policiesare too ambiguous, automated processing will be frustrated.3

Revised 07-12-2016dilute the ability of a policy to describe the website’s actual practices. This study will,however, focus on vagueness where terminology lacks specificity or context.Because privacy policies summarize an organization’s data practices, it is notsurprising that policies include vagueness. There are at least two motivations forintroducing vagueness: (1) the practices include divergent or separate situationswhere actions do and do not occur, in which case the action “may” occur, dependingon what situation the individual encounters; and (2) there are foreseeable, yetunrealized actions that “may” occur in the future, and the policy authors wish to beflexible to accommodate those future actions without changing the policy. In the caseof the first motivation, we believe changes to some policy statements can clarify underwhat situations the action does or does not occur, resulting in a less vague policy.However, the second motivation to accommodate flexibility is at best a form ofinaccuracy whether the result of hedging to cover unknown existing internalpractices or unknown changes and at worst misleading and misrepresentative. 2To demonstrate this effect, we show a few illustrative examples from the Barnes &Noble privacy policy (2013) concerning personal information that are commonlyfound in other policies. The Barnes & Noble policy includes two statements thatdescribe the possibility of collection:(1) “Depending on how you choose to interact with the Barnes & Noble enterprise,we may collect personal information from you . . . .”(2) “We may collect personal information and other information about you frombusiness partners, contractors and other third parties.”In statement (1), the collection is conditioned upon how the user interacts with thecompany. This is vague, because the statement summarizes multiple situations, someof which will include the collection of personal information and some of which willnot. To achieve clarity, it would be reasonable to exclude those situations wherepersonal information is not collected, and to focus on where personal information“will be collected.” In contrast, statement (2) is vague because the conditionalsituations are not described, thus all third-party transactions are summarized into asingle statement. By separating these statements and iterating over the differentcategories, the policy authors can exclude prospective collections (envisioned, but notactual collections) and those situations where personal information is not collected.Another attribute of vagueness concerns the vague conditions and purposes underwhich information is used. In statement (3), below, the Barnes & Noble policy linksthe collection to a broad purpose (improving customer experience) under a generalThe desire for flexibility might also be seen as a reservation by the website to holdan option on the ability to engage in unstated data practices. Even if the languageexplicitly describes an option, the terminology still encompasses alternativemeanings and the user will not know the subjective intent of the website, thus,creating vagueness.24

Revised 07-12-2016assumption about the “necessary” situations that define this broad purpose. Analternative statement would replace the phrase “as necessary” with specific purposesintended to improve customer service.(3) “We collect your personal information in an effort to provide you with asuperior customer experience and, as necessary, to administer ourbusiness.”Vagueness pertaining to standard third-party transactions is also evident in thefollowing two statements from the Barnes & Noble privacy policy:(4) “In addition, we disclose certain personal information to the issuer of theMasterCard . . . .”(5) “If you are accessing our goods and services using a Microsoft account,Microsoft may share your personal information with us . . . .”In statements (4) and (5), the mechanisms for exchanging personal information arecoded in software. In the case of credit card transactions, MasterCard, card issuersand acquiring banks each have transaction processing rules that are updated fromtime to time, but the technical specifications of their respective electronic paymentprocessing infrastructures are less likely to change. The Barnes & Noble policy couldthus restrict the kind of personal information it shares to payment information orinformation for the purpose of completing a purchase. 3 In statement (5), Microsoft’sLive Connect API for OAuth 2.0 access to the Microsoft account is also very explicitabout what information “may” be shared (first and last name, email address, gender,age) and Barnes & Noble can further commit to which of these information types they“will” collect as encoded by their software.As the case study reveals, the contours of vagueness are very complex. Themeasurement of vagueness, thus, becomes a valuable marker to signal whether aprivacy policy is a meaningful notice of a website’s actual policies and practices and anotice that might give rise to a contractual commitment. The first step in themeasurement of vagueness in privacy policies is the development of a rigorous andvalidated taxonomy of terms that can be used to examine a diverse set of onlinesectors such as shopping, news and financial services.Linguistic scholars have identified various forms of ambiguity in the use of languageand have classified textual ambiguity in various ways. (Hoffman et al. 2013; Masseyet al. 2014; Pollack 2007). Some of these classifications reflect terms may have3Statement 4 is also separately confusing because Barnes & Noble would not typically be ableto share data directly with a card issuing bank. Barnes & Noble exchanges data from the pointof sale to its acquiring bank which in turn shares information specified by MasterCard to thecustomer’s card issuing bank through the MasterCard network. The circumstance that mightgive rise to direct sharing from Barnes & Noble to a card issuing bank would be the case of aco-branded card where Barnes & Noble would have a direct relationship with the card issuer.5

Revised 07-12-2016inherent vagueness. (Massey 2014) For example, many privacy policies use the modalverb “might” to describe data processing activities (“we might collect . . . ”) that mayor may not occur in the future. In addition, policies use conditional phrases, such as“when”, “upon”, and “during”, that indicate an event upon which a particularstatement becomes true (“upon consent, we will share . . . ”). When multiple modalverbs and conditional terms are used together, readers struggle to actually determineif the described practices occur, or in what combination, or under which specificconditions or how to satisfy those conditions.For a rigorous analysis of textual ambiguity, the starting point is, thus, theestablishment of a typology of ambiguous terms. Since our objective is to provide aqualitative rating of vagueness, we have chosen to focus on this narrower aspect ofambiguity. This means that our scoring will provide relational comparability, butunderrate overall ambiguity.We define our typology based on grounded theory (Glaser and Strauss 1999) toidentify vague terms and classify those terms into categories. Three researchersmanually performed this analysis using coding (Saldana 2012) to examine a set ofpolicies across a variety of sectors. (Bhatia et al. 2016a). Five policies were used forthe initial identification and classification of relevant terms. (Bhatia et al. 2016a)The analysis resulted in a taxonomy with four categories as shown in Table 1. Froma legal perspective, conditional terms are inherently vague because the performanceof a stated action or activity will be dependent on a variable trigger. (Farnsworth1999 § 8.2) Similarly, generalizations are terms that vaguely abstract informationpractices using contexts that are unclear (e.g. “typically” or “generally”). From thelinguistic perspective, modality (modal verbs, adverbs and non-specific adjectives)creates uncertainty with respect to actual action (von Fintel 2006); this includeswhether an action is possible, likely, permitted or obligatory, among others. If theaction is only permitted, it may never occur, whereas obligatory actions are expectedto occur in the future (the difference between “we may” and “we will”). Similarly,numeric quantifiers that are non-specific create ambiguity as to the actual measure.To assure the completeness of the typology, three researchers reviewed 15 policiesfirst in their entirety and then statement by statement to identify vague phrases anddetermine if they fit these categories or if new categories were required.4 (Appendix,Table A1)This also included an evaluation of cases where the language might appear as a borderlineclassification.46

Revised 07-12-2016Table 1Categories of Vague TermsDescriptionCategoryAction(s) to be performed are dependent on a variableor unclear triggerAction(s)/Information Types are vaguely abstractedGeneralizationwith unclear conditionsModalilty (including Vague likelihood of action(s) or ambiguous possibilitymodal verbs)of action or eventConditionNumeric quantifierVague quantifier of action/information typeTo see how a sentence may reflect these categories, the phrase “we generally mayshare personal information we collect on the Site with certain service providers, someof whom may use the information for their own purposes as necessary” contains acondition, generalization, modal verbs and numeric quantifiers. 5 These vague termsare annotated in the sentence as shown in Figure 1:Figure 1Sentence AnnotationIn combination, these six forms of vagueness combine to allow any organizationsharing personal information under this statement to share it with anyone for anyThe original sentence was extended to include “generally” and “as necessary” for illustrationpurposes; the sentence without these additions is found in Lowe’s website privacy policy onApril 27, 2015 at: http://www.lowes.com/en us/l/privacy-and-security-statement.html.57

Revised 07-12-2016purpose, as long as the recipient is a service provider. The combination of these sixterms further leaves unclear the conditions under which information is shared, andthe number or proportion of service providers that engage in this practice.2.2 Vague TermsTo complete the lexicon of vague terms, we used an established coding frame basedon the taxonomy in Table 1 and three researchers analyzed a set of 15 policies.(Appendix, Table A1). To assure saturation of terms, we examined three diversesectors (shopping, telecommunications and employment) and five policies withineach sector reflecting a diversity of types of websites within each category. 6 We choseprivacy policies of major sites that are visited by large numbers of users. 7 In thisstudy, we reached saturation after analyzing 5 policies (Barnes & Noble, Lowes,Costco, AT&T, and Comcast) reflecting that our taxonomy of terms was complete forthe privacy policy domain.The resulting set of terms for the taxonomy is shown in Table 2.Table 2Results from Applying Taxonomy to Privacy PoliciesCategoryConditionGeneralizationKey Words and e,inappropriate, as needed, as applicable, 7.20%otherwise reasonably, sometimes, from timeto timegenerally, mostly, widely, general, commonly, 3.63%usually, normally, typically, largely, often,primarily, among other thingsModality (including may, might, can, could, would, likely, possible, 70.60%modal verbs)possiblyNumeric quantifieranyone, certain, everyone, numerous, some,most, few, much, many, various, including but 18.60%not limited to*The distribution represents the number of vague terms in the 15 policies belonging to thecategory divided by the total number of vague terms in 15 policies. See Bhatia et al. (2016a).We considered examining policies from top site rankings, but the various rankings did notassure diversity of sectors.7 The small number of policies in each category preclude broad generalizations within andacross categories, but do enable us to show the value of a score for comparison purposes,including comparison against the financial services benchmark68

Revised 07-12-20162.3 Comparative Levels of AmbiguityWhile the taxonomy results in Table 2 present the terms that obscure the clarity ofthe policy descriptions, the taxonomy does not address the relative levels ofambiguity among the terms. For example, the following two statements appear tohave different levels of ambiguity:1) “We may generally collect ”2) “We may collect as necessary ”Each uses a modal verb (“may”), but the first statement containing the generalization“generally” seems less clear than the second statement containing the condition “asnecessary.” The practice described “as necessary” suggests that collection will onlyoccur in exceptional cases while “generally” suggests that collection is likely to occurunder broader circumstances. This qualitative difference in clarity may also be linkedto the degree of flexibility that the textual language provides to the website. Languagedesigned to give websites greater flexibility is likely to be perceived as moreambiguous. The statement “may collect generally” provides greater flexibility to thewebsite than “as necessary.” Consequently, the generalization term “generally”obscures for the user the website’s activities more than the conditional term “asnecessary.”In addition to variations in clarity among the categories, the combination of termsfrom different categories in the same sentence may also affect the level of ambiguityperceived in descriptions of privacy practices. Figure 2 shows our initial hypothesisregarding the possible cumulative effect of vague terms represented as a lattice.Figure 2Combinations of Terms9

Revised 07-12-2016The lattice begins with a modal statement “we may collect” and then in the next rowadds a term from each of the remaining three different categories: a generalizationterm “generally,” a conditional term “as needed” and a numeric quantifying term“some.” Our initial assumption was that additional terms would increase thevagueness of the statement, i.e. reduce the clarity of the description of the datacollection practice. With each successive combination of vague terms, from the firstto the second, third and fourth rows in Figure, vagueness would increase until somedegree of saturation would occur (i.e., adding additional vague terms would have nosignificant impact on increasing vagueness).The relative impact of each possible combination is critical to the development of anaccurate score for a privacy policy’s ambiguity. To evaluate our initial assumption,we conducted a paired comparison survey.8 The survey results show the relationshipof combinations of terms on the level of ambiguity, enables the assignment of relativeweights to different combinations of terms from one or more categories. We used theBradley-Terry model that scales preferences among different pair comparisons tocalculate the weights from the paired comparison data. (Turner and Firth. 2012)These results (Bhatia et al. 2016a) are presented in Figure 3 and Appendix Table A2.Figure 3 shows the Bradley-Terry coefficients for the combinations of conditions (C),generalizations (G), modal terms (M), and numeric quantifiers (N).Figure 3Bradley-Terry Coefficientsfor inter-category combinationsA paired comparison survey is a standard statistical technique that collects multiplepreferences between two statements from multiple judges and, through the aggregation ofthe results, establish a matrix of rating comparisons for all possible combinations of the termsbeing studied.810

Revised 07-12-2016These results show the quantity that each combination of vague terms contributes tothe overall concept of vagueness in the survey: that data practices described withcombinations at the left of the chart (CN, C, CM, ) have greater clarity than thosepractices described with combinations at the right of the chart (GMN, G, GM, ) Whilephrases with both a conditional term and a vague numeric quantifier (CN) areindistinguishably clear from phrases with just a conditional term alone (C), we canobserve how the vagueness taxonomy influences overall vagueness. The arrowmoving left in the chart shows that condition terms increase clarity and reducevagueness: e.g., statements with both a modal term and numerical quantifier (MN)are significantly more vague than similar statements with the addition of aconditional term (CMN). The arrow moving right in the chart illustrates howgeneralizations significantly increase vagueness: e.g., the MN statements with theaddition of a generalization (GMN) are significantly more vague. By comparison,statements with a generalization and modal term (GM) are twice as vague asstatements with a condition and a modal term (CM).The results in Figure 3 present the inter-category vagueness. To measure the intracategory vagueness between terms within each of the categories, we conductedadditional surveys. (Bhatia et al., 2016a). The survey results indicate that termswithin each category have different levels of vagueness. For example, the intracategory vagueness results for the “Generalization” category are presented in Figure4 and the results for all categories appear in Appendix Table A3.Figure 4Bradley-Terry Coefficientsfor “Generalization” termsThe results in Figure 4 show that within the “Generalization” category, vaguenessappears to increase as the adverbs transition from the routine (e.g., typical, normal11

Revised 07-12-2016or usual) to the unrestricted (e.g., widely, largely, mostly). The full results inAppendix Table A3 show similar differentials. Within the “Conditional” category,the term “as appropriate” was several times more vague than the term “asnecessary.” In the “Modal” category, the past tense verbs “might” and “could” areperceived to be more vague than the present tense variants “may” and “can,”respectively. These three observations led Bhatia et al. to conjecture thatvagueness increases along three dimensions: authority, wherein discretionarypractices are perceived to be more vague than mandatory practices (e.g., “asappropriate” is permissive, whereas “as necessary” is obligatory); certainty, which isthe absoluteness with which practices are performed (e.g., “typical” is certain withrespect to common cases, whereas “widely” is blurs the boundary between commonand exceptional cases); and likelihood, which is the possibility that the practice isperformed (e.g., “likely” is more likely than “possibly,” and thus less vague).2.4 Ambiguity through IncompletenessLastly, silence in a privacy policy can often introduce ambiguity. (Marotta-Wurgler2015). For example, if the policy is silent on sharing data with third parties, then thepolicy fails to convey whether and under what conditions data may be transferred toothers. As a result, completeness of the privacy policy will have an impact on thescoring of ambiguity. While there are no legal requirements spelling out all the termsthat must be contained in a privacy policy, various templates might be used todetermine completeness. For example, the Gramm-Leach-Bliley Act only stipulatesthat financial service companies provide notice to customers of their privacy policiesand that the notice at a minimum contain certain types of disclosures. (GrammLeach-Bliley Act, 15 U.S.C. § 6803 (1999). The Federal Trade Commission and theDepartment of Commerce have each articulated several sets of fair informationpractices. (FTC 1998, 2000; Department of Commerce 2000) For purposes of thisanalysis, the existence of four elements will be inventoried: collection, retention,sharing and use. These elements reflect the most significant privacy harmsdemonstrated through litigation that we believe can be resolved by unambiguousprivacy policy statements. (Reidenberg et al 2015)3. SCORING VAGUENESSWith the vagueness taxonomy populated using key words and phrases correspondingto each category, a comparative classification and a completeness indicator can beconstructed to score the degree of affirmation or certainty associated with datapractices for specific types of personal information. Privacy policy statements aboutcompanies that “might collect” are less certain than statements that they “will” or“will not collect” a particular information type. Highly uncertain statements can moreeasily accommodate a company’s future practices, thus providing these companiesmore flexibility in the interim to alter those practices. However, highly uncertainstatements allow for interpretations that may be untrue, thus giving users a false12

Revised 07-12-2016sense of privacy. By contrast, if an organization has a policy that is more certain,particularly with more restrictive practices, any new unstated practices wouldrequire a change in the policy. Such changes would trigger opportunities for users tore-evaluate their relationship with those companies under the new practices. Thisopportunity to evaluate policy changes is necessary if the privacy principle of userconsent is to have any meaning. Policies containing more certain statements aremore likely to increase the opportunity for choice, since those policies will need to berevised each time a new practice is to be covered.To score privacy policies, the first step is to determine if vague terms are common inprivacy policies through an analysis of the landscape of terms found in privacypolicies. The frequent existence of vague terms leads to the definition of a scoringmodel that can then be applied to privacy policies to rank their vagueness againsteach other.3.1 The Landscape of Vagueness in Privacy PoliciesWhen the taxonomy is applied to the set of 15 privacy policies (Appendix Table A1),every policy in the data set contains vague terms. (Bhatia et al. 2016a). As shown inBhati

of sale to its acquiring bank which in turn shares information specified by MasterCard to the customer's card issuing bank through the MasterCard network. The circumstance that might give rise to direct sharing from Barnes & Noble to a card issuing bank would be the case of a