Alexa, Are You Listening? An Exploration of Smart Voice Assistant Use and Privacy in Libraries ARTICLE Alexa, Are You Listening? An Exploration of Smart Voice Assistant Use and Privacy in Libraries Miriam E. Sweeney and Emma Davis INFORMATION TECHNOLOGY AND LIBRARIES | DECEMBER 2020 https://doi.org/10.6017/ital.v39i4.12363 Miriam E. Sweeney (mesweeney1@ua.edu) is Associate Professor, University of Alabama. Emma Davis (edavispatsfan@gmail.com) is Library Specialist, Hoover Public Library. ABSTRACT Smart voice assistants have expanded from personal use in the home to applications in public services and educational spaces. The library and information science (LIS) trade literature suggests that libraries are part of this trend, however there are few empirical studies that explore how libraries are implementing smart voice assistants in their services, and how these libraries are mitigating the potential patron data privacy issues posed by these technologies. This study fills this gap by reporting on the results of a national survey that documents how libraries are integrating voice assistant technologies (e.g., Amazon Echo, Google Home) into their services, programming, and checkout programs. The survey also surfaces some of the key privacy concerns of library workers in regard to implementing voice assistants in library services. We find that although voice assistant use might not be mainstreamed in library services in high numbers (yet), libraries are clearly experimenting with (and having internal conversations with their staff about) using these technologies. The responses to our survey indicate that library workers have many savvy privacy concerns about the use of voice assistants in library services that are critical to address in advance of library institutions riding the wave of emerging technology adoption. This research has important implications for developing library practices, policies, and education opportunities that place patron privacy as a central part of digital literacy in an information landscape characterized by ubiquitous smart surveillant technologies. INTRODUCTION Smart voice assistant use has expanded from personal uses in the home to new applications in customer services, healthcare, e-government, and educational spaces, raising questions from groups like the American Civil Liberties Union (ACLU), among others, about the data privacy implications of these technologies in public and shared spaces.1 Libraries are part of the voice assistant adoption trend, as documented in the American Libraries Magazine article “Your Library Needs to Speak to You” by Carrie Smith.2 Smith gives examples of school, public, and academic libraries adopting smart voice assistants like Amazon’s Alexa and Echo devices for a range of services and programming including “event calendars, catalog searches, holds, and advocacy.” Nicole Hennig points out that there are tremendous opportunities for voice assistants to assist “people with disabilities, the elderly, and people who can’t easily type.”3 In these ways, voice assistants are often presented in the trade literature as part of an exciting new wave of emerging smart technology services that libraries can “get ahead of” and potentially harness for public service and community engagement. At the same time, the key privacy issues inherent in voice assistants are often downplayed as secondary concerns while librarians are encouraged to press forward and experiment with smart technology adoption. We argue that the privacy concerns surrounding voice assistant use in libraries should be treated as fundamental questions for library mailto:mesweeney1@ua.edu mailto:edavispatsfan@gmail.com INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 ALEXA, ARE YOU LISTENING? | SWEENEY AND DAVIS 2 workers to consider as a part of upholding the core values of patron privacy and confidentiality in library services. Voice assistant use in libraries is still nascent, reflecting the emerging nature of these technologies. Given this, it is not surprising that very few empirical studies have explored voice assistant use and potential data privacy implications for libraries. Our research is intended as an exploratory study that contributes to advancing knowledge in this area. The goals of this study are to begin mapping smart voice assistant use in libraries, to assess how aware library workers are of privacy concerns involving these technologies, and document how library workers are educating patrons about privacy and voice assistant use. These are necessary first steps for developing library practices, policies, and education opportunities for voice assistant use that prioritize privacy as a central part of digital literacy in an information landscape characterized by ubiquitous smart surveillant technologies and diminishing data privacy protections. REVIEW OF LITERATURE What is a Voice Assistant? Voice assistants are a type of digital assistant technology, also known as virtual assistants, and can be broadly defined as computer programs designed with human characteristics that act on behalf of users in digital environments using voice interfaces.4 Apple’s Siri, Microsoft’s Cortana, and Amazon’s Alexa are prevalent examples of smart digital assistants that use voice recognition and natural language user interfacing to help learn users’ preferences, answer questions, and manage a variety of applications and personal information. Voice assistants can run on multiple devices and be seamlessly integrated across platforms including networked Internet of Things (IoT) gadgets like smart speakers (e.g., Amazon Echo and Google Home) and other smart-home technologies (e.g., Nest or Ring), along with mobile devices, smart watches, personal computers, and numerous third-party applications. Ubiquitous “always on” features are offered as a convenience to users who can use “wake words” (e.g., “Hey, Siri”; “Alexa”; “OK Google”) to initiate queries and commands. Amazon’s smart speakers and intelligent digital assistants are rapidly becoming pervasive home and personal technologies, with the Amazon Echo leading the market in 2019 with 61 percent market share, followed distantly by the Google Home device with 24 percent market share.5 A recent United States survey by Clutch reported that nearly half of people surveyed owned a voice assistant, with one-third planning to purchase one in the next three years.6 Additionally, the Clutch survey found that 69 percent of voice assistant owners used their devices every day.7 The popularity of voice assistants for personal use has driven the expansion of these technologies for customer service applications outside of the home in shared and public spaces, including in educational settings and health care. In this landscape it is perhaps not surprising that librarians are following suit and exploring the service potentials of voice assistants for libraries. Libraries and Voice Assistant Use The American Library Association’s (ALA) Center for the Future of Libraries initiative identified “voice control” as a trend in their 2017 report, anticipating the relevance of voice assistant technologies for libraries.8 The capability of voice assistants to integrate across platforms through customized applications—which Amazon calls “skills” and Google refers to as “actions”—allows libraries to create specialized uses for these technologies as a part of their regular information services. Additionally, existing third-party vendors like Overdrive (for e-book lending) and Hoopla (multimedia lending) that most public libraries use are preconfigured to connect to voice INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 ALEXA, ARE YOU LISTENING? | SWEENEY AND DAVIS 3 assistants like Amazon’s Alexa. There are many creative and potentially helpful ways that voice assistants could be integrated into the library setting, including enhancing read-along with music and effects, providing accessible services for elderly patrons or individuals with disabilities, and providing an alternative access point for common library queries and institutional information (e.g., searching titles, placing holds, requesting library event information).9 Some libraries have started experimenting with voice assistant services in the library. For example, Iowa State University staff developed Alexa skills for their library so that users could find out information about library history and library collections.10 Other libraries are using voice assistants to strategically engage their communities, as when the Spokane Public Library placed Amazon Echo Dots in the library so patrons could ask questions about upcoming bond elections, an issue that directly impacts library funding.11 The Worthington (OH) Libraries are integrating voice assistant technologies into technology training and “petting zoo kits” which allow their patrons to try out emerging technologies.12 The King County (WA) Library system is taking a novel approach and experimenting with developing their own voice assistant, LIBRO.13 These examples point to the many applications and creative approaches libraries are experimenting with to bring voice assistant technology to their services. Data Privacy Issues As convenient as voice assistants may be for library services, the underlying data infrastructures of these technologies are tightly controlled by the technology companies that design and sell them. The lack of library control (and transparency) over these infrastructures raises questions about how the core values of privacy and confidentiality can be guaranteed in the library setting. 14 Voice assistant technologies capture a wide range of intimate user information in the form of biometric data (e.g., voice recognition), consumer habits, internet-based transactions, personally identifiable information (PII), and geographical information.15 The ubiquitous “always on” feature that makes these technologies so convenient also flags important privacy questions about the extent of user interactions that are recorded; how these files are processed, transcribed, and stored; and how local, state or other law enforcement agencies might compel or otherwise use these records.16 Recently Amazon has confirmed that they have employees dedicated to listening to recordings from Echo devices in order to help “eliminate the gaps in Alexa’s understanding of human speech and help it better respond to commands,” which is concerning for patron privacy in the library context.17 Researchers at Northeastern University and Imperial College London recently did a study about how often smart speakers record “accidentally” and whether or not they are constantly recording. The study found no evidence to support the theory that these devices are constantly recording, however the researchers did report that smart speakers are accidentally activated around 19 times a day, on average. 18 These reports aside, there is still much unknown about what these companies, and the companies they contract out work to, do with the personal data collected from voice assistants. Lastly, Amazon is a known collaborator with US government agencies like Homeland Security and Immigration and Customs Enforcement (ICE), hosting their biometric data on Amazon Web Services (AWS).19 Amazon has a reputation for being one of the least transparent technology companies in terms of data sharing practices, and has routinely evaded questions about if/how much of customers’ Echo data has been turned over to federal authorities.20 Given this data environment, the fact that libraries are beginning to experiment with voice assistant integration in their services poses important questions for patron data privacy and confidentiality. ALA provides library privacy guidelines for third-party vendors that clearly detail INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 ALEXA, ARE YOU LISTENING? | SWEENEY AND DAVIS 4 expectations for use, aggregation, retention, and disclosure of user data. 21 While this document has been helpful for guiding license agreements with digital content providers, program facilitators, and other libraries, it does not quite capture the range of complexities that emerging smart technologies pose in the app-driven IoT landscape. This area is ripe for study and having more information about how libraries of different types are approaching using voice assistants is necessary for developing responsive professional practices that center issues of privacy and critical digital literacy. Our survey explores some of these issues with the purpose of beginning to document voice assistant use, and associated privacy concerns, in library services. RESEARCH METHODS Four main research questions guide this study: (1) How are libraries using smart voice assistant technologies as a part of their library services? (2) How aware are library workers of how voice assistants integrate with third-party digital content platforms? (3) Are libraries educating library patrons about the privacy implications of smart voice assistant technologies? and (4) What kinds of privacy concerns do library workers have about the use of smart voice assistant technologies in their library services and programming? To address these questions, we developed an online survey using Qualtrics web software, and distributed it in fall 2019 to 1,929 public and academic libraries across the US via email solicitation.22 The survey consisted of a mix of 31 multiple choice and open-ended questions designed to address different aspects of the stated research questions (see Appendix A). Since most of the examples of library voice assistant use detailed in the LIS trade literature came from public and academic libraries, these were the library types we identified as most likely to already be experimenting with voice assistants in services and programming. Using purposive sampling techniques, we selected 30 public libraries for each state that represented a range of rural and metropolitan service areas. We selected approximately 10-20 academic libraries per state, the actual numbers ranging based on the total number of universities and colleges in a given state. We identified a cross-section of large state schools, private colleges, and community colleges in each state to account for the variety of higher education institutional settings for academic libraries. We sent email solicitations to each public library, targeting email addresses for library directors where possible. For libraries that had centralized email services, we solicited participation using the contact forms available on the libraries’ websites. Email solicitations to the academic libraries targeted library employees with job titles that included: emerging technology, user services, user experiences, head of public services, and head of technology. Our survey analysis documents the numbers of reported uses, and kinds of integration, of voice assistant technologies across library applications and services. We conducted a qualitative content analysis of the short answer responses, with both researchers independently coding participant comments for emergent themes and categories. As a part of this process both researchers compared and negotiated categories in two iterations of coding to arrive at a common codebook which was then applied in the final pass of the responses. These categories have some distinct features, but also have many overlapping components. Comments that embodied multiple themes were included in all categories that were relevant for describing them, meaning a particular comment might be included in multiple categories. The following sections report on the key findings of this study, organizing the discussion around our original research questions. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 ALEXA, ARE YOU LISTENING? | SWEENEY AND DAVIS 5 FINDINGS Participant Demographics We received 86 total responses for the survey, with the majority of respondents (61 percent) reporting affiliation with public libraries, followed by respondents from academic libraries (38 percent), with one respondent from a school library (1 percent).23 The participants represented libraries from 42 states across the US.24 The vast majority of public library respondents (65 percent) reported serving populations of 25,000 or more, though there was also a large reporting from libraries serving smaller populations of 2,500-9,999. The majority of academic library respondents work for small and medium sized institutions serving populations between 2,500- 9,999 (table 1), with nearly a third of respondents representing medium to large institutions. Admittedly, these are rough demographic sketches to help quickly identify which types of libraries might be using voice assistants. More granular demographic detail would be useful in future studies to further understand how factors like institution type, geographical region, access to resources, and service community demographics shape decisions about emerging technology adoption in libraries. Table 1. Size of service population by library type Total Public Academic School Total Count 84 51 32 1 2,5000 or less 11.9% 2.0% 28.1% 0.0% 2,500-9,999 25.0% 19.6% 34.4% 0.0% 10,000-25,000 16.7% 11.8% 25.0% 0.0% 25,000+ 44.0% 64.7% 12.5% 0.0% I’m not sure. 2.4% 2.0% 0.0% 100.0% How are libraries using smart voice assistant technologies as a part of their library services? Only five respondents (6 percent) in our study reported that their library is currently using Amazon Echo, Google Home, or Apple Siri devices for patron services and programming. Of the voice assistant adopters, three were public libraries using Amazon Echo and Google Home devices, and two were academic libraries using Amazon Echo and Apple Siri (table 2). Table 2. Voice assistant device by library type Total Public Academic School Amazon Echo 3 1 2 0 Google Home 2 2 0 0 Apple Siri 1 0 1 0 Librarians described using voice assistants to “provide basic info about the library and resources,” and on an “ad hoc basis” to promote the library-specific Alexa Skills and Google Home Actions. Other reported uses included “translation services” and as a part of “technology petting zoos.”25 We asked librarians to describe where voice assistants were located in the library to get a better idea of the spatial arrangements of these technologies, which could be important for considering potential surveillant concerns. Several libraries reported that they had voice assistants sitting at front service desks or reference desks for patrons to use in both adult and children’s service areas, INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 ALEXA, ARE YOU LISTENING? | SWEENEY AND DAVIS 6 as well as at circulation desks. As one librarian described, “We are mounting it [the voice assistant] so students/users can ask questions when necessary.” When it comes to using these devices in library programming, the most common response was for use in technology petting zoos and in technology classes where patrons can see technology demonstrations and ask library staff questions, or get on-on-one tutoring sessions: “Our Technology department holds regular ‘Tech Drop-In's’ and carries out one on one assistance by appointment. In the context of these patrons will sometimes bring in their own devices or ask questions about the use of digital assistants.” Other programming applications that librarians mentioned for voice assistants included trivia, 3 -D printing, and makerspaces. Two libraries (one public and one academic) reported that they were circulating Apple Siri devices (e.g., iPads) and Amazon Alexa products (e.g., Echo) for checkout. How aware are library workers of how voice assistants integrate with third-party digital content platforms? The majority of library workers surveyed (70 percent) reported that their libraries use third-party digital media platforms like Overdrive and Hoopla to provide multimedia content like e-books and streaming video to patrons. Both of these platforms support integration with voice assistants like Amazon Alexa through “skills” (the Alexa equivalent of an application). Patrons are able to download a skill for their Alexa-enabled device to access digital content through these platforms, which are often linked to their library accounts (e.g., “Alexa, ask hoopla how many borrows I have remaining.”).26 Around 14 percent of the respondents reported that they were aware that Overdrive and Hoopla integrated with voice assistants, and 3 percent of all respondents reported that their libraries actively inform patrons about Amazon Alexa skills for these services. When patrons begin connecting their personal voice assistant devices with third -party digital content providers that are also linked to their library accounts, different terms of service agreements and privacy policies overlap creating a complex data rights landscape. Almost a third of our respondents (29 percent) replied that they were aware that Amazon has different privacy policies from Overdrive and Hoopla, with 22 percent responding that they were unaware of these differences (the rest were unsure or did not respond). Only 15 percent of respondents reported that their libraries provided patrons with information about Overdrive and Hoopla’s privacy policies. One library worker offered that, “When helping a patron or informing them that we use Overdrive they are encouraged to read all the privacy info.” However, no libraries in this study reported sharing information about Amazon’s privacy policies with patrons, which might also apply to linked accounts. Lastly, 34 percent of the library workers indicated that they were familiar with the ALA guidelines on privacy that pertain to third-party vendors, and 16 percent reported that their library actively refers to these guidelines in information materials for patrons. For instance, “We have a privacy policy on our website, which was based on the ALA library privacy checklists. It states that our vendors have different privacy policies than we do.” These responses indicate that while some library workers are aware of the privacy implications of the integration of voice assistants into third-party digital content platforms, there are opportunities to increase staff and patron awareness about the intersecting privacy policies and terms of service in this landscape. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 ALEXA, ARE YOU LISTENING? | SWEENEY AND DAVIS 7 Are libraries educating library patrons about voice assistant technologies as a part of services and programming? We were curious if the libraries who used voice assistants in their services were taking any particular measures to inform patrons about the privacy implications of these technologies, or offering any other kinds of specific privacy “best practices” guides for use (e.g., how to erase your data records, adjust settings, etc.). The two libraries who reported circulating voice assistants indicated that they did not include any privacy information with voice assistant devices at checkout. Similarly, we asked library workers about the kinds of technology classes or programming that their libraries were offering, since these might be sites where there is potential to educate or provide information about privacy issues raised by smart technologies like voice assistants. We found that 49 (56 percent) of the libraries represented in the survey (37 public, 12 academic) offer technology courses for the public. Of these, 39 libraries (24 public, 15 academic) responded “yes” to our question asking if aspects of “data privacy or data literacy” are included as part of these classes or other related programming.27 Only 3 libraries (2 public, 1 academic) were able to report that their library offers data literacy education that specifically addresses voice assistant technologies. Library workers provided many examples of the kinds of data literacy information that their libraries typically provided in technology classes and programming. Twelve respondents said that their libraries offered some sort of broad data literacy class and several cited classes specifically targeted at personal data practices and security. Topics taught in these classes included: understanding your personal risk profile; password managers and security; how to understand and protect your digital footprint; and sessions on Facebook and Google where staff “walk users through how to find their information and make decisions about it.” Several respondents identified information literacy topics in conjunction with data literacy, noting that their library teaches classes about identifying “fake news,” phishing scams, and evaluating the authority of websites and website content. None of the responses specifically named issues around privacy or data capture by voice assistants or other smart technologies as topics covered in library technology classes. Several library workers noted that technology classes were offered at their libraries through one-on-one sessions, geared to individually address what patrons had questions about. Based on these responses it is unclear how in-depth, or if at all, these one-on-one sessions might go into informing patrons about privacy best practices and risks when using smart technologies like voice assistants. What kinds of privacy concerns do library workers have about the use of smart voice assistant technologies in their library services and programming? Just over half of the library workers surveyed (52 percent) answered “yes” to the question: “Do you have any privacy concerns about the use of Amazon Echo, Google Home, or Apple Siri devices in the library?” Of the other responses, 16 percent reported “no” concerns and 15 percent answered “I’m not sure.” Those who answered yes were asked to further describe their privacy concerns, resulting in robust descriptions that demonstrated a savvy understanding of the voice assistant data landscape. We characterized library workers’ concerns about voice assistants in the library by five major categories: data access and use; surveillance and “always on” features; procedure and operations; legal issues; and professional responsibility. Data Access and Use By far the most prevalent privacy concerns focused on questions about who has access to data collected by smart voice assistants and how this data might be used (or misused) by different INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 ALEXA, ARE YOU LISTENING? | SWEENEY AND DAVIS 8 parties. Library workers were the most concerned about the reach of access that the three major voice assistant parent companies (Amazon, Google, Apple) have to patron data, closely accompanied by concerns with the selling of this data to third-party vendors: “There are known risks in the logging practices of the assistant vendor (Amazon, Google, Apple). There are potentially greater, and unknown, risks of privacy and data security problems with third-party integrators that libraries are working with to create the Alexa skills, Google Home actions, etc.” “These devices are tied to user accounts for vendors that sell goods and services. There are opportunities to make purchases that we do not want to present to our patrons.” “As currently constituted, most of these devices' privacy policies require owners to allow voice recordings to be sent to cloud services for transcription and, in some instances, for storage and for re-listening by staff or 3rd-party contractors.” Another library worker added that they were concerned about the willingness of these parent companies to “share personal, private data with law enforcement agencies.” This observation underscores what is potentially at stake in terms of patron vulnerability in this data environment. Several concerns focused on patrons “unwittingly leaving their sensitive information on devices that we might use.” “Being that anything we use in the library, or check out to our patrons is shared, I have privacy concerns for what data and recordings will be collected by the services while they are either in use in the library or while they are in the patron’s possession.” While some of these concerns were tied back to how parent companies might use this data, others were equally wary of the potentials for “storing information that can be accessed between patron uses” or by library staff: “As with computers in the info commons, I would be concerned whether user information is scrubbed after each user. Or would one user's information persist and become available to a subsequent user.” “I would not want to be able to identify the patron who used the device. In this case, we cannot. We circulate iPads as assistive devices. As soon as the item is returned, the checkout record is purged.” Lastly, library workers expressed cybersecurity concerns about voice assistants, wondering about how voice assistants might be hacked or otherwise manipulated by malicious actors: “The library is public space, these devices are not known for being secure. A device would have to be registered to some university account, but would be prone to algorithmic manipulation from public voice inputs - if that makes sense?” “Just the idea that they (everything!) is [sic] hackable, and hostage-able, and so on, creeps me out personally, but also in terms of privacy and confidentiality of users of that technology.” INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 ALEXA, ARE YOU LISTENING? | SWEENEY AND DAVIS 9 “Alexa and Google home can be hacked to phish passwords and other sensitive information.” Taken together, these concerns gesture to the opacity of the data environment in terms of who might have access to data (companies, law enforcement, patrons, library staff, hackers) and how this data might be used (advertising and marketing, exposure of personal patron information, state surveillance, and exploitation). Surveillance and “Always On” Features The second major area of concern that library workers expressed was about the surveillance potentials of voice assistants via their passive listening features. In order for voice assistants to respond to their various wake words, they need to be “always on” and listening. While there is a difference between always listening and always recording (which recent studies suggest is no t happening), library workers remained wary about devices “constantly monitoring staff or patrons.”28 These concerns have some obvious overlap to the data access and use theme, but differ in that they are specifically concerned with the act of surveilling—monitoring—patron activities, use patterns, and personal information. Three respondents in this category couched their data privacy concerns in terms of ability to exert some control over their data (e.g., deleting data), or the ability to grant permission/consent to be recorded: “These devices are intended for use in the home. They offer some protections for users with management access. For example, the Google Assistant allows review and deletion of recording history. For users without such access there are no such protections.” “...they [voice assistants] are intended to for use inside a single household, learning the voices, habits and preferences of those household members. I feel that this kind of personal information should be the individual's choice to make and not the library's [sic].” “My concern is that my personal data is being collected without my permission. The same concern applies to patrons of the library. Having them present and turned on captures people's conversations and they may not be aware that is happening.” As these comments suggest, passive listening in public spaces opens up the potential for surveilling patrons and library staff who are not intending to interact with the devices, or who have no knowledge that the device is present. In other words, while some patrons might opt to use a voice assistant to ask a question or look a book up in a library catalog, patrons (and library staff) who are merely talking in the vicinity of these devices may still be listened to and recorded by these devices without their knowledge or consent. This group of privacy concerns conveys a lack of transparency around data collection and surveillance in voice assistants, pointing to larger power differentials between parent companies and users in terms of control over data collection and management. Procedure and Operations Library workers discussed the operational challenges that voice assistants present to staff in terms of establishing routine procedures that ensure patron privacy and confidentiality in between patron use: INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 ALEXA, ARE YOU LISTENING? | SWEENEY AND DAVIS 10 “How do we make sure no residual information remains in the device before someone else uses it or that if used during a program 'private' information isn't being broadcast to other devices in the area?” Another library worker alluded to some of the operational considerations that already accompany library use and lending of personal computing devices, “Clearing data, purchasing, maintaining, we already have iPads and other devices and their management with our staff has been a challenge.” This comment points to the extra staff labor that underpins technology services, which is often not considered as a part of infrastructure for offering these services. Similarly, there is a sense from these comments that establishing procedures to maintain privacy and confidentiality are critical for voice assistants. Failure to erase or secure patron data could lead to inadvertently exposing sensitive or personally identifiable information (PII). “Patron's [sic] may inadvertently be saving their information or staff may forget to delete information causing the previous patrons sensitive information to remain for the next patron to discover.” While Google Home and Amazon Alexa devices do provide the ability for individual recordings to be deleted by the account holder, in the case of shared library use of voice assistants, it would likely be incumbent on a library staff member to access and delete recordings. This raises ethical, legal, and operational questions for library staff required to manage any patron data collected by voice assistants. In any case, procedural concerns are a reminder that library staff have an active role to take in ensuring patron privacy. Legal Issues Library workers in this study identified three legal issues posed by voice assistants in the library. The first legal issue raised was the potential for violation of the Family Educational Rights and Privacy Act (FERPA)—the federal law that protects the privacy of student education records—due to the collection of PII by voice assistants. Library workers in many academic settings are required to maintain compliance with FERPA. One of the respondents was concerned that by using voice assistants in their services, libraries would be putting themselves in a position to potentially violate this law. A second set of concerns focused on questions about the liability of the library (or individual library workers) if a patron’s PII is misused by technology companies or the third-party vendors who have access to user data: “I have great concerns regarding the use of this technology in a library setting since it might expose the library to potential liability if, more likely when patron data is misused by the technology providers.” Related to this concern, another library worker asked, “Who owns the info?” Questions about rights and ownership of personal data by technology companies, itself a fraught and opaque legal area, require more ethical and legal probing as libraries become intermediaries to patron use of voice assistants. Lastly, one library worker cited concerns about librarians’ ability to uphold first amendment rights with voice assistants. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 ALEXA, ARE YOU LISTENING? | SWEENEY AND DAVIS 11 “We take our mandated role to uphold first amendment rights and patron privacy very seriously. There are too many issues with the way these for-profit companies collect, store and potentially use information. We see no benefits of service gained that offset these concerns. We are also concerned about the way owners of these products use their wealth to leverage political influence.” This comment identifies privacy as a necessary condition for facilitating free speech, contrasting this with a sketch of the political and economic motives underlying voice assistant development. The concerns raised by these library workers point to the complexity of managing patron data in the context of a variety of existing legal frameworks. Professional Responsibility Three respondents explicitly placed privacy concerns in the context of their professional responsibility as library workers to “protect” patrons and patron privacy. A fourth respondent voiced a twin concern about “the library's inability to protect privacy and patron information” (emphasis added). Beyond descriptions of protecting patrons, these library workers framed privacy as a professional value. Comments such as, “We take our mandated role to uphold first amendment rights and patron privacy very seriously,” emphasize privacy as a professional charge. These kinds of comments tacitly draw on LIS professional core values and ethics statements to position responsible professional practice as the action of upholding privacy. As a result, professional identity is discursively constructed by these library workers as a function of valuing privacy. The following comment, particularly, draws an identity-based line between “us” (library professionals) and “them” (technology companies) that is based on divergent values surrounding privacy: “Since one of the main concerns we (should) have as library professionals is patron privacy; ‘teaming up’ with technology providers who do not have that level of concern is problematic at best.” The assertion that library core values may be in conflict with the technology providers that are designing voice assistants is very astute, and important for libraries to consider when weighing the decision to experiment with these (and other) emerging smart technologies. DISCUSSION: KEY CONSIDERATIONS FOR LIBRARY PROFESSIONALS Our research suggests that library use of voice assistants poses many as-of-yet unresolved privacy issues for library staff and patrons alike. Though voice assistant use is still fairly nascent across public and academic libraries, our study confirms that these tools are already being adopted by some libraries. The adoption of these, and other, smart technologies, is likely to keep trending in library services across institution types, paralleling market trends for personal adoption of voice assistants. Many library workers in our study expressed astute concerns about voice assistants, raising important questions about how patron data was collected, managed, and used across the data lifecycle of these technologies. This is a critical moment, then, for the library profession to take stock of questions of privacy surrounding voice assistants, and an opportunity to set a broader professional agenda for data-privacy that encompasses the complexities of smart technology use in library services. In this spirit, we have identified several main areas of concern that emerged from our study, posited as key considerations about voice assistants for library professionals to grapple with. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 ALEXA, ARE YOU LISTENING? | SWEENEY AND DAVIS 12 Circulation Procedures For libraries who are, or are considering, lending voice assistant-enabled technologies, clear lending rules are needed for patrons that set guidelines for disconnecting their personal Amazon, Apple, or Google accounts before returning the device. Likewise, it is important to develop procedures for library staff to anticipate instances when patrons forget to disconnect their personal accounts. Library workers cannot, and should not, be responsible for disconnecting personal accounts as a protective measure for both staff and patrons, since doing so asks library workers to access and take responsibility for personal patron data, including PII. One suggestion might be to require devices to be restored to factory settings, which could be verified by a library staff member at time of device return. Libraries might also consider including privacy best practices with these devices that outline known privacy risks and provide information about how to adjust settings to limit data sharing or delete records in personal accounts where applicable (e.g., Amazon). Third-party Digital Content Platforms The integration of voice assistants in third-party digital content platforms licensed by libraries is becoming more common, pointing to the complexity of upholding patron data privacy throughout these layered and linked services. This issue speaks to the difficulties navigating overlapping privacy statements and terms of service agreements, which is not unique to voice assistants but does indicate the need for more data protections and consumer-oriented information policies. ALA already does advocacy work on these issues and provides many helpful guidelines , such as the Library Privacy Guidelines for Vendors (http://www.ala.org/advocacy/privacy/guidelines/vendors). Still, the data environment is very much characterized by the unequal power differential between technology companies and users. We are in dire need of more robust information policy frameworks that are predicated on transparency, strict parameters for data collection and use, corporate accountability, and user control and agency. A promising example of this is the General Data Protection Regulation (GDPR) implemented in the European Union in 2018. Something similar is needed in the US to regulate corporate data-sharing practices and give users more control over their data. This would be beneficial across the board for the public, as well as to library patrons using their personal voice assistant devices to access library resources. Education opportunities for expanding digital literacy Library workers in our study reported a range of technology education and digital literacy programming initiatives in their libraries, though none that specifically addressed voice assistants. This suggests that library technology programming might not be targeting the kinds of specific privacy concerns posed by smart technologies like voice assistants. As smart technologies like voice assistants become more common for household/personal use, it would make sense to expand library programming initiatives to include informational sessions that incorporate data privacy considerations for smart technologies in addition to skills-driven sessions. Additionally, some survey responses indicated that library workers may have some knowledge gaps or a lack of concern about voice assistant use. This might point to a need for expanded education, training, and professional development around data privacy issues and emerging technologies for library workers. There has already been a large push in the field to expand digital literacy, defined by ALA as “the ability to use information and communication technologies to find, evaluate, create, and communicate information, requiring both cognitive and technical skills.”29 However, this definition of digital literacy falls short of considering the role of assessing data collection, storage, and use as a core part of digital knowledge. Expanding digital literacy training, for both staff and http://www.ala.org/advocacy/privacy/guidelines/vendors INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 ALEXA, ARE YOU LISTENING? | SWEENEY AND DAVIS 13 patrons, to include awareness of the data ecosystems and privacy concerns that undergird smart technologies is a must for responsive library services. Surveilling Patrons and Staff Voice assistants placed in public service areas, in the library stacks, and in public gathering areas within the library raise the ethical issue of recording patrons (and staff) who either do not wish to be recorded, or do not even know they may be recorded. In the case of library staff, this poses a labor issue where staff may be asked to work in areas where devices may be listening to their interactions during the duration of their shift. For patrons, this could compromise privacy in reference transactions and in other information seeking activities, as well as capturing other personal interactions that take place in the library setting. It is critical that libraries are transparent about using voice assistant technologies, upfront about the potential privacy harms of these technologies, and abide by “opt-in” rather than “opt-out” frameworks. Library workers should consider treating voice assistant records in the same way they have historically treated circulation records, opting to either delete these records or not collect them (meaning, not use voice assistants) at all. Unlike circulation records, however, library workers have far less control over the data captured by voice assistants. This data is stored in the cloud on privately owned servers that remain outside of library control and oversight. Given the incredibly low bar for federal access to information under the USA PATRIOT Act, actively facilitating the collection of patron and staff interactions, particularly without informed consent, should give librarians pause. Opt to NOT Adopt In light of the issues raised in this study, library workers need to seriously weigh whether the benefits of using voice assistants in libraries at this point in time outweigh the vast privacy concerns that we have outlined here. As it stands, these technologies are not currently filling a gap in library services that cannot be otherwise met by more traditional service models that carry fewer potential harms for our patron communities. Importantly, not all patrons are equally vulnerable to harm or exploitation in these data environments. For instance, there is a wealth of research that demonstrates the multitude of ways that Black, Indigenous, People of Color, LQBTQ+, women, and low-income communities are subjected to higher levels of surveillance and data profiling that results in harassment, discrimination, economic penalties, and legal persecution.30 As the current national political landscape is aflame in protests against police violence and anti-Black racism, it is important to identify surveillance technologies as policing technologies. Libraries need to consider that these tools, as extensions of policing data networks, may directly endanger, particularly, Black, Latinx, and Indigenous people who are already subjected to over-policing. In this sense, concerns about patron data privacy are high-stakes and are deeply linked to the professional core value of social responsibility.31 Libraries should consider not using voice assistants until key data privacy concerns are addressed, more robust data protections are in place at a federal level, and the blanket authority for federal agencies and law enforcement to compel user data is revoked. This is not a technophobic stance. On the contrary, we are suggesting that library workers could serve an important role as privacy advocates, which includes critically evaluating the role of emerging technologies in their communities on behalf of public interest. A key part of this must include the library profession taking responsibility for the use of surveillance technologies in their institutions since these technologies are deeply implicated in the policing of disenfranchised communities by state and federal authorities. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 ALEXA, ARE YOU LISTENING? | SWEENEY AND DAVIS 14 CONCLUSION We view this study as a modest starting point for mapping some of the many privacy issues associated with voice assistant use in library services and programming and hope it points a way forward for future research. Future research might address specific case studies of voice assistant use in libraries, data mapping of patron data through third-party library services, use and privacy issues across different institution types, patron digital literacies with voice assistants, and library policies for smart technologies more generally. Plural and diverse vantage points are needed to understand the potential impacts of these technologies across different community types. Such research is critical for developing best practices, guidelines, policies, and education opportunities for voice assistant use (and other smart technologies) that prioritize patron privacy and confidentiality. The use of voice assistants in libraries raises questions about the responsibility of libraries and librarians to actively engage patron data privacy concerns when considering integrating these technologies into services and programming. Indeed, we encourage library workers to consider informed non-adoption of these technologies as a socially responsible professional stance until the key issues we have outlined are addressed. While it is, of course, important for library workers to remain current and innovative in their services, it is also paramount that patron privacy (as a function of safety) stays at the forefront of library services. In other words, it is the responsibility of library workers to anticipate potential privacy issues associated with emerging technologies, rather than treating privacy as a secondary concern to technology adoption. There are tremendous opportunities for library workers to lead the data privacy charge—in collaboration with community stakeholders—in pursuit of privacy-centered library services that are accountable to community members, particularly those who are mostly likely to be harmed by these technologies. INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 ALEXA, ARE YOU LISTENING? | SWEENEY AND DAVIS 15 APPENDIX A: SURVEY INSTRUMENT 1. By selecting the “I agree” button below, I hereby certify: that I am 19 years old or older; that I have read and understand the above consent form; and that this action indicates my willingness to voluntarily take part in the study. a. I agree to participate in the research study described above. b. I do not agree to participate in the research study described above. 2. Do you work in a library setting? a. Yes b. No 3. What kind of library do you work at? a. Public b. Academic c. School d. Other, please specify [fill in the blank] 4. What is the size of your library’s service population? a. 2,500 or less b. 2,500-9,999 c. 10,000-25,000 d. 25,000+ e. I’m not sure. 5. What state is your library located in? [fill in the blank] 6. Does your library have Amazon Echo devices, Google Home devices, or Apple Siri devices available for use by patrons? a. Yes b. No c. I’m not sure. 7. Which of the following digital assistant devices does your library have available for patrons to use? a. Amazon Echo devices b. Apple Siri devices c. Google Home Devices d. Other products, please specify: [fill in the blank] 8. Please provide some examples of how your library patrons use the library's digital assistant technologies. [short answer] 9. Could you describe where these digital assistant technologies are located in the library? [short answer] INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 ALEXA, ARE YOU LISTENING? | SWEENEY AND DAVIS 16 10. Does your library use Amazon Echo devices, Google Home devices or Apple’s Siri devices in any of the following kinds of programming? (Select all that apply) a. Tech “petting zoos” b. Trivia c. Homework Help d. Technology Classes e. Makerspaces f. Not listed, please specify: [fill in description] g. None of the above 11. For the programs you selected, briefly explain how the devices are integrated into programming. [Short Answer] 12. Does your library circulate Amazon Echo, Google Home, and Apple Siri devices to the public for checkout? a. Yes b. No c. I’m not sure. 13. Which devices do you circulate? a. Amazon Echo devices b. Apple Siri devices c. Google Home devices d. Other products, please specify [fill in the blank] 14. Do you provide any privacy information and/or best practice information with the device at checkout? a. Yes b. No c. I’m not sure 15. If so, briefly explain what kind of privacy or best practices information you include. Examples of content covered in this information would be helpful. [Short Answer] 16. Do you have any privacy concerns about the use of Amazon Echo, Google Home, or Apple Siri devices in the library? a. Yes b. No c. I’m not sure 17. Could you describe these privacy concerns? [Short Answer] 18. Does your library offer any sort of technology courses to the public? a. Yes b. No c. I’m not sure INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 ALEXA, ARE YOU LISTENING? | SWEENEY AND DAVIS 17 19. Does your library teach data privacy or data literacy as part of the library's programming? a. Yes b. No c. I’m not sure 20. Does your library offer any data literacy education in programming that specifically addresses digital assistants? a. Yes b. No c. I’m not sure 21. What kinds of data literacy information is provided in these courses taught at your library? Please provide some examples: [Short Answer] 22. Does your library use any of the following services? Select all that apply: a. Overdrive/Libby b. Hoopla c. None of the above 23. Are you aware that both Overdrive and Hoopla have Amazon Echo application integration (called "Skills")? a. Yes b. No c. I’m not sure 24. Does your library inform patrons about Amazon Echo Skills on Overdrive and/or Hoopla? a. Yes b. No c. I’m not sure 25. Are you aware that Amazon's privacy policies differ from those of Overdrive and Hoopla? a. Yes b. No c. I’m not sure 26. Does your library provide any information to patrons about Overdrive and Hoopla's privacy policies? a. Yes b. No c. I’m not sure 27. Does your library provide any information to patrons about Amazon's privacy policies? a. Yes b. No c. I’m not sure 28. Please provide a brief description of the information that you are providing to patrons on this subject, including where this information is located for patron access. [Short Answer] INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 ALEXA, ARE YOU LISTENING? | SWEENEY AND DAVIS 18 29. Are you aware of the guidelines that the American Library Association (ALA) provides on privacy as it pertains to third party electronic vendors? a. Yes b. No 30. Does your library use or refer to these privacy guidelines in any informational materials for patrons? a. Yes b. No c. I’m not sure 31. Please describe these informational materials, including how and where they are distributed to patrons: [Short Answer] INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 ALEXA, ARE YOU LISTENING? | SWEENEY AND DAVIS 19 ENDNOTES 1 Benjamin Herald, “Teacher’s Aide or Surveillance Nightmare? Alexa Hits the Classroom,” Digital Education, Education Week, June 26, 2018, http://blogs.edweek.org/edweek/DigitalEducation/2018/06/alexa_in_the_classroom_teacher s_surveillance.html?cmp=SOC-SHR-FB. 2 Carrie Smith, “Your Library Needs to Speak to You,” American Libraries (June 3, 2019), https://americanlibrariesmagazine.org/2019/06/03/voice-assistants-your-library-needs-to- speak-to-you/. 3 Nicole Hennig, Siri, Alexa, and Other Digital Assistants: The Librarian’s Quick Guide (Santa Barbara, CA: Libraries Unlimited, 2018) 33–8. 4 Adapted from: Brenda Laurel, “Interface Agents: Metaphors with Character,” Human Values and the Design of Computer Technology (1997): 207–19, Cambridge University Press. 5 Emily Clark, “Alexa, Are You Listening? How People Use Voice Assistants,” https://clutch.co/app- developers/resources/alexa-listening-how-people-use-voice-assistants. 6 Clark, “Alexa, Are You Listening? How People Use Voice Assistants.” 7 Clark, “Alexa, Are You Listening? How People Use Voice Assistants.” 8 "Voice Control", American Library Association, http://www.ala.org/tools/future/trends/voicecontrol. 9 Shannon Liao, “Google Home Will Play Music and Sound Effects When You Read Disney Storybooks,” https://www.theverge.com/2018/10/29/18037466/google-home-disney- music-moana-incredibles-coco-storytime; Hennig, Siri, Alexa, and Other Digital Assistants, 35; Susan Allen and Avneet Sarang, “Serving Patrons Using Voice Assistants at Worthington,” Online Searcher 42, no. 6 (November-December 2018): 49–52. 10 Smith, “Your Library Needs to Speak to You.” 11 Smith, “Your Library Needs to Speak to You.” 12 Allen and Sarang, “Serving Patrons Using Voice Assistants at Worthington.” 13 King County Library System, “Voice Assistants, Connecting You to Your Library,” https://kcls.org/voice/. 14 “Core Values of Librarianship,” American Library Association, http://www.ala.org/advocacy/intfreedom/corevalues. 15 Miriam E. Sweeney, “Digital Assistants,” in Uncertain Archives: Critical Keywords for Big Data, ed. Nanna Bonde Thylstrup, Daniela Agostinho, Annie Ring, Catherine D’Ignazio , and Kristin Veel (Baltimore, MD: MIT Press, 2021), 151–60. http://blogs.edweek.org/edweek/DigitalEducation/2018/06/alexa_in_the_classroom_teachers_surveillance.html?cmp=SOC-SHR-FB http://blogs.edweek.org/edweek/DigitalEducation/2018/06/alexa_in_the_classroom_teachers_surveillance.html?cmp=SOC-SHR-FB http://blogs.edweek.org/edweek/DigitalEducation/2018/06/alexa_in_the_classroom_teachers_surveillance.html?cmp=SOC-SHR-FB http://blogs.edweek.org/edweek/DigitalEducation/2018/06/alexa_in_the_classroom_teachers_surveillance.html?cmp=SOC-SHR-FB https://americanlibrariesmagazine.org/2019/06/03/voice-assistants-your-library-needs-to-speak-to-you/ https://americanlibrariesmagazine.org/2019/06/03/voice-assistants-your-library-needs-to-speak-to-you/ https://clutch.co/app-developers/resources/alexa-listening-how-people-use-voice-assistants https://clutch.co/app-developers/resources/alexa-listening-how-people-use-voice-assistants http://www.ala.org/tools/future/trends/voicecontrol https://www.theverge.com/2018/10/29/18037466/google-home-disney-music-moana-incredibles-coco-storytime https://www.theverge.com/2018/10/29/18037466/google-home-disney-music-moana-incredibles-coco-storytime https://kcls.org/voice/ http://www.ala.org/advocacy/intfreedom/corevalues INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 ALEXA, ARE YOU LISTENING? | SWEENEY AND DAVIS 20 16 Anthony Cuthbertson, “Amazon Admits Employees Listen to Audio From Echo Devices,” https://www.independent.co.uk/life-style/gadgets-and-tech/news/amazon-alexa-echo- listening-spy-security-a8865056.html. 17 Matt Day et al., “Amazon Workers Are Listening to What You Tell Alexa,” https://www.bloomberg.com/news/articles/2019-04-10/is-anyone-listening-to-you-on- alexa-a-global-team-reviews-audio. 18 Daniel J. Dubois et al., “When Speakers Are All Ears: Understanding When Smart Speakers Mistakenly Record Conversations,” Mon(Iot)r, February 14, 2020, https://moniotrlab.ccis.neu.edu/smart-speakers-study/. 19 Karen Hao, “Amazon is the Invisible Backbone of ICE’s Immigration Crackdown,” MIT Technology Review, October 16, 2019, https://www.technologyreview.com/s/612335/amazon-is-the- invisible-backbone-behind-ices-immigration-crackdown/. 20 Zack Whittaker, “Echo Is Listening, but Amazon’s Not Talking,” ZDNet, January 16, 2018, https://www.zdnet.com/article/amazon-the-least-transparent-tech-company/. 21 American Library Association, “Library Privacy Guidelines for Vendors,” http://www.ala.org/advocacy/privacy/guidelines/vendors. 22 This research protocol (19-08-2671) was approved in October 2019 by the University of Alabama’s institutional review board (IRB). 23 Note, participants were not required to answer every question, so some questions have fewer than 86 total responses due to participants electing to not respond. Also, even though we were targeting public and academic libraries, we did receive a response from someone identifying their institution as a school library and decided to include it in the results. 24 We did not receive responses from libraries in Arizona, Arkansas, Connecticut, Delaware, Pennsylvania, Vermont, Virginia, or Wyoming. 25 “Technology petting zoos” are areas where patrons can experiment or try out technologies and gadgets. 26 Hoopla, “Alexa, Meet Hoopla,” July16, 2018, http://hub.hoopladigital.com/whats- new/2018/7/alexa-meet-hoopla. 27 We purposely couched questions about “data literacy” and “data privacy” broadly in the survey to allow for a range of interpretations by respondents in an attempt to capture the range of information that might be taught under this umbrella. 28 Daniel J. Dubois et al., “When Speakers Are All Ears: Understanding When Smart Speakers Mistakenly Record Conversations.” 29 American Library Association, “Digital Literacy,” https://literacy.ala.org/digital-literacy/. 30 Examples of critical research in this area include: Toby Beauchamp, Going Stealth: Transgender Politics and U.S. Surveillance Practices (Durham, London: Duke University Press, 2019); https://www.independent.co.uk/life-style/gadgets-and-tech/news/amazon-alexa-echo-listening-spy-security-a8865056.html https://www.independent.co.uk/life-style/gadgets-and-tech/news/amazon-alexa-echo-listening-spy-security-a8865056.html https://www.bloomberg.com/news/articles/2019-04-10/is-anyone-listening-to-you-on-alexa-a-global-team-reviews-audio https://www.bloomberg.com/news/articles/2019-04-10/is-anyone-listening-to-you-on-alexa-a-global-team-reviews-audio https://moniotrlab.ccis.neu.edu/smart-speakers-study/ https://moniotrlab.ccis.neu.edu/smart-speakers-study/ https://moniotrlab.ccis.neu.edu/smart-speakers-study/ https://www.technologyreview.com/s/612335/amazon-is-the-invisible-backbone-behind-ices-immigration-crackdown/ https://www.technologyreview.com/s/612335/amazon-is-the-invisible-backbone-behind-ices-immigration-crackdown/ https://www.zdnet.com/article/amazon-the-least-transparent-tech-company/ http://www.ala.org/advocacy/privacy/guidelines/vendors http://hub.hoopladigital.com/whats-new/2018/7/alexa-meet-hoopla http://hub.hoopladigital.com/whats-new/2018/7/alexa-meet-hoopla https://literacy.ala.org/digital-literacy/ INFORMATION TECHNOLOGY AND LIBRARIES DECEMBER 2020 ALEXA, ARE YOU LISTENING? | SWEENEY AND DAVIS 21 Virginia Eubanks, Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor (St. Martin’s Press, 2018); Safiya U. Noble, Algorithms of Oppression: How Search Engines Reinforce Racism (New York: NYU Press, 2018). 31 “Core Values of Librarianship,” American Library Association. ABSTRACT Introduction Review of Literature What is a Voice Assistant? Libraries and Voice Assistant Use Data Privacy Issues Research Methods FINDINGS Participant Demographics How are libraries using smart voice assistant technologies as a part of their library services? How aware are library workers of how voice assistants integrate with third-party digital content platforms? Are libraries educating library patrons about voice assistant technologies as a part of services and programming? What kinds of privacy concerns do library workers have about the use of smart voice assistant technologies in their library services and programming? Data Access and Use Surveillance and “Always On” Features Procedure and Operations Legal Issues Professional Responsibility Discussion: Key considerations for library professionals Circulation Procedures Third-party Digital Content Platforms Education opportunities for expanding digital literacy Surveilling Patrons and Staff Opt to NOT Adopt Conclusion Appendix A: Survey Instrument ENDNOTES