Impact of a Discovery System on Interlibrary Loan 643 Impact of a Discovery System on Interlibrary Loan Linda R. Musser and Barbara M. Coopey Linda R. Musser is Head of the Earth and Mineral Sciences Library and Barbara M. Coopey is Head of Interlibrary Loan at The Pennsylvania State University; e-mail: lrm4@psu.edu, bmc4@psu.edu. © 2016 Linda R. Musser and Barbara M. Coopey, Attribution-NonCommercial (http://creativecommons.org/ licenses/by-nc/3.0/) CC BY-NC. Web-scale discovery services such as Summon (Serial Solutions), World- Cat Local (OCLC), EDS (EBSCO), and Primo (Ex Libris) are often touted as a single search solution to connect users to library-owned and -licensed content, improving discoverability and retrieval of resources. Assessing how well these systems achieve this goal can be challenging, however. This article describes the use of interlibrary loan borrowing request data as a means to evaluate whether the Summon discovery system was achieving its goal of improving retrieval. Four years following implementa- tion of the discovery system, there was a 22 percent decrease in interli- brary loan borrowing requests. Requests by undergraduates dropped at a higher rate than faculty, staff, and graduate students. Undergraduate requests for articles dropped more than requests for loans. Two years’ postimplementation of the discovery system, the number of interlibrary loan borrowing requests by undergraduates for locally owned or licensed materials dropped by 57 percent, freeing up interlibrary loan staff to ex- pand services into new areas. This study indicates that implementation of a discovery system can lead to a reduction in interlibrary loan borrowing. any libraries have implemented or are considering implementing discov- ery systems, such as Summon (Serial Solutions), WorldCat Local (OCLC), EDS (EBSCO), and Primo (Ex Libris). These systems purport to simplify the research process by providing a single search interface to aggregated content, composed of locally owned or licensed resources as well as publicly available content on the Internet. One of the primary reasons libraries choose to use these unified solutions is to not only simplify the search and discovery process (single search box married with relevancy ranking algorithms, and histograms of search results) but to more easily connect users to available content. Libraries invest significant amounts of money in online, full-text resources and want to ensure that both locally owned and licensed resources are easily discovered and accessed. Various studies have examined the impact of discovery systems by examining changes in usage of resources.1 Others have examined user behavior and other criteria.2 In this study, the effect of imple- menting the Summon discovery service was examined by looking at changes in the number of interlibrary loan borrowing requests. The impact on requests submitted by undergraduate students for locally available items (that is, requests for materials already owned or licensed for local use) was also examined. doi:10.5860/crl.77.5.643 crl15-800 644 College & Research Libraries September 2016 Background Penn State has an enrollment of almost 100,000 students, located at more than twenty campus locations in Pennsylvania plus an online campus, called World Campus. It is a Research 1 (very high research activity) university according to the Carnegie Clas- sification of Institutions of Higher Education and awards more than 18,000 degrees annually in more than 160 fields. Supporting these students is the Penn State University Libraries, which ranks eighth among North American research libraries as measured on the Association of Research Libraries Investment Index Rankings for 2013–2014.3 The libraries’ collections number some seven million items, including more than 400,000 e-books and other online resources. The vast majority of collections are described by bibliographic records in the library catalog, with portions of the maps and techni- cal reports collections the only major collections not yet fully represented. Both the libraries’ discovery system, LionSearch, and the library catalog display holdings for all library locations. In July 2011, as part of an effort to simplify access to library resources and improve retrievability, the university libraries implemented LionSearch, the local name given to Penn State’s instance of the Summon web-scale discovery service. Prominently dis- played on the libraries’ homepage, LionSearch, with its Google-like single search box, was designed to be the first choice for users looking for library resources. LionSearch aggregates results from myriad resources owned or licensed by the library, as well as open access resources, and provides direct links to full-text resources such as articles and e-books. LionSearch also incorporates library catalog information, displaying loca- tion information as well as offering the option to request a physical item. For articles that are not available in full text, LionSearch provides a request button that populates the needed item’s bibliographic information into a request form for document delivery. The libraries promote LionSearch as a fast and simple integrated search of the majority of the libraries’ resources. The libraries function as “one library geographically dispersed” across multiple campuses, reporting to a single dean of libraries and with a single library faculty, including the medical and law libraries. All library resources and services, including interlibrary loan (ILL), are offered free of charge to Penn State students, faculty, and staff, regardless of their campus affiliation or location. There are no request limits or restrictions for using resource-sharing systems, and links to interlibrary loan request forms are provided in multiple locations on the libraries’ website as well as being in- tegrated into many of the online resources and databases. ILLiad, a resource-sharing management software developed by Atlas Systems, is used by the ILL department to manage requests for interlibrary loan material. LionSearch and licensed databases interface easily with ILLiad through link resolvers and can automatically populate an ILLiad request form with bibliographic information, thus simplifying the submission process for users. With most of the interlibrary loan request form fields automatically completed, all the user needs to do is submit the request. The libraries’ interlibrary loan staff are largely centralized and located at the Univer- sity Park campus and serve users from all campus locations as well as distance users. Staff at branch and campus library locations assist with interlibrary loan by processing requests for material located at their libraries. With a dispersed user population and many library locations, service-oriented interlibrary loan staff are aware that users may not know that an item is held by the libraries or recognize where it is located. Plus, it may not be convenient or possible for a user to go to the specific library housing the item to search the stacks for the book or print journal. Thus, the libraries offer full document delivery for all faculty, students, and staff. Requests are submitted via ILLiad; incoming requests are first checked for local availability, and, if items are present locally, holds are Impact of a Discovery System on Interlibrary Loan 645 placed on those available items. Interlibrary loan staff also scan from print volumes or download full-text articles available locally and post these on the user’s ILLiad account. For items not available locally, interlibrary loan staff request these from libraries and document suppliers throughout the world. Requested articles are posted on the user’s ILLiad account; loaned materials are delivered similarly to locally owned material. Previous Studies Many studies investigating the impacts of discovery services have focused on changes in use of materials, changes in user behavior, or the user experience. We hypothesized that the discovery service would make our resources, particularly our article resources, more discoverable. To assess the impact, we examined changes in requests for locally owned or licensed materials placed as interlibrary loan requests. Since all Penn State users currently have document delivery privileges, our study of the impact on requests for locally owned materials was restricted to data from a three-year period during which undergraduate students had access to interlibrary loan but not document delivery. This period coincided with the implementation of our discovery system. If our hypothesis was correct, interlibrary loan borrowing request rates for locally owned or licensed materials would be reduced following implementation. Additionally, we examined the impact of the discovery system on overall borrowing rates. Ours is not the first study to examine request rates for locally owned or licensed content.4 Although the time periods and type of institution studied varies significantly, when taken as a whole, the median rate of cancellation due to local ownership across all these studies is 11 percent. Several studies looked at the number of cancellations due to local ownership as a percentage of all cancellations. Robb Waltner’s estimate that 30 percent of cancellations were for locally owned items is in line with the findings of Yue and Syring, as well as Beaubien, who stated, “For most libraries surveyed, at least 25% of borrowing requests were cancelled because of local availability.”5 A 1991 study by Catherine Dwyer et al. of cancelled requests at the University of Illinois discussed reasons for cancellations and reported that users submitted requests for already owned articles at a much higher rate than monographs, a finding repeated in later studies by Beaubien et al., Page and Kuehn, Murphey and Greenwood, and Gaffney.6 These studies postulated various reasons why users placed requests for locally owned or licensed material. These reasons can be categorized as follows: a. items were locally owned but unavailable—circulating, on reserve or reference, and so on; b. items were available but in an undesirable format—microformats tend to be unpopular, users may not have been aware of print holdings, or simply pre- ferred online format; c. users found it more convenient to request from ILL than to seek it themselves— ILL delivers, links to ILL forms are easily available, users assumed that their library did not own item and so did not bother to look; d. link resolvers did not display locally owned or licensed content, or there was an error in the link; e. multiple records can lead to requests for owned items (that is, holdings are displayed on another record); f. references were inaccurate or incomplete; g. users’ lack of expertise or understanding of how to use the catalog, or other interfaces to access content or local holdings, led to their not finding content. As libraries introduce discovery services, the enhanced discoverability of library resources should help reduce some of the obstacles associated with the reasons men- tioned above. 646 College & Research Libraries September 2016 Research on the impact of discovery systems on interlibrary loan borrowing has been uneven. There have been a large number of studies examining the impact of the WorldCat Local discovery system on interlibrary loan operations. Thomas Deardorff and Heidi Nance reported that, after the WorldCat Local implementation at the University of Washington, “ILL saw large shifts in the volume and type of requests.” They reported a 92 percent increase in overall borrowing requests, with a 339 percent increase in requests by undergraduates.7 Gaffney reported an overall increase of 8 percent in the use of interlibrary loan at the University of Delaware, with some months having 50 percent more requests than the previous year. She stated, “WorldCat Local makes interlibrary loan a much more visible and accessible service, especially when it is promoted as a library’s primary search interface.” She also stated, “The majority of requests received via WorldCat Local are for loans of returnable items.” This was true also at the University of Washington, which experienced a 150 percent increase in loan requests.8 Similarly, Washington State University reported a huge jump in ILL usage of more than 70 percent following implementation.9 The two Big Ten Academic Alliance libraries using WorldCat Local reported similar results in response to an e- mail question posted on the ILL Directors listserv by the authors in 2014.10 ILL directors were asked if they experienced any change in ILL borrowing requests after a discovery service was implemented at their library. Both the University of Maryland and Ohio State University stated in the survey that they implemented WorldCat Local in 2011 and by 2012 saw an increase in the number of requests, especially for loans of returnable items. Ohio State experienced an 81 percent increase in the number of loan requests, whereas the University of Maryland saw a 14 percent increase and then another 21 percent increase for 2013. Conversely, Colleen Harris-Keith reported an overall drop of more than 30 percent two years after implementing WorldCat Local but noted that additions to library subscriptions had probably impacted the trend.11 To summarize, most institutions using the WorldCat Local discovery system experienced an increase in interlibrary loan borrowing following implementation. Studies reporting the impact of other discovery services on interlibrary loan are less common. Calvert examined the impact of the EDS discovery system on interlibrary loan borrowing over a two-year period, reporting that there was a decrease in article requests and an increase in loan requests.12 In other studies of EDS, Jody Fagan and Meris Mandernach reported an initial increase in ILL requests followed by a dropoff, while Amy Fry quotes Michael Kucsak from the University of North Florida stating that “Interlibrary loan use has been flat.”13 Our literature review did not reveal any studies reporting on the impact of Summon or other discovery services on interlibrary loan. Penn State’s Study Prior to LionSearch, users were confronted with a multitude of starting points for their research, from databases to web pages, from e-journal lists to the local catalog. It was not uncommon for users to become frustrated and turn to the interlibrary loan department to find books and articles that were readily available online or on the shelf—an example of reason c) convenience. As one user pointed out, “I first look on the Libraries website for what I need. However, if I don’t find it after a few minutes, I place a request in ILLiad and a few hours later get the article from ILL.”14 ILL staff were spending considerable time downloading articles from locally available resources to fulfill requests, referring users to licensed full-text resources, or placing holds on books, all of which users could have done on their own. We wondered if all of these requests for local material were submitted to ILL only because the user found it more convenient for staff to search for the item for them or if, in fact, the user could not find the item locally. If the latter is true, we hypothesized that the implementation of the Impact of a Discovery System on Interlibrary Loan 647 discovery system would reduce the number and/or type of ILL requests for locally owned or licensed resources, since these items would be more readily discoverable. To test this hypothesis, we examined the number of requests for locally owned or licensed materials submitted via ILLiad, before and after implementation of LionSearch. While we recognized that a percentage of users submit ILL requests as a way to avoid taking the time to do their own searching, we assumed these users and the number of their requests would not be affected by the availability of a discovery system. In addition to examining cancelled requests, we also looked at the impact of the discovery system on overall borrowing rates. Around the time LionSearch was being implemented, and in response to changing user expectations, the Interlibrary Loan department was shifting its service philosophy and practices toward fulfilling requests with locally available items instead of sending cancellation e-mails. Our document delivery service, whereby articles from the librar- ies’ print journal collection were scanned and posted on ILLiad, was expanded over several years. Document delivery service for faculty and staff was begun in January 2008, extended to graduate students in 2011, and finally offered to undergraduate students during fall semester 2013. Once document delivery services were extended to undergraduates, ILL staff no longer cancelled requests received for locally owned or licensed materials, assuming rather that these were document delivery requests. This meant that, for this part of the study, we had only two years after the July 2011 implementation of LionSearch to extract undergraduate cancellation data. Data on requests for locally owned or licensed resources for three years were examined: from fiscal year 2010/11 through 2012/2013 (our fiscal year runs July–June). Overall inter- library loan borrowing data were gathered through fiscal year 2014/2015. Data were derived using custom Microsoft® Access queries linked to the ILLiad database, with additional loan data derived from the regional consortium resource-sharing system. Queries for cancelled requests for locally owned or licensed resources were created to extract requests with the cancellation e-mails with either of two subject lines: 1. “Libraries Owns Requested Item”—This e-mail directed the student to the library location and call number of the book or journal. At times, if staff placed a hold for the user, the e-mail message was altered to indicate that a hold was placed for the user. 2. “Requested Material Available Full Text”—This e-mail supplied a link to the full text. Another e-mail that was used when staff opted to post the full text on the user’s ILLiad account for convenience was the “Requested Item Delivered Electronically.” For the study, the following conditions generated a cancellation e-mail: • Item was available for immediate loan (in other word, it was on the shelf) and could be requested via our “I Want It” button, which displays in the catalog and in LionSearch records. • Journal article was available from the user’s local campus library (that is to say, it was on the shelf). • Full text was available and there was an accurate and usable URL link within the record. Cancellation notices were not generated for requests for journal material at our remote storage facilities or for requests for articles in print journals at a campus library location other than the undergraduate’s home campus, since article delivery service in these instances was already provided to them. Results Figure 1 summarizes the data on cancelled undergraduate requests over three years 648 College & Research Libraries September 2016 beginning with the year prior to implementing LionSearch. In fiscal year 2010/11, there were a total of 2,120 requests cancelled because they were locally owned or licensed. In fiscal year 2011/12, the first year after implementation of LionSearch, the number of cancelled requests dropped by 33 percent; after two years, the reduction totaled 57 percent. Taken separately, the overall reduction in requests for locally owned or licensed content was 58 percent for articles and 56 percent for loans, respectively. While the reduction in cancelled requests was impressive, the impact is lessened when viewed as a percentage of all undergraduate interlibrary loan requests. Prior to LionSearch, in fiscal year 2010/11, the cancellation rate due to local ownership was 15 percent of all undergraduate interlibrary loan activity. The rate dropped to 13 percent in the first year of LionSearch, then to 10 percent of all undergraduate interlibrary loan borrowing activity in fiscal year 2012/13. To control for other variables that might have affected these data, we looked for changes in our user population and available resources. During this time period, there were no significant changes in the undergraduate student population. According to the Penn State Fact Book, the total undergraduate population across all campuses changed from 75,523 undergraduates in 2010 to 77,492 in 2013, less than 3 percent growth.15 Another possibility that could have contributed to the drop in cancelled requests for locally owned or licensed material was if the libraries had made major acquisitions over this time period, such as new online resources or collections. The libraries’ collections, both print and online, grew at a consistent rate of approximately 8 percent annually during this time period, from 5.8 million to 6.9 million volumes. An analysis of acqui- sitions of online resources during this time period found that the libraries acquired primarily unique content along with some online content for materials already owned in print. The latter content included both monographic and serial resources (that is to say, electronic books and periodical back files online). It is possible that some ILL FIGURE 1 The Number of Cancelled ILL Requests by Undergraduates for Locally Owned or Licensed Resources* 0 200 400 600 800 1,000 1,200 1,400 1,600 1,800 2010/11 2011/12 2012/13 Number of Cancelled ILL Requests by Undergraduates for Locally Owned or Licensed Resources Ar�cle Book 1,649 471 1,056 372 697 207 *The number of cancelled ILL requests due to local ownership dropped by 57% two years after implementing LionSearch. Impact of a Discovery System on Interlibrary Loan 649 requests submitted due to reason b) not in preferred format may have been mitigated by the purchase of this online content and thus may have contributed to the reduction in borrowing requests for locally owned or licensed material. However, it was not pos- sible to determine the impact of this duplicate online content with the data available. Borrowing Rates In addition to examining whether LionSearch improved retrievability of locally owned or licensed material as reflected by the change in numbers of cancelled requests, we wanted to determine if LionSearch was also improving discoverability of additional resources and whether this would be evidenced by a noticeable change in ILL bor- rowing activity. For example: • Did interlibrary loan borrowing requests increase, decrease, or remain constant after implementing LionSearch? • How did interlibrary loan borrowing request activity change for the under- graduate population in comparison to other populations? Overall, there was a 22 percent reduction in ILL borrowing requests four years after implementation, as shown in figure 2. While borrowing activity dropped overall, there was significant variation in the changes in borrowing rates among populations. As illustrated in figure 3, undergraduate requests showed a 26 percent reduction in the number of ILL borrowing requests during the inaugural year (FY2011/12) of LionSearch compared with the previous year. And, if this comparison is extended to FY2014/15, there was a 48 percent decrease. In contrast, graduate student and faculty requests had 20 percent and 11 percent reductions, respectively, over the same five-year period. Figure 4 illustrates that the major change in undergraduate borrowing was in article requests. By FY2014/15, there was a 71 percent reduction in article requests, whereas book requests had a 20 percent reduction. FIGURE 2 ILL Borrowing Requests* 0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 2010/11 2011/12 2012/13 2013/14 2014/15 *There was a gradual reduction in ILL borrowing requests for all users during this time period. 650 College & Research Libraries September 2016 Discussion At the beginning of this study, we hypothesized that implementing a discovery system would affect positive change in the findability of library-owned or -licensed resources if the number of interlibrary loan requests for these resources dropped following implementation. Such appears to be the case. For each year after the LionSearch implementation, there was a progressive reduction in the number of ILL requests by undergraduate students for locally owned or licensed material. Overall, there was a 57 percent reduction in those requests by undergraduates two years after implementation FIGURE 3 ILL Borrowing Requests by User Status* 0 5,000 10,000 15,000 20,000 25,000 30,000 Undergraduate Graduate Faculty/Staff *There was a continuous reduction in undergraduate ILL borrowing requests compared to faculty and graduate students. FIGURE 4 Undergraduate Borrowing Requests by Item Type* 0 2,000 4,000 6,000 8,000 10,000 12,000 2010/11 2011/12 2012/13 2013/14 2014/15 Book Ar�cle *There was a significant decrease in the number of undergraduate article requests compared to book requests. Impact of a Discovery System on Interlibrary Loan 651 of LionSearch, indicating that the discovery system had a significant positive impact on the findability of locally owned or licensed content for undergraduate students at Penn State. Data for undergraduate interlibrary loan borrowing requests four years after imple- mentation showed a large reduction variance between article (71%) and loan (20%) requests, implying that undergraduates were successful in discovering relevant article content in LionSearch to meet their needs and thus did not need to use interlibrary loan services for that material. In addition, by having the majority of library resources search- able from one interface, LionSearch is saving time for undergraduates in conducting their research, as well as reducing the time required for ILL staff to process requests. It appears that different discovery systems have different impacts on interlibrary loan activity. As noted by Asher et al., the ranking algorithm used by a given discov- ery service appears to be what matters most in determining discoverability of desired resources and concomitantly what resources users choose to use.16 The Summon dis- covery service, as implemented at Penn State, prioritizes retrieval of online and locally owned material; therefore, it was not unexpected to observe a reduction in interlibrary loan borrowing requests by undergraduates. The amount of the reduction, however, was startling. Although other libraries have implemented Summon, we were unable to obtain comparable data; therefore, additional studies investigating the impact of Summon on interlibrary loan are needed to determine whether the reduction in ILL borrowing requests experienced at Penn State is the norm. Impact on Interlibrary Loan Operations Over the years, and to varying degrees, changes in technology have reshaped interli- brary loan services and processes. The implementation of a web-scale discovery service is no exception to this pattern. LionSearch has had a positive impact on interlibrary loan operations. More patron article requests are imported into ILLiad with correct citation information. More important, the ISSN data imports into ILLiad from an ar- ticle request initiated within LionSearch. This facilitates requests being automatically searched in RapidILL, a resource-sharing system with expedited delivery developed by Colorado State University. RapidILL quickly returns article requests that can be fulfilled locally or sends the request on to another library for fulfillment. Though the quantity has lessened, we continue to receive requests for items that are available lo- cally and clearly marked as being accessible online. The reasons users continue to place such requests remain unclear, and further study is required to examine this behavior. With less time being spent on processing requests for locally owned or licensed material, ILL staff have more time to enhance and expand services to meet the needs of our changing user base. We were able to extend document delivery services to all Penn State users and began an office delivery service whereby loans are sent through campus mail to faculty and staff offices. In addition, ILL staff are taking the opportunity to interact with users in new ways by assisting at both physical and virtual service desks. Notes 1. Doug Way, “The Impact of Web-Scale Discovery on the Use of a Library Collection,” Serials Review 36, no. 4 (2010): 214–20; Tony Greiner, “How Does Switching to a Discovery Tool Affect Circulation?” (2011), available online at www.ala.org/acrl/sites/ala.org.acrl/files/content/confer- ences/confsandpreconfs/national/2011/papers/how_does_switching.pdf [accessed 17 September 2015]; Jody Fagan and Meris Mandernach, “Discovery by the Numbers: An Examination of the Impact of a Discovery Tool through Usage Statistics” (2011), proceedings of the Charleston Library Conference, available online at http://dx.doi.org/10.5703/1288284314967 [accessed 17 September 2015]; Lisa O’Hara, “Collection Usage Pre- and Post-Summon Implementation at the 652 College & Research Libraries September 2016 University of Manitoba Libraries,” Evidence-Based Library and Information Practice 7, no. 4 (2012): 25–34; J. Kemp, “Does a Web-Scale Discovery Make a Difference? Changes in Collections Use After Implementing Summon,” in Planning and Implementing Resource Discovery Tools in Academic Libraries, eds. M.P. Popp and D. Dallis (Information Science Reference, 2012), 456–68; Eugene Barsky, Sarah Jane Dooley, Tara Mawhinney, Zoey Peterson, and Michelle Spence, “Influence of Discovery Search Tools on Science and Engineering E-Books Usage,” Paper ID #5841, American Society for Engineering Education Annual Conference, Atlanta, Ga., 2013, available online at www.asee.org/public/conferences/20/papers/5841/download. [accessed 17 September 2015]; Michael Levine-Clark, John McDonald, and Jason Price, “Discovery or Displacement? A Large Scale Longitudinal Study of the Effect of Discovery Systems on Online Journal Usage,” plenary session for Charleston Conference 2013, available online at www.slideshare.net/jpricein/mlc-jdm- jsp-charleston-2013-slideshare-final-nxpowerlite [accessed 17 September 2015]; Michael Levine- Clark, John McDonald, and Jason Price, “Examining the Impact of Discovery Systems on Usage” (July 21, 2014), Exchanges blog, available online at http://exchanges.wiley.com/blog/2014/07/21/ examining-the-impact-of-discovery-systems-on-usage/ [accessed 17 September 2015]; Kristin Calvert, “Maximizing Academic Library Collections: Measuring Changes in Use Patterns Owing to EBSCO Discovery Service,” College & Research Libraries 76, no. 1 (Jan. 2015): 81–99. 2. John S. Spencer and Christopher Millson-Martula, “Discovery Tools: The Next Genera- tion of Library Research,” College & Undergraduate Libraries 19, no. 2/4 (2012); Andrew D. Asher, Lynda M. Duke, and Suzanne Wilson, “Paths of Discovery: Comparing the Search Effectiveness of EBSCO Discovery Service, Summon, Google Scholar, and Conventional Library Resources,” College & Research Libraries 74, no. 5 (2013): 464–88; Stephen Bull, Edward Craft, and Andrew Dodds, “Evaluation of a Resource Discovery Service: FindIt@Bham,” New Review of Academic Librarianship 20, no. 2 (2014): 137–66; Courtney Lundrigen, Kevin Manual, and May Yan, “‘Pretty Rad’: Explorations in User Satisfaction with a Discovery Layer at Ryerson University,” College & Research Libraries 76, no. 1 (Jan. 2015): 43–62; Jessica Mussell and Rosie Croft, “Discovery Layers and the Distance Student: Online Search Habits of Students,” Journal of Library & Information Services in Distance Learning 7 (2013): 18–39. 3. ARL Investment Index Rankings as of 2013–14, available online at www.arlstatistics.org/ analytics [accessed 17 September 2015]. 4. Scott Seaman, “Online Catalog Failure as Reflected through Interlibrary Loan,” College & Research Libraries 53, no. 2 (1988): 113–20; Catherine M. Dwyer, Eleanor A. Gossen, and Lynne M. Martin, “Known-Item Search Failure in an OPAC,” RQ 31, no. 2 (1991): 228–36; Elaine Yountz, Priscilla Williams, and Jane Anne Carey, “Interlibrary Loan Requests for Locally Held Items: Why Aren’t They Using What We’ve Got?” Journal of Interlibrary Loan, Document Delivery & Information Supply 11, no. 1 (2000): 119–28; Kevin O. O’Brien, “Cancelled Requests: A Study of Interlibrary Loan Requests for Items Held in a Medical Library Collection,” Reference Services Quarterly 23, no. 3 (2004): 17–23; Paoshan W. Yue and Millie L. Syring, “Usage of Electronic Journals and Their Effect on Interlibrary Loan: A Case Study at the University of Nevada, Reno,” Library Collections, Acquisitions and Technical Services 28, no. 4 (2004): 420–32; Lynn Wiley and Tina E. Chrzastowski, “The Impact of Electronic Journals on Interlibrary Lending: A Longitudinal Study of Statewide Interlibrary Loan Article Sharing in Illinois,” Library Collections, Acquisitions and Technical Services 29, no. 4 (2005): 364–81; Karen L. Janke, “The $64,000 Question Answered: Why Do Patrons Place ILL Requests for Items that the Library Already Owns?” available online at http://hdl.handle. net/1805/901 [accessed 17 September 2015]; Jessica R. Page and Jennifer Kuehn, “Interlibrary Ser- vice Requests for Locally and Electronically Available Items: Patterns of User, Users and Canceled Requests,” portal: Libraries and the Academy 9, no. 4 (2009): 475–89; Anne K. Beaubien, Jennifer Kuehn, Barbara Smolow, and Suzanne M. Ward, “Challenges Facing High-Volume Interlibrary Loan Operations: Baseline Data and Trends in the CIC Consortium,” College & Research Libraries 67, no. 1 (2006): 63–84; Rebecca M. Murphey and Judy T. Greenwood, “Feasibility of Analyzing Interlibrary Loan Requests to Determine Instruction Needs,” Journal of Access Services 6 (2009): 359–87; Thomas Deardorff and Heidi Nance, “WorldCat Local Implementation: The Impact on Interlibrary Loan,” Interlending & Document Supply 37, no. 4 (2009): 177–80; Kimberley Reycraft, “Interlibrary Loan Requests Canceled Due to Availability: Problem or Opportunity?” Journal of Interlibrary Loan, Document Delivery & Electronic Reserve 20, no. 1 (2010): 33–42; Nancy Kress, Darcy C. Del Bosque, and Thomas A. Ipri, “User Failure to Find Known Library Items,” New Library World 112, no. 3/4 (2011): 150–70; Megan Gaffney, “Interlibrary Loan Requests for Locally Available Materials: WorldCat Local’s Impact,” College & Research Libraries 73, no. 1 (2012): 68–77; Ruth S. Connell and Tschera H. Connell, “Teaching Patrons to Fish: The Educational Value of Cancelling Requests for Locally Available Materials,” Library Faculty Publications, Paper 32 (2014), avail- able online at http://scholar.valpo.edu/ccls_fac_pub/32 [accessed 15 September 2015]; Colleen S. Harris-Keith, “Evaluating the Staffing of an Interlibrary Loan Unit: An Exercise in Data-Drive Decision and Debunking ‘Anecdata’,” Journal of Access Services 11 (2014): 150–58; Beth Ashmore, Impact of a Discovery System on Interlibrary Loan 653 Emily Allee, and Rebekah Wood, “Identifying and Troubleshooting Link-Resolution Issues with ILL Data,” Serials Review 41, no. 1 (2015): 23–29. 5. Robb Waltner, “A Sardonic View of Interlibrary Loan,” Journal of Access Services 3, no. 1 (2005): 95–99; Yue and Syring, “Usage of Electronic Journals,” 430; Beaubien et al., “Challenges Facing High-Volume Interlibrary Loan Operations,” 77. 6. Dwyer et al., “Known Item Search Failure,” 232; Beaubien et al., “Challenges Facing High- Volume Interlibrary Loan Operations,” 77; Page and Kuehn, “Interlibrary Service Requests,” 481; Murphey and Greenwood, “Feasibility of Analyzing Interlibrary Loan Requests,” 365; Gaffney, “Interlibrary Loan Requests,” 73. 7. Deardorff and Nance, “WorldCat Local Implementation,” 178. 8. Gaffney, “Interlibrary Loan Requests,” 70, 73. 9. Zinthia C. Brice-Rosales, “OCLC WorldCAT: Interactivity and Mobility Create a Winning Combination,” in “Discovering What Works: Librarians Compare Discovery Interfaces,” Library Journal Reviews (Dec.7 2011), available online at http://reviews.libraryjournal.com/2011/12/refer- ence/discovering-what-works-librarians-compare-discovery-interface-experiences/ [accessed 15 September 2015]. 10. Barbara Coopey, e-mail message to Big Ten Academic Alliance ILL Directors’ electronic mailing list, August 2014. Five libraries responded but only two provided usable data. 11. Harris-Keith, “Evaluating the Staffing of an Interlibrary Loan Unit,” 155. 12. Calvert, “Maximizing Academic Library Collections,” 94. 13. Fagan and Mandernach, “Discovery by the Numbers,” 571; Amy Fry, “Discovery Systems: The Promise and the Reality: A Report of the RUSA MARS Local Systems & Services Program, American Library Association Annual Conference, Anaheim, June 2012” (2013), University Li- braries Faculty Publications Paper 11, available online at http://scholarworks.bgsu.edu/ul_pub/11 [accessed 15 September 2015]. 14. Barbara Coopey, e-mail message to author from a Penn State faculty member, October 2011. 15. Penn State Fact Book, available online at http://budget.psu.edu/factbook/default.aspx [ac- cessed 17 September 2015]. 16. Asher et al., “Paths of Discovery,” 471.