white.p65 502 College & Research Libraries November 1998 Cost-Benefit Analysis of Electronic Information: A Case Study Gary W. White and Gregory A. Crawford Library services and products have associated costs, including direct monetary costs and indirect costs such as time. The decision to acquire or provide a particular product or service should involve an examination of its costs and benefits to library customers. One technique for analyz­ ing cost-effectiveness is to perform a cost-benefit analysis (CBA). CBA involves analyzing the benefits, or potential benefits, of offering a prod­ uct or service and comparing them to the costs of offering that product or service. This article describes a study in which CBA was used to examine the cost-effectiveness of an electronic database. Librarians can use the results of CBA studies to justify budgets and acquisitions and to provide insight into the true costs of providing library services. ibraries make tremendous in­ vestments in the products and services they offer, especially in collections and electronic resources. Over the past decade, there has been a substantial increase in the reliance on electronic resources, including online public access computer systems (OPACs), CD-ROM databases, online services, document delivery systems, and the In­ ternet. Often the decision to acquire these types of services is made without com­ pletely understanding the actual costs involved and without knowing whether the new resources are economically bet­ ter choices than more traditional re­ sources (i.e., print). Furthermore, library budgets are often stagnant or shrinking, and librarians frequently are asked to jus­ tify these types of acquisitions and to prove their value. Librarians can use cost- benefit analysis for these purposes. What Is a Cost-Benefit Analysis? Cost-benefit analysis (CBA) has been used for years in business and industry as a way to measure productivity. Thus, most descriptions of CBA have this focus. One such definition is that CBA is the “meth­ odology in which all potential gains and losses from a proposal are identified, con­ verted into monetary units, and compared on the basis of decision rules to determine if the proposal is desirable.”1 This defini­ tion is strictly quantitative. More generally, CBA also can be defined as a measure that helps determine how the benefits of a product or service compare to its costs. There are two general types of CBA studies. First, CBA can be performed be­ fore undertaking a project and involves estimating costs and benefits. Second, CBA can be performed after a purchase or project and involves measuring past costs and benefits. Gary W. White is Business Reference Librarian in Heindel Library at Penn State Harrisburg; e-mail: gww@psulias.psu.edu. Gregory A. Crawford is Head of Public Services in Heindel Library at Penn State Harrisburg; e-mail: gac2@psu.edu. 502 mailto:gac2@psu.edu mailto:gww@psulias.psu.edu Cost-Benefit Analysis 503 FIGURE 1 Cost-Benefit Model Inputs Service Benefits Costs Value Businesses typically use one of three common CBA models: return on invest­ ment (ROI), present value analysis, or payback period. ROI models determine the amount of profit, or return, a product or service provides and compares this fig­ ure to its cost. Present value analysis models compare the cost of the product or service to its future estimated annual rate of return. Payback period models look at how long it will take before the profit or return pays for the cost of the product or service, and compares this time period to the estimated life of the product or service. Asking users directly is usually not reliable because they typically do not know the value of the informa­ tion or are biased toward overvalu­ ing their own information. There are several problems with apply­ ing any of these models in libraries. First, most libraries are not-for-profit organiza­ tions. All three of the CBA models rely on the use of the return or profit the prod­ uct or service generates. Second, the ben­ efits that arise from products or services in libraries are usually not quantifiable. How can “value” of a piece of informa­ tion be measured? How is it possible to put a figure on such nontangible benefits as user satisfaction or faster delivery of information? Moreover, measuring ben­ efits is a problem for several other rea­ sons. Asking users directly is usually not reliable because they typically do not know the value of the information or are biased toward overvaluing their own in­ formation. Accurately projecting benefits over a long time period also is difficult, if not impossible. Not only are the benefits difficult to measure, but the costs also are difficult to calculate accurately. Although direct costs (subscription costs, equipment costs, etc.) are typically easy to measure, indirect costs are much harder to ascertain. Indi­ rect costs include items such as staff time in assisting users, training and instruc­ tion time and materials, or troubleshoot­ ing and problem correction. In addition, the concepts of fixed, variable, and mar­ ginal costs need to be considered. Fixed costs are costs that do not change regard­ less of level of service or number of cus­ tomers. An example is the cost of keep­ ing a building open and running. Vari­ able costs are costs that increase with each level of output. For example, every time another page is photocopied, the costs for paper and toner increase. Marginal costs are related to variable costs and are the measure of each additional unit of out­ put. Finally, there is an inherent bias in looking at costs versus benefits. Whereas costs typically are more immediate and somewhat more accurately known, ben­ efits are much more difficult to measure and typically are spread out over a much longer time period. Cost-Benefit Analysis in Libraries What is the solution for performing CBA in a library? A framework for conducting CBA consists of the following compo­ nents: purpose, method, data collection, analysis of data, results, policy changes, and future studies (see figure 1). The first thing to consider is the reason for con­ ducting the cost-benefit analysis. What is expected to be determined from the 504 College & Research Libraries November 1998 study? Second, and most important, is the method for conducting the study. The method chosen will ultimately determine the type of data collected, their analysis, and the type of results generated. Data collection arises directly from the meth­ odology used. Results are a function of method and data collection. In addition to satisfying the purpose of the study, re­ sults potentially can lead to meaningful policy changes and ideas for future study. CBA requires a study of both costs and benefits, or potential costs and benefits, of a product or service. As mentioned ear­ lier, direct costs are relatively easy to iden­ tify. However, indirect costs are just as im­ portant to identify. Factors such as time, tangential costs such as paper or ink car­ tridges (or any other somewhat “hidden” costs), costs for training and materials, or any other factors that add to the cost of providing a service or product are con­ sidered indirect costs. An exact figure for indirect costs is extremely difficult to cal­ culate. Estimates of the true costs, both direct and indirect, are necessary to more accurately calculate the total cost of the product or service. Some data, such as salaries and fringe benefits, may be diffi­ cult to obtain due to employee confiden­ tiality. Measuring benefits in a not-for-profit environment can be even more difficult. Basically, there are two schools of thought: attempt to place a dollar figure on the benefits derived, and/or measure benefits as a decrease in costs in other areas. The accuracy of the first method is somewhat questionable, so the second method is usually the primary choice. Most CBAs use a blend of both methods. Literature Review Most studies of the costs and benefits of library services have focused on the col­ lections and staffing of libraries. In his review of the cost analysis literature of librarianship, Colin K. Mick divided the studies he found into four types: those that study a function or service, those that look at the organization, those that exam­ ine the structure within which the library is located (university, town, or corpora­ tion), and those that compare costs across similar types of libraries.2 However, his review predates the advent of electronic services within libraries. A. Craig Hawbaker and Cynthia K. Wagner performed a CBA in which they compared the costs of providing full-text access to two full-text business databases, Business ASAP and Business Index, with the costs of owning or subscribing to the periodicals included in these sources.3 They found that providing access to both indexes increased access from the 242 periodicals to which their library sub­ scribed to 513 periodicals at a cost increase of approximately 15 percent. This trade- off of increased costs versus increased access is an issue that every library must address. Hawbaker and Wagner indi­ cated that other issues, such as savings from increased shelving availability or increased costs from computer hardware and software, also should be studied. The basic research question was: Are the expenses of a full-text database justified when compared to tradi­ tional methods of information delivery? David Everett analyzed interlibrary loan (ILL) requests in his library for one year to determine whether providing ac­ cess to periodicals included in a number of full-text databases would be more ef­ fective in providing information re­ quested by library patrons.4 He found that the full-text sources provided access to less than 4 percent of the titles requested. Although this percentage has certainly increased (as the present study shows), the comparison of traditional ILL to full- text sources is an important consideration in the decision to access full-text sources. In a 1989 article, Paul B. Kantor pre­ sented a method for performing func­ tional cost analysis.5 He emphasized the costs of materials and services that are allocated to a set of library functions which are of direct service to library us­ Cost-Benefit Analysis 505 ers. He then used these functional costs to reconcile the entire library budget for the purpose of planning, management, and budget justification. Thus, he in­ cluded costs such as salaries, space, over­ head, and materials with most attention given to books, yet he stated that a simi­ lar methodology could be used for com­ puter resources, audiovisual equipment, and online databases. In 1993, Marianne Broadbent and Hans Lofgren wrote about applying priority and performance evaluation and CBA to current awareness services in Australian information centers.6 And in a more gen­ eral article, Wendy Smith discussed the “cost-benefit potential” of fee-based ser­ vices, including online searching and ILL, in libraries.7 Richard W. Meyer did a cost-benefit comparison of mediated searching, end- user searching, and locally mounted da­ tabases at Clemson University.8 He com­ puted a cost-per-hour and cost-per-search figure for each electronic resource used in the library and then compared each fig­ ure to a baseline established in the library to determine which resources would be offered. Marilyn M. Browning and Leslie M. Haas performed a cost analysis and user survey of Business Periodicals Ondisc (BPO).9 They analyzed the costs of BPO, including subscription costs, supplies, and royalty fees (which are no longer in­ cluded in the price of BPO). They also addressed various methods of paying for the database—through donations or charging patrons for printing, or by cut­ ting costs, for example, through periodi­ cal cancellations. A survey showed that 78 percent of BPO users were business majors. Some disadvantages addressed included system downtime, system crashes, and length of time to print ar­ ticles. Advantages discussed included ease of use, quality of the final product, and convenience. Adele F. Bane also conducted a user survey and discussed the impact of ac­ quiring BPO in her library.10 Users iden­ tified the following items as positive as­ pects: availability of full-text articles, timely access, ease of use, access to a large number of journals, immediate feedback, and lengthy abstracts. The downsides included system downtime, time limits (imposed by the library), lack of down­ loading capability, lack of total full-text availability (approximately 50% of the indexed articles are full text), lack of re­ mote access, limited time coverage (full­ image coverage begins with 1987 journal issues), and the manual loading of CDs. She also looked at the impact on ILL in her library. In 1994, BPO provided access to approximately 12,000 articles that oth­ erwise would have been acquired through ILL. Bane estimated that this would have cost $360,000 that year, whereas BPO costs approximately $20,000 per year. Other issues she addressed were the potential for periodical cancellations, increased expectations of users (especially the reluctance to return to print indexes), and additional labor in training and us­ ing electronic resources. Methodology To better understand the costs and ben­ efits of full-text resources, the authors undertook a study funded by a grant from the Penn State Harrisburg Faculty Re­ search Council to see how the addition of the BPO system had affected requests for periodical articles through ILL. The basic research question was: Are the expenses of a full-text database justified when com­ pared to traditional methods of informa­ tion delivery? In 1994, the Heindel Library acquired the full-image ABI/Inform database BPO from UMI. The BPO database consists of the digitally scanned images of articles from more than 400 business journals and magazines on a series of CD-ROMs. The library hoped to achieve two objectives through this acquisition: to increase ac­ cess to business journals, and to reduce ILL traffic. The acquisition of BPO in­ creased the library’s access to full-text business journals from approximately 210 (to which it subscribed to the print ver­ sions) to 425. http:library.10 506 College & Research Libraries November 1998 FIGURE 2 ILL Trends 86 87 88 89 90 91 92 93 94 95 96 0 2,000 4,000 6,000 8,000 10,000 12,000 86 87 88 89 90 91 92 93 94 95 96 BPO begins As part of the project, the library de­ cided to examine trends in its interlibrary loans and to analyze requests for 1993 (the full year prior to BPO) and 1995 (the full year after BPO). Because BPO was brought in midyear, the library decided not to include 1994 for fear that it would complicate and distort the findings. Us­ ing an estimated cost for each ILL trans­ action, the library wanted to calculate its ILL savings (if any) and compare the ben­ efits to the costs associated with subscrib­ ing to BPO. Access to the ABI/Inform da­ tabase also was a factor. The library of­ fered ABI/Inform on a single workstation until August 1993, at which time it became available on its OPAC. By 1995, ABI/In­ form was available through the library’s CD-ROM network, on the OPAC, and through BPO. Interlibrary loan is a major public ser­ vice point in the Heindel Library of Penn State Harrisburg and is used heavily, es­ pecially by the school’s graduate stu­ dents. Over time, the number of requests for materials through ILL had been grow­ ing steadily, as shown in figure 2. Total ILL transactions for each year from 1986 through 1996 were gathered from statis­ tical information kept in the library. Thus, the total number of ILL transactions was easy to gather and analyze. These figures were used to provide an overall picture of the library’s ILL activity, both before and after the introduction of electronic full-text databases. Next, every ILL re­ quest for periodical articles from 1993 and 1995 was analyzed individually to deter­ mine whether the publication was in­ dexed on the ABI/Inform database for that particular year. ABI/Inform indexed more than 800 journals and magazines in 1993 and more than a thousand titles in 1995. The total number of interlibrary loans for materials indexed in ABI/In­ form was gathered and compared to the total number of ILL requests for the in­ stitution for each year in the study. Ad­ ditional cost data gathered consisted of ILL costs, subscription costs to ABI/In­ Cost-Benefit Analysis 507 FIGURE 3 ILL Requests for Materials Indexed in ABI/Inform • 199q: 922 (19.4% of article requests) 4,764 total article requests • 1995: 464 (11.8% of article requests) 3,946 total article requests scribing to the paper cop­ ies versus the cost of access­ ing the articles through BPO for which ILL requests had been made. In 1993, the cost to subscribe to the titles from which ILLs were requested was $69,414.35. form, costs of copyright compliance, and costs of subscribing to the journals from which interlibrary loans were ob­ tained. Results In 1993, the Heindel library made 4,764 ILL requests, of which 922 (19.4%) were from titles indexed in ABI/INFORM. In 1995, the library made 3,946 ILL requests, of which 464 (11.8%) were titles from ABI/ INFORM (see figure 3). Thus, the data show that acquiring BPO cut the library’s interlibrary loans for ABI/INFORM titles by about 50 percent. A 1993 report from the Association of Research Libraries/Research Libraries Group estimated that the average trans­ action cost for an ILL was $18.62.11 Using this figure as a standard measure, in 1993, the cost to acquire articles indexed in ABI/Inform through ILL was $17,168. In 1995, this figure declined to $8,640, a sav­ ing of $8,528. The library also paid less for copyright compliance for ABI/INFORM titles. In 1993, copyright payments were $847.45 for 135 articles compared to $285.65 for 33 articles in 1995, a saving of $563. ILL costs and copyright saving constituted a total ILL saving of $9,091 (see figure 4). The cost of the journals acquired through BPO can be considered a poten­ tial cost saving; that is, the cost of sub- FIGURE 4 Cost Savings (Library Benefit) • Copyright: $563 savings • ILL cost: $8,528 savings • Total savings: $9,091 In 1995, this cost increased to $75,726.37, although fewer articles were requested. This result probably can be attributed to the increased access to a va­ riety of new journals because the cover­ age of BPO and ABI/Inform increased greatly over this period. Thus, students were finding more articles from journals that were not included in BPO in full text and to which the library did not subscribe. The major cost increase the library had to absorb initially was the actual subscrip­ tion cost. In 1993, the library subscribed to the networked CD-ROM version of ABI/INFORM at a cost of $5,250. In 1995, the total cost for BPO (which also in­ cluded the networked CD-ROM version of ABI/INFORM) was $15,950, or a net increase of $10,700. The cost of BPO over the library’s three-year contract also in­ cluded a computer and a laser printer, which potentially lowered the overall cost increase because the computer could be used for other functions. Discussion There are several ways to look at the data for total costs and benefits. First, the li­ brary can look strictly at the impact on ILL and ignore the potential journal costs. Because it was highly unlikely that the library would have subscribed to the ad­ ditional 200+ journals in paper copy, per­ haps this is the more accurate measure. Given this, the library saw a cost increase of $10,700 as compared to a benefit of about $9,091. This is a comparison of di­ rect costs versus direct benefits. At first glance, and looking strictly at the num­ bers, it appears that the library lost more (in terms of dollars) than it gained. How­ ever, several other issues must be consid­ ered. First, the present value of the com­ puter equipment also must be included. http:75,726.37 http:18.62.11 http:69,414.35 508 College & Research Libraries November 1998 The estimate for 1995 was that the com­ puter and the printer were worth at least the $1,700 difference. Even without con­ sidering the cost of the equipment, the library more than doubled its access to business-related journal titles (425 versus 210) at a direct cost of only $1,700, about $7.91 per title. One of the main questions the library now receives is: “Why isn’t everything full text?” Another way to examine the data from this study is to include the potential costs of the journals for which ILL requests had been made previously. Such an analysis assumes that the library would have had to subscribe to the requested journals to provide the same level of service. When these amounts are entered into the analy­ sis, the total saving for 1995 increases greatly and can be estimated at more than $65,000. If these data were extended to the other years that BPO has been available in the library, it would have paid for it­ self many times over. However, this type of analysis provides only a measure of po­ tential benefits because most libraries can­ not afford to subscribe to all the journals from which articles are requested by ILL. Implications This research has several implications. First, expenses often are not reduced when new services are offered which, on the surface, are thought to provide cost reductions; they are merely shifted to a different line of the budget. The subscrip­ tion to BPO is covered by the materials budget whereas most of the costs of ILL are paid for through the operating bud­ get. Although the library did not save money directly by subscribing to BPO, users may have received many benefits that were not examined by this research. However, new services such as BPO also may increase customer expectations. One of the main questions the library now re­ ceives is: “Why isn’t everything full text?” In addition, such systems increase the complexity of the library. Each new sys­ tem seems to work differently from all the others in the library, leading to confusion for both library customers and librarians, who must cope with myriad different search interfaces and indexing practices. In addition, several nontangible ben­ efits must be considered. First, the full- text articles for the journals to which the library did not subscribe were accessible immediately (as opposed to having to wait one or two weeks for ILL). In infor­ mal surveys, patrons reported great sat­ isfaction with the service. Thus, BPO pro­ vides additional benefits that would be difficult to measure in terms of dollars, including a decrease in ILL work for li­ brary staff and wait time for articles, as well as more satisfied patrons and more time to devote to ILL requests for other items. As a result of these data, the library concluded that acquiring BPO proved to be a good economic choice. Conclusions Cost-benefit analyses, as has been shown through this case study, can support the contention that the choice to offer a prod­ uct or service is economically sound. In the case of providing BPO, the library hoped merely to break even or at least not spend too much additional money in pro­ viding the service. Sometimes the cost of a new product or service may not out­ weigh its direct benefits. In such cases, librarians must decide whether the nontangible benefits of offering the prod­ uct or service are worth the potential added costs. As with any new venture, there is some inherent risk that the deci­ sion may not meet expectations. CBA also can be used as a marketing tool, which is an extremely important function. The addition of a new product or service usually involves a significant initial investment of money and time. All librarians have run into the problem of administrators who are reluctant to part with money, especially as budgets shrink and costs grow. CBAs, even those per­ formed by other institutions, are useful in proving to administrators that even though a new product or service may cost a lot up front, the returns and/or cost sav­ ings in other areas can easily outweigh the initial expense. At Penn State Harris­ burg, the results of the study are being used to justify the acquisition of other ex­ pensive full-text products. Although these Cost-Benefit Analysis 509 findings may be insufficient to persuade some administrators, they do lend cred­ ibility to the argument and also serve to show administrators, faculty, and the public that librarians are capable of mak­ ing appropriate, cost-effective decisions. Notes 1. Tefik F. Nas, Cost-Benefit Analysis: Theory and Application (London: Sage, 1996), 1–2. 2. Colin K. Mick, “Cost Analysis of Information Systems and Services,” Annual Review of Information Science and Technology 14 (1979): 37–64. 3. A. Craig Hawbaker and Cynthia K. Wagner, “Periodical Ownership versus Fulltext On­ line Access: A Cost-Benefit Analysis,” Journal of Academic Librarianship 22 (Mar. 1996): 105–10. 4. David Everett, “Full-Text Online Databases and Document Delivery in an Academic Li­ brary: Too Little, Too Late?” Online 17 (Mar. 1993): 22–25. 5. Paul B. Kantor, “Library Cost Analysis,” Library Trends 38 (fall 1989): 171–88. 6. Marianne Broadbent and Hans Lofgren, “Information Delivery: Identifying Priorities, Per­ formance, and Value,” Information Processing and Management 29 (Nov./Dec. 1993): 683–701. 7. Wendy Smith, “Fee-based Services: Are They Worth It?” Library Journal 118 (June 1993): 40–43. 8. Richard W. Meyer, “Locally Mounted Databases: Making Information as Close to Free as Possible,” Online 16 (Jan. 1992): 15, 17–24. 9. Marilyn M. Browning and Leslie M. Haas, “Is Business Periodicals Ondisc the Greatest Thing Since Sliced Bread? A Cost Analysis and User Survey,” CD-ROM Professional 4 (Jan. 1991): 37–41. 10. Adele F. Bane, “Business Periodicals Ondisc: How Full-Text Availability Affects the Li­ brary,” Computers in Libraries 15 (May 1995): 54–56. 11. Marilyn M. Roche, ARL/RLG Interlibrary Loan Cost Study (Washington, D.C.: ARL, 1993), 4.