mccarthy.p65 128 College & Research Libraries March 1997 128 Expectations and Effectiveness Using CD-ROMs: What Do Patrons Want and How Satisfied Are They? Cheryl A. McCarthy, Sylvia C. Krausse, and Arthur A. Little As academic libraries wrestle with how to provide more CD-ROMs and more online remote access to databases to keep pace with demands, they first need to evaluate the effectiveness of their current services. This study identifies students’ preferences and effectiveness using CD- ROMs and assesses the whole CD-ROM environment at one univer- sity library. A questionnaire was used to ask students their preferences, confidence levels, skills, and training in searching CD-ROM databases, as well as their satisfaction with CD-ROM services (see the appendix). Analyses of the data revealed that users are satisfied and prefer CD- ROM databases over the print indices. Although students indicated they are confident searching, they admitted that they need to know basic search strategies and that they want more personal assistance, hands- on training, and remote online access to databases. Moreover, how can academic libraries meet increasing user demands for more data- base access and services when their resources are not increasing? ince CD-ROM database prod- ucts first emerged in academic libraries in 1986, they have pro- liferated and so too have stud- ies investigating this technology. How- ever, a literature review revealed that most articles on CD-ROM use are lim- ited in scope, focusing on a single data- base or a single issue, and involve only a limited population of users. The most common method of inquiry is the sur- vey instrument. Although academic li- braries have invested heavily in technol- ogy and online services for the past ten years, the demand for more end-user computer services continues to increase. Along with the new technology, it ap- pears that library users expect more as- sistance and training. Until recently, aca- demic libraries continue to purchase CD-ROM products at a growing rate without first carefully evaluating the ef- fectiveness of their current services. As academic libraries continue to face finan- Cheryl A. McCarthy is an Assistant Professor in the Graduate School of Library and Information Studies at the University of Rhode Island; e-mail: chermc@uriacc.uri.edu. Sylvia C. Krausse is Professor and Chair of Public Services at the University of Rhode Island; e-mail: skrausse@uriacc.uri.edu. Arthur A. Little is a doctoral student at the University of Rhode Island who provided statistical data analyses and input to the design and methodology section; e-mail: alit1837@uriacc.uri.edu. Expectations and Effectiveness Using CD-ROMs 129 cial constraints and staff shortages in the 1990s, how are they going to respond to the increasing demands of users in light of decreasing resources? A recent study on effective library use at the University of Rhode Island (URI) revealed that students most frequently request better availability of materials and more assistance and training, espe- cially on CD-ROM databases.1 Thus, this study is a follow-up investigation of stu- dents� expectations, effectiveness, and satisfaction with CD-ROM services at the URI Library. During spring semester 1995, training sessions on CD-ROM databases were provided for URI students in Writing 101. The goal of the training was to produce effective and independent searchers on CD-ROM products. Students in these ses- sions were surveyed to assess the train- ing they received. In addition, all users of CD-ROM databases at the URI Library during the month of April 1995 were asked to complete a survey to evaluate their needs, skills, and satisfaction using CD-ROM products and services. All re- sponses were voluntary and anonymous. The CD-ROM Environment at the University of Rhode Island Library The process of acquiring CD-ROMs at URI began in 1987 as a joint purchase of Compact Disclosure with the Business School. Because of the popularity of this first CD-ROM, the reference librarians evaluated the library�s DIALOG and BRS online services in order to justify the pur- chase of additional CD-ROM databases. After an evaluation of user statistics, it became apparent that the most popular and frequently requested online data- bases were MEDLINE, PsycLit, and ERIC. Based on these findings, a CD- ROM policy was created that addressed the priorities for purchasing reference CD-ROM databases. The reference librar- ians recommended the purchase of MEDLINE, PsycLit, ERIC, the Govern- ment Documents Catalog Service, MLA, and UMI�s Periodical Abstracts as the first tier of CD-ROM databases. During the following three years, the library gradually added four of the Wil- son titles and additional workstations. In 1990, LANTASTIC was acquired and ten workstations were networked. The popu- larity of the CD-ROMs skyrocketed and students queued up to use them, even refusing to use the print versions. Mul- tiple-user subscriptions had to be pur- chased to ease the congestion. A declin- ing budget was stretched to purchase more hardware and multiple-user li- censes. In 1992, Sylvia Krausse wrote a grant proposal to equip a CD-ROM room with appropriate workstations and net- worked printers. The Champlin Foun- dations granted the proposal and pro- vided funding for a local area network (LAN), connecting eighteen worksta- tions with two servers and several CD- ROM towers. The library added more subscription databases, namely UMI�s Newspaper Abstracts, Dissertation Ab- stracts, Newsbank, CINAHL, PAIS, Bio- logical Abstracts, Engineering Index, Textile Technology Digest, America: His- tory and Life, Historical Abstracts, and Moody�s International. In the fall of 1993, a LAN of CD-ROM workstations was officially connected; and by the spring of 1995, there were twenty-two CD-ROM databases net- worked using LANTASTIC. Ever since the first tier of CD-ROMs was acquired, statistics showed a steady increase in CD- ROM use. With the addition of the CD- ROM room and more requests for assis- tance, it became apparent that during the busiest hours of the day full staffing of the CD-ROM room was a necessity. With the opening of the CD-ROM room, stu- dents were hired between the hours of eleven in the morning and two in the af- ternoon. Because of the pressing demand for assistance in the room, one hour was added each semester until the CD-ROM room was fully staffed from ten in the 130 College & Research Libraries March 1997 morning until five in the afternoon, and again from six to nine in the evening during the week and on the weekend during the majority of the hours that reference service was provided. In addition to personal assistance, every two workstations share an instruc- tional notebook that contains one-page instructional guides by database as well as by interface (e.g., Silverplatter, UMI, or Wilson). During the times that the room is not staffed, patrons either use these binders or go to the Reference Desk for additional help. It was soon evident that although users want assistance or instruction at point of use, the librarians also would need to offer additional training sessions in a classroom setting to larger groups to ease the demand in the CD-ROM room for one-on-one in- struction. Thus, general hands-on train- ing sessions were offered in addition to the instruction during the regular bib- liographic instruction classes. Literature Review A survey of the literature reveals that the introduction of CD-ROMs in academic libraries has created an increase in de- mand for point-of-use assistance by the reference staff. With the technology ex- plosion in libraries, the number of work- stations, databases, and queries for as- sistance have multiplied in reference departments, but the number of staff has not increased accordingly.2 Moreover, the increases in CD-ROM products and the on-demand requests for training may be affecting the quality of refer- ence services and staff morale, and may even cause staff burnout.3 In one research study on bibliographic in- struction methods on CD-ROM data- bases, Dorothy F. Davis offered a prac- tical solution to alleviate staff stress: �librarians must provide more effective group instruction or offer self-instruc- tional alternatives.�4 Vendors, however, have promoted their products for end users, claiming little or no assistance is needed from pro- fessional staff. Although these products are relatively easy for end users, most patrons are not familiar with the multi- tude of subject-specific databases and/ or do not know which one(s) to select. Students faced with multiple databases are often overwhelmed by the many choices and need to seek help. CD-ROM products, however, do offer a cost-effec- tive way for users to search databases compared to mediated online searching, which is usually costly and time-con- suming. Thus, several studies have ad- dressed the increased demands for assis- tance using CD-ROMs and identified the impact of CD-ROM products or services on libraries and staff.5 Other CD-ROM studies have evalu- ated specific training or bibliographic instruction programs in academic librar- ies.6 Some studies have investigated user perceptions about, or satisfaction with, using specific CD-ROM products.7 On the questionnaires used to assess CD- ROM use, most respondents revealed a high degree of user satisfaction. A study titled �Value-added Bibliographic In- struction� by Stanley D. Nash and Myoung Chung Wilson showed that al- though students were satisfied with their searches, an analysis of their searches revealed that more than one-third of the citations retrieved by students were use- less or inappropriate for their topics.8 Although these studies contributed to a broader understanding of user needs and satisfaction with CD-ROMs, profes- sional concerns continue to grow about these products. Without a standard for CD-ROM database products, each ven- dor develops a unique interface system, code, terminology, and search strategy Thus, general hands-on training sessions were offered in addition to the instruction during the regular bibliographic instruction classes. Expectations and Effectiveness Using CD-ROMs 131 that is not easily transferred from one to the other by users. Students are often confused about which databases to use in order to find appropriate sources for their research needs. Although some in- terfaces provide easier search strategies, many undergraduates are unaware of specialized subject databases and are fa- miliar only with one general CD-ROM database from their high school or pub- lic library experiences. Furthermore, they are unfamiliar with the differences in vendor or interface systems, serials in- dexed, or thesauri. The literature review provided insights from numerous stud- ies addressing some of these concerns related to CD-ROM use and instruction, but none of the investigations assessed the total CD-ROM environment in an academic library. One significant investigation, how- ever, assessed the CD-ROM environment (fifteen commercial products networked) at the University of North Carolina at Chapel Hill and provided the most broad-based perspective.9 The goal of that study was to evaluate the CD-ROM �system�s success to gather information that would enable the staff to evaluate and improve the service.�10 Although an electronic survey was used, apparently no further inquiry was performed to as- sess either user training or satisfaction with CD-ROM products and services. Utility of Study What follows, therefore, is an attempt to build on these previous studies by evaluating the whole CD-ROM envi- ronment at one academic library at URI. The goal of this study is to deter- mine users� preferences and needs, their confidence in searching, the ef- fectiveness of their training, and their overall satisfaction with CD-ROM ser- vices. The authors hope that this study will help library staff to evaluate their CD-ROM environment and to plan im- provements in the total electronic deliv- ery of databases. Design and Methodology This study is the result of a collaboration between the principal investigators, Cheryl McCarthy and Sylvia Krausse. Arthur Little, a doctoral student, per- formed statistical analyses of the data from the questionnaires and provided input to the Design and Methodology section. This study was designed as a needs assessment for users of CD-ROM data- bases at the URI Library. After an initial study indicating student demands for more instruction using CD-ROMs, the ref- erence librarians initiated a series of hands- on CD-ROM training sessions for fresh- men in Writing 101 classes in the library�s new electronic classroom (ECL).11 A sur- vey was designed to evaluate both CD- ROM needs and services for students in CD-ROM training classes, as well as for a sample of students using the CD-ROM services during the spring of 1995. The specific research objectives for this study were: 1. to determine users� preferences in searching CD-ROM databases versus print indices; 2. to assess users� confidence in their searching skills; 3. to identify what users need to be- come more effective searchers on CD- ROMs; 4. to assess effectiveness of CD-ROM training sessions; 5. to assess users� satisfaction with CD- ROM services. An eleven-item questionnaire address- ing these objectives was pretested and revised to eliminate ambiguity in lan- guage and meaning. This process pro- vided both a reliability and a validity check to ensure that the survey items re- flected the research objectives of the prin- cipal investigators. A total of 750 ques- tionnaires were distributed�to both the students in the Writing 101 classes who attended the training and everyone who used the CD-ROM facility during the month of April 1995. A return box was 132 College & Research Libraries March 1997 provided at the CD-ROM desk, and 489 completed questionnaires were returned. No attempt was made to survey either nonusers or users of print indices in other locations. Data Analysis Analyzing the questionnaire data, using both quantitative and qualitative mea- sures, provided the researchers with an in-depth interpretation of the five re- search objectives. Quantitative statistical methods were used to evaluate the nu- merically measurable survey variables. These included demographic and usage information, database preference infor- mation, and three five-point Likert scales to measure perceived CD-ROM confi- dence levels (question five), satisfaction levels (question eleven), and effectiveness of training (question ten�applied to Writing 101 groups). Qualitative means were used to analyze open-ended follow- up queries to the Likert scale questions. These follow-up items were phrased in the form of �Please explain� or �Why?� Content analysis was chosen as one way to analyze these data because of its ad- vantages �in making inferences by ob- jectively and systematically identifying specified characteristics of messages� into categories.12 One advantage in us- ing content analysis is that specified char- acteristics occurring in open-ended ques- tions can be quantitatively identified, studied, and rank-ordered to assess com- mon traits or problems. By employing both qualitative and quantitative proce- dures in content analysis, insights can be gleaned from both what is seen and not seen, and from what is reported and not reported. Moving between the quantita- tive statistical analysis and the qualita- tive content analysis, the investigators gained insights into the full meaning of the data. Techniques for the coding, dis- playing, and verifying of data are adapted from recommendations by Ole Holsti.13 The principal investigators also made unobtrusive observations of CD- ROM users. Thus, a process of triangu- lation was used to achieve �intersubjectivity� and full insight into the meaning of the data.14 Furthermore, the investigators were able to draw in- ferences into students� effectiveness in and satisfaction with training and CD- ROM services. The quantitative data from the 489 completed questionnaires were con- verted into machine-readable form and imported into the statistical analysis package SPSS/PC+. The data were then tabulated and examined for accuracy and for �out-of bounds� conditions, which were then corrected. For example, twenty of the questionnaires were completed by nonstudents, so they were removed from the analysis. The responses from the five- point confidence scale were collapsed into the following three categories so that a two-way contingency table using this variable would have no expected fre- quency below five for any cell: �Less Confident� combined responses one and two; �Moderately Confident� represents response three; and �More Confident� combined responses four and five.15 Prior to gathering these data, the investigators set their alpha level to .05 to reduce the probability of a Type I error. In order to test the degree of association between preferences for the CD-ROM databases and their equivalent print indices, the number of responses for each database was summed. These sets of values were then ranked for each medium and a Spearman�s Rank Order Correlation Co- efficient (Spearman�s Rho, r s ) between the two sets of ranked databases was calcu- lated using the formula r s = 1-((6D^2)/ (N(N^2-1))).16 The principal investigators used con- tent analysis to code responses from the The principal investigators also made unobtrusive observations of CD-ROM users. Expectations and Effectiveness Using CD-ROMs 133 open-ended questions and observations. The notes were coded into categories adapted from the responses. Categories were collapsed to combine similar com- ments into broader subject areas. To en- sure intercoder reliability, data coded by one principal investigator were verified by the other. Interpretation of Data User Profile Of the 489 users, 79 percent were under- graduate students, 17 percent were graduate students, and 4 percent were faculty, staff, or other. Of the undergradu- ates, 50 percent were upper level and 29 percent were lower level. Thus, the sur- vey respondents reflect the overall uni- versity population of approximately 80 percent undergraduates, who were the intended target audience for this study. When asked if this was the first time us- ing CD-ROM databases this academic year, 16 percent indicated yes, compared to 71 percent who said they had used the databases between two and ten times this academic year and 29 percent who had used them more than eleven times. Thus, with the exception of the freshmen in Writing 101, most students surveyed were experienced CD-ROM users, hav- ing used them previously this year (see tables 1 and 2). User Preferences in Searching CD-ROMs To determine users� needs and prefer- ences in searching CD-ROM databases versus print indices, the questionnaire asked them to identify the CD-ROM da- tabases or print indices most frequently used from the list of the twenty-two ver- sions. Table 3 shows the rank order of databases and print indices selected and the Spearman�s Rho comparing the strength of the relationship between the CD-ROM version and the print version. The Spearman�s Rho shows a high level of agreement between the two database sets indicating a clear pattern of similar database preferences, which is indepen- dent of format. It is also curious to note TABLE 1 Survey Participants, Reported Academic Status Question 1: “Please mark your academic status below.” N Percent Undergrad.–Yr. 1 110 22.49% Undergrad.–Yr. 2 33 6.75 Undergrad./lower total 143 29.24 Undergrad.–Yr. 3 98 20.04% Undergrad.–Yr. 4 126 25.77 Undergrad.–Yr. 5 20 4.09 Undergrad./higher total 244 49.90 Graduate–MA/MS 57 11.66% Graduate–Ph.D. 25 5.11 Graduate students total 82 16.77 Faculty 5 1.02% Staff 6 1.23 Other 9 1.84 Nonstudent total 20 4.09 Total 489 100% TABLE 2 Survey Participants, Reported CD-ROM Usage Question 2: “Is this the first time you have used the CD-ROM databases?” N Percent Yes 77 15.75% No 412 84.25 Total 489 100% Question 3: “How many times have you used the CD-ROM databases this year?” N Percent 2–5 times 165 40.05% 6–10 times 128 31.07 11–20 times 62 15.05 21 or more 57 13.83 Total 412 100% 134 College & Research Libraries March 1997 that students indicated using a print ver- sion of Disclosure, despite the fact that no print version exists. The most frequently cited CD-ROM databases were similar to the print ver- sions, revealing the top eight preferences as Social Sciences Index and Periodical Abstracts, closely followed by Humani- ties Index, ABI Inform, PsycLit, News- paper Abstracts, Newsbank, and ERIC. Although the rank order was similar, the CD-ROM versions were cited at least three times more frequently than the print versions. For example, the number one CD-ROM database preferred was Social Sciences Index cited by 104 users, whereas the print version was the sec- ond highest preferred and cited by only twenty-three users. No print version was cited more frequently than its compa- rable CD-ROM version. Furthermore, when users were asked which version they prefer, 85 percent chose the CD-ROM version compared to 11 percent for print, 4 percent checked both. The most frequently cited reasons given for CD-ROM preference by 77 per- cent of respondents include: easier to use, better subject access, faster and more ef- ficient search process. An additional 8 percent of respondents preferring CD- ROMs gave the following reasons: can get a printout of sources, provides ab- stracts, can download, provides more TABLE 3 Rank Order Preference of Databases Question 4 presented a list of 22 CD-ROM and print databases, and the participants were asked to indicate which they used most often. The responses were summed for each database by type of medium and then rank ordered. A Spearman Rank Correlation Coefficient (rs) was computed to find the association between the format: rs = +.954, p<.001. CD-ROM Database N Rank Print Database N Rank Social Sciences Index 104 1 Periodical Abstracts 28 1 Periodical Abstracts 89 2 Social Sciences Index 23 2 Humanities Index 81 3 ABI Inform 20 3 ABI Inform 71 4.5 Newspaper Abstracts 19 4 PsycLit 71 4.5 PsycLit 18 5 Newspaper Abstracts 69 6 ERIC 16 6 Newsbank 64 7 Newsbank 14 7 ERIC 58 8 Humanities Index 13 8 Disclosure 45 9 Moody’s International 12 9 Moody’s International 43 10 MLA 11 10.5 Medline 42 11 Medline 11 10.5 PAIS 34 12.5 Disclosure 10 12 Biological Abstracts 34 12.5 Biological Abstracts 9 13.5 MLA 32 14 PAIS 9 13.5 Historical Abstracts 27 15 American History & Life 8 15 Dissertation Abstracts 26 16 Historical Abstracts 7 16 Applied Science & Tech. Index 25 17 Applied Science & Tech. Index 6 17 American History & Life 23 18 Dissertation Abstracts 4 18 CINAHL (Nursing Index) 22 19 CINAHL (Nursing Index) 3 20.5 Biological & Agricultural Index 18 20 Biological & Agricultural Index 3 20.5 Engineering Index (Compendex) 13 21 Engineering Index (Compendex) 3 20.5 TTD ( Textiles) 7 22 TTD ( Textiles) 3 20.5 Expectations and Effectiveness Using CD-ROMs 135 recent dates, and the print source un- known. On the other hand, 11 percent of respondents indicate that they like print sources better. They offered the follow- ing reasons: like hard copy, can find more items in print, CD-ROM misses items, easier and faster to search print, and do not know how to use the CD-ROM ver- sion. Slightly more than 4 percent of us- ers chose both versions, citing that CD- ROMs and print indices are both valu- able and useful for research and that com- bined they include more dates. A couple of users stated they had no preference (see table 4). User Confidence in Searching CD-ROM users were asked to rank their level of confidence on a Likert scale, where one equaled �Not at All Confi- dent� and five equaled �Extremely Con- fident.� It is not surprising to note that lower-level undergraduate students are less confident in using CD-ROMs whereas upper-level undergraduate and graduate students are more confident. Table 5 reveals the increase in user confi- dence as the students� class level progresses by collaps- ing and labeling con- fidence levels one and two as �Less Confi- dent,� labeling re- sponse three as �Mod- erately Confident,� and combining and labeling levels four and five as �More Confident.� The ma- jority of students indi- cated that they are confident in search- ing, with 71 percent of lower-level under- graduates, 82 percent of upper-level under- graduates, and 83 per- cent of graduate stu- dents expressing moderately and more confident. It is clear from table 5 that as students progress in class level through college and have more experience search- ing, their confidence level also increases when searching CD-ROM databases. User Needs to Become More Effective Searcher When users were asked �What more do you need to know to help you become more effective at searching CD-ROMs?� student responses were rank-ordered as �How to develop a search strategy,� �How to choose the right CD-ROM da- tabase,� �How to use the various soft- ware interfaces,� and �How to limit search� (see table 6). When observing stu- dents using CD-ROMs, one of the most frequently asked questions was �Which one (database) do I use?� Some students appear to search the databases randomly TABLE 4 Expressed Preference for CD-ROM or Print Database by Reason Question 5: “Which do you prefer, the CD-ROM version or the printed version?” N Percent Prefer CD-ROM Faster, easier, more efficient, better access 213 77.46% Features: printout, download, abstracts, currency 21 7.64 Don’t know print source exists 0 0 Prefer Print Better for research, find more, has more dates 16 5.81 Saves time, easier to use 13 4.73 Don’t know how to use CD-ROM version 0 0 Prefer Having Both Both useful, easier to understand, includes 12 4.36 more dates Total 275 100% One student searching ERIC complained that �this computer doesn�t have anything on Sports Medicine.� 136 College & Research Libraries March 1997 without either developing a search strat- egy or identifying what type of database they need. When they are unable to find their topic in one database, they search another until they are frustrated and ask for help. One student searching ERIC com- plained that �this computer doesn�t have anything on Sports Medicine.� It appears to the investigators that he lacked an un- derstanding of the subject databases and blamed the computer. Thus, as they in- crease the number of databases, academic librarians also will need to educate stu- dents on the types of information found in each specialized database and how to search it, so that the students will use the appropriate database and find relevant sources for their research needs. Other research studies have shown, and reference librarians have confirmed, that what patrons want most is hands- on assistance with searching at point of need, especially when using multiple TABLE 5 Contingency Table—“Confident” by “Academic Status” Question 6: “How confident are you about your searching skills on the CD-ROM databases you use most often?” Undergrad. Undergrad. Grad. Row Lower Higher MA-Ph.D. Total Less Observed f 25 32 11 68 confident Expected f 17.77 37.17 13.07 20.42% Row % 36.76% 47.06% 16.18% Column % 28.74% 17.58% 17.19% Total % 7.51% 9.61% 3.30% Moderately Observed f 40 79 18 137 confident Expected f 35.79 74.88 26.33 41.14% Row % 29.20% 57.66% 13.14% Column % 45.98% 43.41% 28.13% Total % 12.01% 23.72% 5.41% More Observed f 22 71 35 128 confident Expected f 33.44 69.96 24.60 38.44% Row % 17.19% 55.47% 27.34% Column % 25.29% 39.01% 54.69% Total % 6.61% 21.32% 10.51% Column 87 182 64 333 Total 26.13% 54.65% 19.22% 100% Chi-square (Pearson) 15.67 (df = 4) p = .0035 Min. expected f 13.07 TABLE 6 Participants’ Perceived Needs to Being More Effective at Searching Question 7: “What more do you need to know to be more effective at searching CD-ROMs?” How to: N Develop a search strategy 158 Choose the right CD-ROM database 155 Use the software interfaces 154 Limit search (by year, language, etc.) 154 Design & modify search procedures 142 Save search results to floppy disk 131 Use language, keywords, thesaurus 117 Use AND, OR, & NOT operators 98 Use the equipment 86 Expectations and Effectiveness Using CD-ROMs 137 CD-ROM databases. When users were asked in question 8 what types of train- ing they would like, the rank-ordered re- sponses were: personal assistance from reference staff, hands-on workshop, online Help screen, and printed guide- lines. The least frequently desired item was lecture/demonstration (see table 7). It is not surprising that patrons want help at the point of need. Ironically, per- sonal assistance at point of need is the most costly, most time-consuming, and most demanding on reference staff. Some studies indicate that these new demands are leading to reference staff burnout, yet it is the item most desired by students. Users� attitudes seem to suggest that they want immediate gratification and one- stop shopping when using CD-ROMs. Students complain that they must per- form several steps including checking citations with serial holdings and retriev- ing journals on microform or hard copy. They would prefer full text of articles on CD-ROM in addition to citations. The results confirm the expectation that when it comes to using electronic databases, users find more citations and demand more resources, faster delivery, and more hands-on assistance. Effectiveness of CD-ROM Training Sessions From the Writing 101 classes that at- tended lecture/demonstration sessions early in the spring semester of 1995, five instructors decided to return in April, when the new ECL opened, for hands- on instruction on how to use CD-ROMs. The ECL is equipped with fifteen student workstations and one teacher station with a 42-inch monitor. The instructors and students were introduced to four Wilson CD-ROM databases: Social Sci- ences, Humanities, Applied Science and Technology, and Biological and Agricul- tural Index. The purpose of the session was to enable students to become more effective searchers. Of the eighty-one stu- dents, twenty students indicated the training was less than effective, thirty- seven students indicated it was moder- ately effective, and twenty-four students felt it was highly to extremely effective in helping them search (see figure 1). When asked to explain their comfort levels after the training session, students offered a variety of responses. Their rea- sons were split, with twenty-nine stu- dents commenting positively and twenty-eight commenting negatively. The positive responses indicative of com- fort level with training included: easier to understand this database, feel comfort- able with CD-ROMs, know how to search, clear explanation, good demon- stration, easier to use, helpful. The nega- tive comments indicative of discomfort after training included: still need to know how to use CD-ROMs, need to know terms to search, still confused, too much information, and need hands-on training to use the computer.Although the major- ity of students felt more comfortable af- ter the training, at least one-quarter of the students still needed and wanted more training with hands-on computer use. One can infer that one hour of training is not enough, especially considering the fact that students in each class have vary- ing levels of computer expertise. When teaching new computer skills, learners need to be very familiar with the key- board and have several hands-on sessions in order to master the skills presented. This TABLE 7 Participants’ Preferred Types of Training Question 8: “What types of training would you like to have in order to become an effective searcher?” N Personal assistance from staff 198 Hands-on workshop 184 Online Help screen 141 Printed guidelines 131 Lecture/demonstration 43 138 College & Research Libraries March 1997 FIGURE 1 Effectiveness of CD-ROM Training Sessions Question 10: “How effective was the training in helping you become a more effective searcher?” methodology is consistent with the users� preference for either personal assistance or hands-on workshops over the lecture/ demonstration method. Thus, it appears that students know what they need in or- der to become more effective searchers, and they expect training and assistance in academic environments. Overall User Satisfaction with CD-ROM Services Users of URI�s CD-ROM databases were overwhelmingly positive when asked �How satisfied are you with the CD- ROM services?� Ninety-three percent indicated they are moderately to ex- tremely satisfied, whereas only 7 per- cent indicated dissatisfaction. Figure 2 confirms other studies revealing that CD-ROM users are mostly satisfied with products and services with 284 out of 305 students moderately to ex- tremely satisfied (see figure 2). Although 93 percent said they were satisfied, only about 50 percent gave positive comments in response to this question. Negative comments included complaints about services, equipment or systems hassles, and suggestions for im- provements to CD-ROM services. For evaluating and planning future re- sources and services, the comments are most useful for the reference staff. When asked to explain their level of satisfaction, 141 users commented. Comments were collapsed into four categories: positive, negative, equip- ment hassles, and a wish list. The fol- lowing views were categorized as posi- t i v e c o m m e n t s : ve r y h e l p f u l s t a f f ; someone is available to help; good va- riety of databases; good access to in- formation; great tool for research; easy to use; convenient, fast, efficient, and effective services. The following opin- ions by thirty-two users were coded as Expectations and Effectiveness Using CD-ROMs 139 negative comments: still need help; confusing or overwhelming; hard to know what is available; difficult to under- stand system, terms, or what database to use; sometimes staff is too busy to help; difficult to find periodicals after using CD-ROM. In addition, twelve users complained of equipment hassles: the system is too slow, computers are not always available, need more terminals, need more printers, need larger space, CD-ROMs slow and temperamental, and problems with the printer. Perhaps the most valuable comments were suggestions for improvements of- fered by twenty-six users. A rank order of the most frequently requested items in- clude: 1. need more periodicals to match ci- tations; 2. CD-ROMs should note whether URI has journal; 3. need journals on shelf; 4. make more staff available to help with searches; 5. provide full-text access on CD- ROMs; 6. offer remote access to CD-ROM da- tabases; 7. offer full-text databases; 8. provide better print capability; 9. buy more CD-ROMs to provide bet- ter access to older materials; 10. buy more current materials; 11. sell disks in the library to down- load; 12. buy laser printers; 13. purchase Philosopher�s Index and Art Index on CD-ROM; 14. require mandatory learning ses- sions on CD-ROMs. Conclusions and Recommendations Users of CD-ROMs at URI clearly indi- cated their preferences for CD-ROM ser- vices: CD-ROMs over print indices, online remote access to CD-ROM data- bases, more access to journals cited, and more training or assistance searching. Moreover, the majority of students feel FIGURE 2 User Satisfaction with CD-ROM Services Question 11: “How satisfied are you with the CD-ROM services?” 140 College & Research Libraries March 1997 confident searching, although upper- level undergraduate and graduate stu- dents expressed a higher confidence level than lower-level undergraduates. To be- come more effective searchers, students identified what they need to know: how to develop a search strategy, how to choose the right CD-ROM database, how to use the various software interfaces, and how to limit searches. Furthermore, when asked what training methodology they would prefer, the majority selected personal assistance at point of need or hands-on workshops. Even though the majority of users were satisfied with current services, a small percentage did offer suggestions to improve services, such as providing more journals linked to CD-ROM citations, more online access to databases with full text of journal articles, and more staff to assist with searches. The question now remains: How real- istic are these requests for the URI Library or other academic libraries where bud- gets are shrinking and staff is decreas- ing? In January 1996, the library was forced again to cancel more journals; ap- proximately $200,000 worth of journals were eliminated and future increases in projected revenue will be insufficient to pay for projected inflationary increases on the library�s serial subscriptions. Therefore, further reductions in the num- ber of subscriptions is likely in the fu- ture. Ironically, although the budget has decreased, the CD-ROM and electronic database services have increased�but at the cost of eliminating other resources, especially print indices that are available in CD-ROM or online format. One answer to dwindling budgets and increasing demands for more electronic access is more cooperation and shared resources. Since the formation of a con- sortium of five academic libraries in Rhode Island, HELIN (Higher Education Library Information Network), the URI Library has moved one step further in that direction. The OPAC called HELIN was purchased by the consortium con- sisting of the University of Rhode Island, Rhode Island College, Community Col- lege of Rhode Island, Providence College, and Roger Williams University. The con- sortium also negotiated its first joint online database, the Expanded Academic Index (EAI) 2000, which includes 1,500 journals with one-third also available in full text. This cooperative venture en- abled URI to offer remote online access to a popular database because of the shared cost among consortium members. In addition, two laser printer worksta- tions are dedicated for printouts from the EAI database. Under further consider- ation is a networked print solution for all electronic databases, including the OPAC. With the added ability to search the EAI database remotely, congestion and requests for assistance in the CD-ROM room have been reduced. Because of the popularity of remote access to databases, the HELIN consortium continues to ne- gotiate access to other resources available from Internet providers. After a three- month trial period in early 1996, several databases are now also offered via FirstSearch. As a member of other con- sortia, URI also acquired Britannica Online and the Engineering Village. With more demand for online access to databases and full-text journal articles, the library will need to continue to evalu- ate which services are essential and which resources are to be eliminated. In- novative ways to fulfill these demands have to be provided. Current library policy calls for the elimination of dupli- cate resources. Thus, some CD-ROM da- tabases and print indices would be can- Ironically, although the budget has decreased, the CD-ROM and electronic database services have increased. Expectations and Effectiveness Using CD-ROMs 141 celed if they were also provided by the comprehensive database EAI. To ease the loss of canceled journals, the library also purchased UNCOVER�s customized gate- way to provide effective document deliv- ery service and to subsidize unmediated article ordering for faculty and graduate students. This service is not currently available for undergraduate students. The librarians continue to offer hands- on training sessions for the OPAC, CD- ROM databases, EAI, and other online services via the Internet, such as FirstSearch and UNCOVER, which in turn should alleviate some demand for personal assistance in the CD-ROM room at point-of-use. The library will need to develop a public relations campaign to inform the university community about its changing resources and services, in- cluding both CD-ROM and on-line sources. In addition, staffing the public services area may require creative ap- proaches, such as internships and reshifting of personnel from other units. Furthermore, academic libraries need to rethink reference services as advocated by David W. Lewis�s call for organizational change in academic libraries.17 Future Implications CD-ROM databases were much in demand for the past decade, and they continue to be a popular choice among students and researchers in academic libraries. With the expansion of database access by Internet providers, however, there is an increasing demand for more online remote access to full-text databases via Internet providers. It appears obvious that academic libraries are currently wrestling with the following unresolved questions regarding electronic access to databases: 1. Cost: Can libraries afford to provide remote access to electronic databases with more subscriptions and more site licenses while maintaining and increasing CD- ROM networks? Should librarians aban- don CD-ROM networks in favor of re- mote access to databases via the Inter- net? 2. Medium: What formats will be avail- able in the future for libraries in order to meet the demands of users? Will librar- ies be able to provide a variety of for- mats including duplication of resources? Is it advisable to rely on the Internet, or would it be more prudent to develop campus-based tape loads with more ex- pensive equipment? 3. Long-range implications: What for- mats will or will not be available in ten years, and how can libraries ensure ad- equate archival resources if they are leas- ing services they do not own? Do librar- ies own the data when a CD-ROM sub- scription ends or vendors change? 4. Service: What changes will occur in the delivery of reference services? Will academic libraries offer more personal assistance at point of need, more inten- sive bibliographic instruction including hands-on training with new technolo- gies, or more remote online services? Will librarians provide mediated online searching or electronic reference queries to help users with their online searches? Will libraries offer an amalgamation of services, including CD-ROM, tape loads, Internet, LAN, WAN, and print indices? Or will there be another unforeseen tech- nological opportunity? Academic libraries and librarians are at a crossroads and will need to choose what resources to offer, regardless of me- dium. Libraries will not be able to afford a full array of formats for all resources, and thus they will need to decide which resources and which media to offer in the future. Cooperative collection de- v e l o p m e n t a n d s h a r e d r e s o u r c e s among libraries in consortia seem to be one of the best routes for academic libraries to take. 142 College & Research Libraries March 1997 Notes 1. Cheryl A. McCarthy, �Students� Perceived Effectiveness Using the University Library,�College & Research Libraries 56 (May 1995): 26�41. 2. David W. Lewis, �Making Academic Reference Services Work,� College & Research Libraries 55 (Sept. 1994): 445�56. 3. Charles R. Hixson, �CD-ROM and the Undergraduate: Reference and Instruction at Risk,� Reference Services Review 21, no. 3 (fall 1993): 31�34. 4. Dorothy F. Davis, �A Comparison of Bibliographic Instruction Methods on CD-ROM Da- tabase,� Research Strategies 11 (summer 1993): 156�63. 5. See Gillian Allen, �CD-ROM Training: What Do the Patrons Want?� RQ 30, no. 1 (fall 1990): 88�93; Susan K. Charles and Katharine E. Clark, �Enhancing CD-ROM Searches with Online Up- dates: An Examination of End-User Needs, Strategies, and Problems,� College & Research Libraries 51, no. 4 (July 1990): 321�28; Elaine C. Clever and David P. Dillard, �What Do CD-ROM Users Really Need?� Information Services and Use 11,. no. 3 (July 1991): 141�53; Hixson, �CD-ROM and the Undergraduate�; Trudi E. Jacobson, �All I Need Is in the Computer,� Reference Librarian 38 (1992): 221�28; Deanna Nipp, �Back to Basics: Integrating CD-ROM Instruction with Standard User Education,� Research Strategies 9, no. 1 (winter 1991): 41�47; Margaret Sylvia and Leigh Kilman, �Searching on CD-ROM in an Academic Environment,� Computers in Libraries 11, no. 10 (Nov. 1991): 44�46. 6. See Kimberly Amato and Margaret W. Jackson, �CD-ROMs: Instructing the User,� CD- ROM Librarian 5 (June 1990): 14�21; Caroline Blumenthal, Mary Jo Howard, and William R. Kinyon, �The Impact of CD-ROM Technology on a Bibliographic Instruction Program,� College & Research Libraries 54 (Jan. 1993): 11�16; Rebecca Bostian and Anne Robbins, �Effective Instruction for Search- ing CD-ROM Indexes,� Laserdisk Professional 3 (Jan. 1990): 14�17; Nancy Broughton, Patricia Herrling, and Nancy McClements, �CD-ROM Instruction: A Generic Approach,� CD-ROM Li- brarian 6 (Nov. 1991): 16�19; Gregory A. Crawford, �The Effects of Instruction in the Use of PsycLIT on Interlibrary Loan,� RQ 31, no. 3 (spring 1992): 370�76; Davis, �A Comparison of Bibliographic Instruction Methods on CD-ROM Database�; Danise G. Hoover and Victoria Clayton, �Graduate Bibliographic Instruction in ERIC on CD-ROM,� Behavioral & Social Sciences Librarian 8. nos. 1�2 (1989): 1�12; Richard F. Kenny and Eileen Schroeder, �An Evaluation of a Training and Assistance Program for the CD-ROM Databases: Reflections on the Process,� Reference Services Review 20, no. 2 (summer 1992): 41�48; Bruce Leach, �Computer-Based CD-ROM Tutorials�Providing Effec- tive On-Demand Instruction,� CD-ROM Professional 6 (July 1993): 113�14+; Ida B. Lowe, �Inte- grating CD-ROMs into Your Bibliographic Instruction Program,� CD-ROM Professional 3 (Nov. 1990): 16�19; Patricia Lynn and Karen Bacsanyi, �CD-ROMs: Instructional Methods and User Reactions,� Reference Services Review 17, no. 2 (summer 1989): 17�25; John Maxymuk, �Consider- ations for CD-ROM Instruction,� CD-ROM Professional 4 (May 1991): 47�49. 7. Stanley D. Nash and Myoung C. Wilson, �Value-Added Bibliographic Instruction: Teach- ing Students to Find the Right Citations,� Reference Services Review19, no. 1 (spring 1991): 87�92. 8. Ibid. 9. Tim Bucknall and Rikki Mangrum, �U-Search: A User Study of the CD-ROM Service at the University of North Carolina at Chapel Hill,� RQ 31, no. 4 (spring 1992): 542�53. 10. Ibid., 542. 11. McCarthy, �Students� Perceived Effectiveness Using the University Library.� 12. Ole R. Holsti, Content Analysis for the Social Sciences and Humanities (Reading, Mass.: Addison- Wesley, 1969), 40. 13. Ibid., 30. 14. Abraham Kaplan, The Conduct of Inquiry (Scranton,: Chandler Pub., 1964). 15. Marija J. Norusis, SPSS/PC+ 4.0 Base Manual for the IBM PC/AT and PS/2 (Chicago: SPSS, Inc., 1990). 16. Wayne W. Daniel, Applied Nonparametric Statistics (Boston, Mass.: Houghton-Mifflin, 1978). 17. Lewis, �Making Academic Reference Services Work.�