Bao.p65 A Comparative Study 251 251 A Comparative Study of Library Surveys of Internet Users at Seton Hall University in 1998 and 2001 Xue-Ming Bao Xue-Ming Bao is an Assistant Professor/Librarian in University Library at Seton Hall University; e- mail: baoxuemi@shu.edu. The author acknowledges the assistance of Arthur W. Häfner, Professor and Dean of the University Libraries, and Richard E. Stern, Associate Professor, Coordinator of Reference Services and Chair of the Library Home Page Task Team, for their critical reviews and helpful comments. This article compares data collected in library surveys of Internet users at Seton Hall University in 1998 and 2001. The aim of the comparison is to provide information on changes in library users’ Internet use behav- ior, satisfaction levels, and problems in using the Internet. The response distribution of the people in different categories represents the popula- tion of Seton Hall University in approximate proportions in both surveys. This article discusses the challenges that were met with visible, moder- ate, and no improvements. It points out what remains to be the major challenge for librarians. hat changes have occurred with regard to the Internet as a component of electronic ser- vices in academic libraries in the past five years? In a 1996 ALA survey, Mary J. Lynch reported that 87 percent of the academic libraries in doctorate-grant- ing institutions included in the survey provided information access through home pages on the World Wide Web, 84 percent provided formal Internet training for faculty, and 90 percent provided stu- dent training.1 A sign of change was re- flected in Mick O’Leary’s article, “New Academic Information Model Bypasses Libraries,” published in 2001.2 He wrote that several ambitious, well-founded in- formation distributors such as XanEdu, Questia, and Ebrary seek to replace the library–vendor partnership with new channels that cut the library out of the loop. On the other hand, Mark Y. Herring gave ten reasons why the Internet is no substitute for a library in an article pub- lished in April 2001.3 Herring asserted that the Internet has not made libraries obso- lete. The author conducted a library survey of Internet users at Seton Hall University (South Orange, New Jersey) in 1998. The survey’s purpose was to collect data that the university’s librarian faculty and ad- ministration could use to analyze user satisfaction with information services provided through the Web. Seton Hall classroom faculty and students completed 786 questionnaires. About 80 percent of the respondents reported using the Web on a daily or weekly basis. The results revealed valuable information about the Internet users’ search strategies and lev- els of satisfaction with using the Web.4 252 College & Research Libraries May 2002 Seton Hall University Library’s home pages had experienced many positive changes since 1998. For example, they provided access to more academic sub- scription databases in 2001 than they had three years earlier; the design of the li- brary home pages reflected more of the collective wisdom of the librarians; and a larger assortment of interactive library Web forms were provided. The new changes in the library home pages raise two key questions: How do the new changes meet the needs of students and faculty at Seton Hall University? and What strategies should Seton Hall Uni- versity librarians pursue to improve li- brary services through the home pages? Purpose The purpose of the 2001 survey was to collect data that Seton Hall University li- brarian faculty and administration could use to analyze library users’ Internet search behavior, satisfaction levels, and the needs of the information services pro- vided through the library’s home pages. This study is a descriptive survey and a follow-up to the 1998 survey.5 The data will provide information on changes in library user habits between 1998 and 2001. In general terms, the objective of the study is to serve as “an aid … in planning, im- proving public relations, and even mar- keting.”6 Survey Instrument The online survey instrument described in this research was pilot-tested by ten people at Seton Hall University: four stu- dents, two teaching faculty members, and four librarians and library administrators. It was modified based on their responses and suggestions. The instrument at- tempted to elicit answers to the follow- ing questions repeated from the 1998 sur- vey distributed in paper form: 1. Who is using the Internet and the library’s home pages? 2. What are the academic majors or fields of study of the users? 3. How frequently does the user search the Internet? 4. What does the user search for on the Internet? 5. What is the user ’s satisfaction level with search results? 6. How long does it take for the user to find “satisfactory” results? 7. What problems has the user en- countered when searching the Internet? 8. Has the user visited the university library’s home pages? 9. Does the user find the university library’s home pages helpful in search- ing the Internet? 10. How does the user find information on the Internet? 11. What are the user ’s favorite Internet search engine(s)? 12. Academically, whose responsibility is it to teach the user how to search the Internet effectively? 13. What training courses would the user attend? Data Collection In Applied Statistics, John Neter, William Wasserman, and G. A. Whitemore stated that a census of a finite population is a study that includes every element of the population and a census is appropriate when it is easy to reach everyone included in the study.7 In Descriptive Statistical Tech- niques for Librarians, Arthur W. Häfner stated: “Whether the population is sampled or surveyed completely (census), the librarian will want to apply algo- rithms to reorganize and summarize the raw data.”8 This author collected the 2001 survey data by means of a census of the 8,000+ people, including students, faculty, administrators, and staff, on the university’s main campus via a direct e- mailing to their Seton Hall e-mail ac- counts. The e-mail cover letter provided: • the researcher ’s affiliation with the university; • a brief description of the project; • a statement of the voluntary nature of the project; • a statement of anonymity/confi- dentiality of the subject’s data. People were asked to click on a hypertext link in the e-mail cover letter A Comparative Study 253 TABLE 1 Percent of Respondents of Different Categories Filling Out the Survey 1998 2001 n = 786 n = 581 Undergraduate 75.8% 57.3% Graduate 20.4% 18.8% Faculty 3.8% 8.6% Admin/Staff 10.2% Other 5.2% to access the online questionnaire.9 The online instrument required about five minutes to complete. The survey e-mail cover letter was sent on January 16, 2001, a week after the be- ginning of the new spring semester. A total of 581 responses were received, of which 567 were received within the first two weeks of the direct e-mailing and nine were received between the third and fifth weeks. The mass direct e-mailing to 8,000+ people caused concern within the university’s Division of Computing Ser- vices because e-mailings for surveys were new to the campus and it was thought that they might clog up the university’s e-mail server. The second announcement of the survey was published on February 26, 2001, in the university’s community announcement bulletin board. Only five more responses were received after the second announcement. The 2001 online survey yielded fewer responses (581) than the 1998 paper survey did (786). The 1998 survey focused on students and faculty only; the 2001 survey in- cluded students, faculty, administra- tors, and staff. If 8,000+ were the popu- lation, 581 responses represented a low response rate (7%). In Survey Research Methods, Floyd J. Fowler Jr. remarked that a 5 to 20 per- cent response rate was low but added: “There is no agreed-upon standard for a minimum acceptable response rate. … One generalization that seems to hold up for most mail surveys, though it is inferential, is that people who have a particular interest in the subject matter or the research itself are more likely to re- turn mail questionnaires than those who are less interested. This means that mail surveys with low response rates may be biased significantly in ways that are re- lated directly to the purposes of the re- search.”10 With this caution in mind, the author qualified the survey projection to the students, faculty, administrators, and staff who are interested in using the Internet only, rather than the entire popu- lation of Seton Hall University. To analyze the data, the Statistical Package for the Social Sciences (SPSS) was employed. Data from the survey instru- ments were coded and subjected to SPSS. Frequency tabulations were applied to obtain descriptive measures of the re- sponses. Results User Demographics Table 1 shows that 786 people completed the paper survey instrument in 1998 and 581 completed the online survey instru- ment in 2001. In 1998, print survey instru- ments were sent to the teaching faculty via campus mail to be distributed to stu- dents. Therefore, administrators and staff were not included in the 1998 survey. In 2001, the online survey form’s URL was sent to all people with a Seton Hall e-mail account so that administrators and staff were able to participate in the survey. Because the exact responses of the two TABLE 2 Percent of Respondents� Academic Major or Field 1998 2001 n = 786 n = 581 Arts & Sciences 44.7% 44.4% Business 23.3% 16.4% Diplomacy 0.1% 2.8% Education 18.2% 14.6% Medical Education 4.1% 3.3% Nursing 5.2% 2.1% Theology 1.7% 2.6% Other 2.7% 13.9% 254 College & Research Libraries May 2002 TABLE 4 Percent of Respondents Searching Information Related to Academic Studies or Not Related to Academic Studies 1998 2001 n = 786 n = 581 Information related to academic studies 83.2% 88.1% Information not related to academic studies 73.9% 81.6% TABLE 5 Percent Of Respondents� Levels Of Satisfaction For Internet Search Results 1998 2001 n = 786 n = 581 1 (High Satisfaction) 7.8% 18.1% 2 36.1% 43.0% 3 37.0% 27.4% 4 11.8% 6.4% 5 (Low Satisfaction) 3.1% 1.2% surveys are different, it is only meaning- ful to compare percentages of the re- sponses rather than the exact counts of the responses. A higher percentage of fac- ulty (8.6%) participated in the online sur- vey in 2001 than in the paper survey in 1998 (3.8%). Undergraduate participation dropped in 2001 (57.3%), compared to 1998 (75.8%). However, the response dis- tribution of the people in different catego- ries represented the population of Seton Hall University in approximate propor- tions in both the 1998 and 2001 surveys. Table 2 shows that in both surveys, majority responses were from three aca- demic majors: Arts & Sciences (44.7% in 1998; 44.4% in 2001), Business (23.3% in 1998; 16.4% in 2001), and Education (18.2% in 1998; 14.6% in 2001). Other re- sponses from academic majors included Nursing (5.2% in 1998; 2.1% in 2001), Graduate Medical Education (4.1% in 1998; 3.3% in 2001), Theology (1.7% in 1998; 2.6% in 2001), and Diplomacy (0.1% in 1998; 2.8% in 2001). The “other” category’s percent- ages increased from 2.7 per- cent in 1998 to 13.9 percent in 2001 due to the participation of administrators and staff in the 2001 survey. Internet Use Frequency and Satisfaction Levels Table 3 shows that daily use of the Web doubled in 2001 (84.3%), compared to 1998 (40.2%). Weekly use (38.3% in 1998; 10.5% in 2001) and monthly use (10.7% in 1998; 0.9% in 2001) declined sharply in 2001. Almost no one claimed “seldom” use of the Internet in the 2001 survey (0.3%), compared to 1998 (6.6%). Table 4 shows that respondents searched the Internet for information re- lated to both their academic (83.2% in 1998; 88.1% in 2001) and nonacademic studies (73.9% in 1998; 81.6% in 2001), with a slight increase in both categories in 2001. Using a scale of five levels of satisfac- tion, where one indicates the highest and five the lowest, table 5 shows that the re- spondents’ levels of satisfaction for Internet search results increased substan- tially. In 1998, only 7.8 percent of respon- dents indicated a high level of satisfac- tion, compared to 18.1 percent in 2001. In 1998, 36.1 percent rated their satisfaction for Internet search results as level two, compared to 43.0 percent in 2001. In con- trast, the percentages of satisfaction lev- TABLE 3 Percent of Respondents� Frequency of Using the World Wide Web on the Internet 1998 2001 n = 786 n = 581 Daily 40.2% 84.3% Weekly 38.3% 10.5% Monthly 10.7% 0.9% Seldom 6.6% 0.3% Other 2.7% 13.9% A Comparative Study 255 els three (37.0% in 1998; 27.4% in 2001), four (11.8% in 1998; 6.4% in 2001), and five (3.1% in 1998; 1.2% in 2001) all de- creased substantially. Internet Search Results and Problems Table 6 shows that more respondents were able to obtain satisfactory search results within a short period. In 1998, only 5.0 percent of respondents achieved satisfactory results in less than ten minutes, compared to 10.0 percent in 2001. In 1998, 23.5 percent of respon- dents spent 11 to 20 minutes, compared to 24.1 percent in 2001. However, the per- cent of respondents spending twenty minutes or more to obtain satisfactory search results decreased in 2001: 30.7 per- cent spent 21 to 30 minutes in 1998, com- pared to 28.9 percent in 2001; and 33.6 per- cent in 1998 spent 30 minutes or more, compared to 31.7 percent in 2001. Table 7 identifies three major problems encountered by users when searching the Internet: • do not find information needed (49.2% in 1998; 54.7% in 2001); • no full-text information can be cited for academic study and/or research (43.8% in 1998; 52.8% in 2001); • too many hits (38.4% in 1998; 50.1% in 2001). Use of the University Library Home Page Table 8 shows that less than half of the respondents (41.3% in 1998) visited the li- brary home page; however, the percent- age increased to 71.1 percent in 2001 when conducting their research. Table 9 shows that out of the 41.3 per- cent of respondents who accessed the li- brary home page in 1998, 19.2 percent found it helpful, 13.7 percent found it somewhat helpful, 5.3 percent found it very helpful, and 2.5 percent found it not helpful. In 2001, more respondents viewed the library home page in a favor- able way. Of the 71.1 percent of the re- spondents who accessed the library home page in 2001, 32.9 percent found it help- ful, 23.8 percent found it somewhat help- ful, 11.2 percent found it very helpful, and 3.8 percent found it not helpful. Internet Search Strategies Table 10 shows that using Internet search engines was the preferred way for re- spondents to search the Internet in both 1998 (83.6%) and 2001 (84.7%), followed by directly entering a URL or Web ad- dress (63.9% in 1998; 69.5% in 2001). The percentage of respon- dents using categorized gateway Internet r e- source listings (e.g., the library’s Web listings by subject) doubled from 17.8 percent in 1998 to 33.6 percent in 2001. The percentage of respon- dents using academic subscription databases tripled from 12.1 percent in 1998 to 41.7 percent in 2001. TABLE 6 Percent Of Respondents� Time Spent On Searching Satisfactory Results Per Session 1998 2001 n = 786 n = 581 5�10 minutes 5.0% 10.0% 11�20 minutes 23.5% 24.1% 21�30 minutes 30.7% 28.9% 31+ minutes 33.6% 31.7% TABLE 7 Percent of Respondents� Problems Encountered When Searching the Internet 1998 2001 n = 786 n = 581 Too many hits 38.4% 50.1% Do not find information needed 49.2% 54.7% No full-text information can be cited for academic study and/or research 43.8% 52.8% Other 16.3% 11.5% 256 College & Research Libraries May 2002 TABLE 8 Percent of Respondents Using the Library�s Home Pages 1998 2001 n = 786 n = 581 Yes 41.3% 71.1% No 54.3% 24.8% Table 11 shows that Yahoo remained the favorite Internet search engine among the respondents in 2001 (69.7%); however, the percentage dropped sharply in com- parison with 80.5 percent in 1998. The search engines that increased the percent- age of users included: Alta Vista (27.6% in 1998; 34.6% in 2001), Excite (27.7% in 1998; 33.2% in 2001), Lycos (17.0% in 1998; 33.9% in 2001), Hotbot (3.8% in 1998; 17.6% in 2001), and MetaCrawler (2.3% in 1998; 7.4% in 2001). The search engines that decreased the percentage of users included InfoSeek (36.1% in 1998; 24.6% in 2001) and WebCrawler (19.6% in 1998; 11.2% in 2001). The following three search engines that were either not on the 1998 survey or not available in 1998 were used in 2001 by more than 20 percent of the re- spondents: Ask Jeeves (31.2%), Netscape (29.4%), and Google (25.8%). Internet Search Training Table 12 shows the opinions of respon- dents concerning which persons or agencies within the university should be responsible for teaching how to search the Internet effectively. The re- sponses included: • academic college (39.4% in 1998; 32.2% in 2001); • university computing services (44.5% in 1998; 44.2% in 2001); • university librarian faculty (18.7% in 1998; 28.4% in 2001); • other (15.8% in 1998; 16.4% in 2001). Table 13 identifies which Internet train- ing courses the respondents would likely attend if the courses were offered by the librarian faculty. Respondents indicated that they were most interested in learn- ing advanced Internet searching skills in both 1998 (55.2%) and 2001 (52.3%). The percentage for basic Internet search train- ing dropped from 44.1 percent in 1998 to 21.0 percent in 2001. The percentage for academic subscription database training increased from 24.4 percent in 1998 to 32.9 percent in 2001. The percentages for the training of gateway Internet resource list- ings remained more or less the same (30.5% in 1998; 29.8% in 2001). Discussion In the report of the 1998 survey, the au- thor discussed three challenges faced by Seton Hall University librarians in view of the 1998 data.11 This section discusses how these three challenges have been met in comparison with the 2001 survey data. The Challenge Met with Visible Improve- ments The 1998 survey revealed that the main challenge for Seton Hall University librar- ians appeared to be how to raise the us- ers’ level of satisfaction when providing information services through the Internet. This challenge has been met with visible improvements in the categories of high level of satisfaction and high-speed searching for satisfactory results. In 1998, only 7.8 percent of the respondents indi- cated a high level of satisfaction with the TABLE 9 Percent of Respondents� Opinions on the Library�s Home Pages in Facilitating Searching of the Internet 1998 2001 n = 786 n = 581 Very helpful 5.3% 11.2% Helpful 19.2% 32.9% Somewhat helpful 13.7% 23.8% Not helpful 2.5% 3.8% The need for subscription database training increased from 24.4 percent in 1998 to 32.9 percent in 2001. A Comparative Study 257 Internet; the percentage more than doubled to 18.1 percent in 2001. In 1998, only 5 percent of the respondents were able to obtain satisfactory results within five to ten minutes; again, the percent- age doubled in 2001 to 10 percent. How- ever, the percentages in the middle and low levels of satisfaction and eleven minutes or more for satisfactory search results remained more or less the same with a slight change toward a positive direction. The Internet continued to be an im- portant source of information for aca- demic studies as evidenced by its daily use, which increased from 40.2 percent in 1998 to 84.3 percent in 2001. About 10 percent of the respondents self-iden- tified as having seldom or never used the Internet in 1998; the percentage di- minished to 0.3 percent in 2001. The university library home pages have been accepted and used by more and more respondents. The percentage of respondents using the library’s home pages () in- creased from 41.3 percent in 1998 to 71.1 percent in 2001. The percentage of re- spondents’ opinions on the library’s home pages in terms of facilitating Internet search in the category of “very helpful” increased from 5.3 percent in 1998 to 11.2 percent in 2001; the percent- ages in the categories of “helpful” and “somewhat helpful” increased from 32.9 percent in 1998 to 56.7 percent in 2001. The Challenge Met with Moderate Improvement The percentage of respon- dents who believed that responsibility for teaching Internet searching belongs to university computing services and/or to indi- vidual academic colleges remained more or less the same, around 40 percent in both 1998 and 2001. The percentage of respondents who believed that respon- sibility for teaching Internet searching belongs to university librarian faculty increased from 18.7 per- cent in 1998 to 28.4 percent in 2001. This TABLE 10 Percent of Respondents Searching the Internet in Different Ways 1998 2001 n = 786 n = 581 Very helpful 5.3% 11.2% Use Internet search engines 83.6% 84.7% Type in a Web address directly 63.9% 69.5% Use categorized gateway Internet resource listings 17.8% 33.6% Use subscription databases 12.1% 41.7% Other 2.3% 3.6% TABLE 11 Percent of Respondents� Favorite Internet Search Engines 1998 2001 n = 786 n = 581 About 5.2% Allthesites 1.5% Alta Vista 27.6% 34.6% Ask Jeeves 31.2% Business Web 1.5% DrKoop 2.1% Economic Indicator 0.7% Excite 27.7% 33.2% FindLaw 4.6% Google 25.8% Hotbot 3.8% 17.6% Inference Find 1.2% InfoSeek 36.1% 24.6% Look Smart 6.2% Lycos 17.0% 33.9% Magellan 4.5% MetaCrawler 2.3% 7.4% Netscape 29.4% Northern Light 6.9% RefDesk 2.1% Skworm 0.5% Snap 3.4% WebCrawler 19.6% 11.2% Yahoo 80.5% 69.7% Other 3.2% 11.2% 258 College & Research Libraries May 2002 may be the result of the librarian faculty’s continuing discussions within the aca- demic community to make sure that this area of teaching is their responsibility. The Challenge Met with No Improvement The three major problems that did not improve or “worsened” for respondents when searching the Internet were as fol- lows: do not find information needed (49.2% in 1998; 54.7% in 2001), no full-text information can be cited for academic study and/or research (43.8% in 1998; 52.8% in 2001), and too many hits (38.4% in 1998; 50.1% in 2001). These percentages of the 2001 survey are puzzling. With the increase in the library’s subscription to Web-based periodical and newspaper da- tabases, one would speculate that the per- centage of respondents who may encoun- ter these problems would be lower in 2001 than in 1998. However, the training needs of the respondents may provide some clues. The need for subscription database training increased from 24.4 percent in 1998 to 32.9 per- cent in 2001. This may mean that librarians need to focus their training on subscription databases, advanced Internet search skills, and useful cur- ricula-related Web resources. Basic Internet search skills are not in demand because the surveys show that the percentage dropped from 44.1 percent in 1998 to 21.0 percent in 2001. The librar- ians need to keep up with their professional training to improve their own Internet search skills in order to main- tain an edge in the opinions of students, faculty, admin- istrators, and staff as Web- based information search specialists. Conclusions The comparison of the sur- vey data between 1998 and 2001 shows that the Internet continues to be an important component of library services in today and tomorrow’s library. The majority of the respondents used the Internet on a daily basis in 2001, as opposed to less than half the respondents in 1998. The strategies of reallocating library budget resources and improving the design of the univer- sity library home pages collectively may have worked in increasing the library us- ers’ levels of satisfaction and in attract- ing more library users to use the library home pages. The librarians’ proactive ini- tiatives with the classroom teaching fac- ulty also may have worked to improve the respondents’ opinions of the librar- ians’ role in teaching Internet searching. The survey data show that the librarian’s major challenge in the new century remains that of how to help li- brary users find what they are looking for through the Internet. Library users need to know what subscription academic da- tabases and what high-quality free Web TABLE 12 Percent of Respondents� Opinions on Who Should Be Responsible for Teaching Them How to Search the Internet Effectively 1998 2001 n = 786 n = 581 Academic college 39.4% 32.2% University computing services 44.5% 44.2% University librarian faculty 18.7% 28.4% Other 15.8% 16.4% TABLE 13 Percent of Respondents Who Would Likely Attend Internet Training Courses Offered by the University Librarian Faculty 1998 2001 n = 786 n = 581 Academic college 39.4% 32.2% Basic Internet searching 44.1% 21.0% Advanced Internet searching 55.2% 52.3% Gateway Internet resource listings 30.5% 29.8% Subscription databases 24.4% 32.9% A Comparative Study 259 resources are available for them to search and how to search these databases and Web resources effectively and efficiently. This suggests that librarians need to fo- cus on opportunities for providing train- ing on subscription academic databases, advanced Internet search skills, and use- ful curricular Web resources. Notes 1. Mary J. Lynch, Electronic Services in Academic Libraries: ALA Survey Report (Chicago: ALA, 1996). 2. Mick O’Leary, “New Academic Information Model Bypasses Libraries,” Online 25 (July/ Aug. 2001): 72–74. 3. Mark Y. Herring, “10 Reasons Why the Internet Is No Substitute for a Library,” American Libraries 32, no. 4 (Apr. 2001): 76–78. 4. Xue-Ming Bao, “Challenges and Opportunities: A Report of the 1998 Library Survey of Internet Users at Seton Hall University,” College & Research Libraries 59 (Nov. 1998): 535–43. 5. Helen M. Gothberg, “Library Survey: A Research Methodology Rediscovered,” College & Research Libraries 51 (Nov. 1996): 553–59. In this article, Gothberg stated: “Most descriptive sur- veys describe one library or a group of libraries within a system or even a state. Such surveys deal with quantitative data and are used to compare with similar statistics from the previous year, or other libraries. They may also seek opinions and/or demographic data about users.” 6. Ibid. 7. John Neter, William Wasserman, and G.A. Whitemore, Applied Statistics (Boston: Allyn and Bacon, 1982), 188. 8. Arthur W. Häfner, Descriptive Statistical Techniques for Librarians, 2nd ed. (Chicago: ALA, 1998), 6. 9. http://pirate.shu.edu/~baoxuemi/SHU_library_survey_2001_cover_letter.htm. Accessed on Nov. 25, 2001. 10. Floyd J. Fowler Jr., Survey Research Methods (Newbury Park, Calif.: Sage, 1993), 40–41. 11. Bao, “Challenges and Opportunities.”