515 How Unique Are Our Users? Comparing Responses Regarding the Information-Seeking Habits of Engineering Faculty Sarah Robbins, Debra Engel, and Christina Kulp Sarah Robbins is Web Services Coordinator, Debra Engel is Associate Dean of Public Services, and Christina Kulp is Sciences Librarian at University of Oklahoma Libraries; e-mail: srobbins@ou.edu, dhengel@ou.edu, ckulp@ou.edu, respectively. © Sarah Robbins, Debra Engel, and Christina Kulp Studies of information-seeking behaviors are common in the professional literature for library and information studies. This study examines the gen- eralizability of findings of single- institution studies to other institutions by performing an institution-to-institution comparison of the results obtained from an information-seeking behavior survey sent to engineering faculty at twenty research institutions. n an age when libraries are fighting for limited resources and justifying their existence when “everything can be found online,” it is critical that librar- ians have an understanding of their users’ needs. To gain an understand- ing of user needs, many libraries have undertaken assessment studies, the results of which are often published in the professional literature. However, these articles, written by practitioners, frequently present a study conducted at a single institution with little attention paid to generalizing the findings to a broader audience; readers are left to wonder how, if at all, the results apply to them. Since undertaking such studies can be costly and time consuming, it is important for the profession to learn how to present findings in a meaning- ful way to maximize generalizations to similar library settings so that a profes- sional culture of sharing assessment responsibilities is developed. For the current study, identical user surveys related to the information-seek- ing behavior of engineering faculty were conducted at twenty research institutions. However, rather than looking at the data in aggregate to determine information be- haviors of engineering faculty members, this study examines results by institution and compares the findings at other insti- tutions included in the study to ascertain to what extent librarians can apply the findings of single-institution research to their own situations. It is not uncommon for us to think our situation and users are unique, but do these beliefs reflect reality? Review of the Literature Practitioner-Researchers in Libraries In many academic libraries, librarians hold tenure-track faculty positions and are required to perform duties similar to those crl-156 516 College & Research Libraries November 2011 of faculty in other disciplines, including teaching, research, and service. In addi- tion, many academic librarians merely hold a master’s degree in library and infor- mation studies and may or may not have taken a research methodology course as a part of this program; a doctorate degree is often not required for such positions. As a result, many academic librarians are required to publish as a part of their job but may not have adequate training on rigorous research methodologies. Rebecca Watson-Boone analyzes twenty-four articles written by practitio- ner-researchers as they appeared in the Journal of Academic Librarianship from 1985 through 1995. She indicates that practitio- ner-researchers “approach projects and problems in ways that yield (1) solutions, (2) an enlarged understanding of their actual field of work—their practice, and (3) improvements in that practice.”1 In her study, she finds that library practitioner- researchers often concentrate on applied research problems because it results in resolving current problems within their practice.2 In his book The Practitioner-Researcher: Developing Theory from Practice, Peter Jarvis discusses connections between research and practice, the nature of prac- tice, research in practice, and practice and theory.3 Jarvis’ definition of practitioner- research echoes the definition supported by Watson-Boone. He writes, “Practitio- ner-researcher research usually begins with a question about practice, rather than a question about theoretical interpreta- tions of practice. Sometimes the question is prompted by curiosity or interest; other times it might stem from a management requirement; and so on.”4 Again, the em- phasis of practitioner-research is to gather information to apply to the practitioner’s work situation or to inform workplace decision-making; it usually is not about research simply for the sake of expand- ing the professional knowledge base or developing theory. According to Jarvis, practitioner- research fails to gain legitimacy with traditional researchers for several reasons. He notes, “The findings of small-scale re- search are regarded as anecdotal; they are not scientific in any way, so they are not considered valuable to genuine research- ers.”5 He also notes that, for practitioner- researchers, “convenience forms a major criterion in the way they select their ap- proach to their project.”6 While other sam- pling procedures might be better suited to the purpose of the research, samples of convenience often trump more appro- priate sampling methods in practitioner- research. In addition to using samples of convenience, practitioner-researchers also use relatively small samples, which “actually constitutes a whole population if the research question relates only to a specific location of practice.”7 By framing research questions within a limited con- text, practitioner-researchers make the re- search more manageable for themselves. Another limitation of practitioner-re- search mentioned by Jarvis is the unique practice of the practitioner that means “findings from practice situations cannot be applied to other situations” and while “[t]here may well be similarities within unique and transitory practices… this is not an essential criterion for the validity of all case studies.”8 While practitioner- research has an undeniable place in the professional literature, to what extent it can assist in developing a broad profes- sional knowledge base is questionable. In a more recent analysis of practitioner- research compared to scholarly research, it was found that “[a]cademic scholars con- duct significantly more site-independent studies than practitioners.”9 The research- ers also found that practitioners conducted most of their research in a university set- ting and that their preferred methodology includes survey questionnaires, observa- tional methods, and content analysis.10 A noted problem with LIS research is a lack of attention to “validity and reliability, study limitations, and future research.”11 While practitioner-research is of value, an awareness of the limitations of this type of research is critical. How Unique Are Our Users? 517 Nature of Research in Library & Information Studies Numerous editorials and articles have been published recounting the myriad of problems with published research in library and information studies. While these publications vary in methodology and recommendations for improving the state of research in the field, they all seem to agree that there is room for consider- able improvement. During the 1980s, several articles were published discussing the nature of research in library and information sci- ence.12 McClure and Bishop analyze the formal studies of LIS research and outline the key findings and conclusions of stud- ies denoting the quantity, quality, and importance of research to the discipline.13 From their analysis of these articles, they identify key issues concerning research in the field and then interviewed twenty- three heavily published researchers in the field about their thoughts on the topic. Interviewees indicated that the profession “suffered from a lack of curiosity and lim- ited interest in understanding the ‘whys’ that would generate a knowledge base for LIS” and that they were concerned “about forcing academic librarians to publish and ‘conduct research’ for promotion and ten- ure when most lacked the knowledge and skills, the time, and the interest.”14 Despite this, overall, the interviewees expressed “guarded optimism” about the future of LIS research. While the state of research in library and information studies was a topic of interest and discussion in the 1980s, the issues remained in the 1990s with heightened awareness but little forward progress. In 1991, Van House noted, “LIS research needs less inward examination and more outward linkages, both to learn from other fields and to communicate to them the value of LIS and the importance of the questions to be addressed in this growing age of information.”15 She also notes that, for LIS action research to have a signifi- cant impact on practice, it must include: descriptive relevance, goal relevance, operational validity, non-obviousness, and timeliness, but she then warns that action research “is of limited applicability beyond the local library.”16 Similar to the sentiment echoed by interviewees in the McClure and Bishop study, Van House writes: “The role of practitioners in research needs to be clarified. Requiring academic librar- ians to publish without the training and institutional support afforded faculty does them a disservice.… If action studies are to be published (and often they should be), they should be in different, practitioner- oriented journals if possible.”17 Defining the role of practitioner-research in LIS as well as providing academic librarians who must publish adequate training and support is a running theme throughout the literature discussing the state of LIS research. In an editorial for Library and Informa- tion Science Research, Michaels provides several reasons for the “sorry state of library and information science research.” These include: (1) There are too many library journals; (2) Too many librarians are forced to publish (both to fill up these journals and because often their jobs require it); (3) LIS researchers fruitlessly mimic the re- search of other academic disciplines in an attempt to receive scholarly recognition, and have no vision of their own LIS research; and (4) A chasm exists between those who do LIS research and those who actually work as librarians.18 He goes on to write, “In our field it is not enough for our research to purport to be research. To claim academic respect and to save more library schools from closing, our research must be useful, nontrivial, and utilized by fellow scholars in the field.”19 Those bemoaning the state of LIS research often share this desire to be viewed as a legitimate field of research rather than merely a vocational training ground. 518 College & Research Libraries November 2011 The criticism of LIS research continues to this day. In an editorial for The Journal of Academic Librarianship, Peter Hernon provides commentary on the nature of re- search in library and information studies and provides suggestions for focusing the discipline’s research agendas. He writes, “It is time to move the literature and the discussion of research forward as the profession challenges its researchers and editors to explore new issues and to raise the profession’s expectations regarding what comprises good research.”20 Hildreth and Aytac provide an extensive literature review highlighting articles document- ing the traditional problems with library literature dating as far back as 1967. “If there is a common theme in past reviews of the LIS research literature it is this: library-related research is substandard, although developments over the last few decades of the twentieth century indicate that improvements are continually being made.”21 Continual improvement over the span of decades only raises LIS research to “substandard” quality in the minds of many. Several authors have commented on the apparent disconnection between research findings and practice. Rowena Cullen raises an interesting question about how survey findings are used within libraries when she asks, “Why is it that the LIS profession finds it so hard to accept the findings of research carried out in its own institutions and is so slow to act upon it?”22 Hernon and Schwartz talk about this lack of connection with action research and note that, while ac- tion research is “intended to aid local planning and specific management deci- sions, there is little evidence that study findings, in fact, have been used for such purposes.”23 Schlichter and Pemberton discuss why survey data are not readily accepted by the profession and give two reasons for this: “(1) Prejudice against and distrust of quantitative evaluation on the part of many librarians and (2) the tendency to seek the easiest available information sources and to ignore data conflicting with preconceived views.”24 Hiller points out that not all surveys may ask the correct questions in the proper ways, adding, “[w]hether the survey results are statistically reliable, repre- sentative, valid, or significant, doesn’t necessarily mean that they provide information that can be used to assess and improve library service quality.”25 It is this sentiment that drives the current study—to what extent can survey data from other institutions be generalized to one’s own institution? There is also much discussion about survey research in LIS. In a review of library practitioner-research from 1998 through 2001, Hildreth and Aytac found that practitioners most often used surveys for data collection but were increasingly triaging data collection methods.26 They also found that, since few reports use randomized, probability sampling, “most library-related research has no external validity and the findings can only be viewed as preliminary and tentative. Follow-up studies that address these limitations are simply not found in the literature.”27 Watson-Boone notes, “When survey research focuses on people, the researcher is generally studying percep- tions, attitudes, or behaviors of members of a group so as to generalize findings to the entire group.… [T]he goal always is to allow broader generalizations to be made from the findings.”28 While this is a laudable goal, do practitioner-researchers have the proper training to choose cor- rect sampling procedures and accurately understand the data collected? User Behavior Studies and User Satisfaction Surveys Studies on user behavior by discipline or university status abound in the literature. Researchers often use surveys to assess user information needs as well as user satisfaction with a host of library services. As Hiller so aptly states: Library user surveys have become widespread in academic libraries How Unique Are Our Users? 519 during the past twenty years and have often been used as a tool to assess service quality, library per- formance, library use patterns, and user satisfaction….When properly designed and administered, user surveys can provide both quantita- tive and qualitative data directly from the target population. If the sample or survey response is large enough and representative of the surveyed population, data and re- sults can be used to generalize for the population as a whole.”29 The proliferation of literature on how to conduct user surveys or reporting find- ings of a survey speaks to the value of this tool in the profession. In an article about conducting user surveys, George Plosker argues that it “is always preferable to take a more scientific approach to the key issue of determining the needs of users, both existing library users and potential groups of users who are not currently using library services.”30 He claims that surveys can help to review services issues, identify needs, help with marketing, improve resource use, and assist with strategic planning. Plosker advocates using a mix of open- and close- ended questions with no more than 10 percent of the questions being open-end- ed. The article includes tips for successful surveying as well as tips for tabulating re- sults. He ends the article with a challenge for the reader-researcher to “Write about your experience for the professional lit- erature. Let others in the profession learn from what you’ve done.”31 A review of the literature published in this area seems to indicate that the profession values read- ing about others’ studies regardless of the generalizability. It is not uncommon for researchers to include disclaimers of sorts in their discussions or concluding remarks that remind readers that study findings are for a specific institution and may not be generalizable to other institutions. For example, in the conclusion of their study about the factors impacting the selection of information resources among science faculty at the University of Michigan, the researchers note, “While it is not our claim that these findings and conclusions can be extrapolated to the entire scientific com- munity, we believe that they will be help- ful to us in understanding the concerns of our local community of researchers, and may be of interest to other academic sci- ence librarians.”32 Other authors note that some research is “based on scientists at large, elite research institutions and throw little light on what is happening in the nation’s small colleges and universities.”33 While it is important to note limitations of a study, these researchers are relying on the reader to discern the applicability of the findings to his or her own institution. Aked et al. performed a study on the faculty usage of the reference collection at the University of Toledo.34 They write, “Even though the University of Toledo faculty were probably no different in their library use as compared with faculty described in the literature review articles, reference librarians had only anecdotal evidence to support this perception.”35 It is interesting to note that the authors had read the literature yet did not trust that their users were similar to those at other institutions and felt compelled to conduct their own study to prove this. The literature suggests that many librar- ians are distrustful of findings at other institutions, but is it reasonable to believe that faculty members or students at one institution are all that different from fac- ulty and students at another institution? Budd and Connaway studied faculty attitudes toward networked information by surveying faculty in six different dis- ciplines at eight different institutions that represented a mix of public and private governance structures that were geo- graphically dispersed around the United States. The researchers indicate that they have “no pretense that the results of this survey provide an absolutely ‘scientific’ response to that general question, or that they are necessarily reflective of faculty 520 College & Research Libraries November 2011 at all institutions.”36 What distinguishes this study from others is their attempt to represent multiple disciplines and institutions of varying size from distinct geographical areas. Cecelia Brown studied the informa- tion-seeking behavior of astronomers, chemists, mathematicians, and physicists at the University of Oklahoma.37 While Brown is clear to indicate the findings are representative of faculty members at a single institution, she also notes that “the results may be extrapolated to sci- entists at other academic institutions.”38 Her reasoning for this is because some of the OU faculty indicated looking beyond OU’s local collection and using electronic resources to meet their needs. Brown acknowledges that, while her study includes only faculty at one institution, her findings may be relevant to librar- ians at other institutions. Nancy Dewald studied business faculty acceptance of online resources for student research. She notes the limitation of her study when she writes, “A larger sample of business faculty would make the results of the cur- rent study more generalizable, but in any case these findings suggest several ways in which librarians can assist business fac- ulty.”39 While limitations are mentioned, similar to Brown, Dewald indicates that the findings at her institution may be applicable to the larger business faculty community. The University of Washington (UW) has been actively involved in assess- ment activities for years and have made the results of their assessment activities available from a library assessment Web site.40 Lizabeth Wilson discusses what developing a culture of assessment has meant at UW from conducting triennial surveys and focus groups to gathering data to inform decision-making. She chal- lenges the profession when she writes, “In our environment of continual change and new opportunities, we need to focus explicitly on the user. We must invest in continuously assessing the landscape, listening to our users, and looking for places where we can make a difference in connecting people with knowledge.”41 The library has extensively studied the bioscience community at UW, and their findings are available on their assessment Web site. With such rich data sources available to the profession, it is important for librarians to understand what they can extrapolate from another library’s assess- ment activities and user studies to inform their own decision-making and planning. Methodology To test whether the findings of user studies conducted by researchers at a single library can be generalized to other, similar libraries, the researchers conducted a user study of engineering faculty members at multiple institutions and compared institution-to-institution results. The researchers chose engineer- ing faculty because they were aware of a similar, national study being conducted on academic scientists, and they thought the comparison of findings might be beneficial.42 A separate paper presents the aggregated results of the information- seeking behaviors of engineering faculty based on the survey data. The researchers surveyed engineering faculty members at twenty research insti- tutions from across the United States. The 12-item survey consisted of demographic, open- and close- ended questions (see Appendix). The survey gathered both qualitative and quantitative data and was designed to take less than ten minutes to complete. The survey was derived from the surveys used by Brown, Hem- minger et al., Quigley et al., and Stieg and Charnigo.43 In September 2009, an e-mail invitation to participate in an online survey was sent to approximately 4,900 engineer- ing faculty members at twenty public research institutions. The institutions were selected as a purposive sample and represented different regions of the United States with engineering programs and relatively large libraries. Student assistants gathered e-mail addresses of How Unique Are Our Users? 521 all faculty listed on the in- stitutions’ Web sites for their e n g i n e e r i n g d e p a r t m e n t or college. This typically included both tenured and nontenured faculty as well as researchers and faculty emeriti; the survey was sent to the entire population as denoted on the institutional Web sites. Faculty members were given three weeks to respond; a reminder e-mail was sent after two weeks. Results & Discussion Of the 4,905 e-mail invitations sent, 903 engineering faculty members respond- ed, for an overall response rate of 18.41 percent. By institution, the response rate ranged from 12.14 percent to 30.17 percent at the researchers’ home insti- tution. If fewer than twenty responses were received from an institution, that institution’s data were removed from the data set. This left the data from sixteen institutions to be included in the analysis. A chi-square test of association was conducted on the data gathered by ques- tions 4–11 to determine the statistical significance, if any, of the relationship be- tween the respondent’s institution and the answers given in response to the survey. The results of this analysis are presented in tables 1–8. Few of the p values indicate a statis- tically significant (p≤0.05) association between a given response and the respon- dent’s institution. This suggests that, for most types of information gathered, the results found at one institution would mirror the results found at another in- stitution when engineering faculty were asked the same questions. The p values presented in the tables do not indicate the level of importance associated with any of the given services, merely that the answers given were or were not statisti- cally significant in their association to a particular institution. Departmental Duties In the survey, faculty members were asked to provide answers to several demographic questions such as area of specialty within engineering, their length of time in the field, their institutional rank, and what was included in their depart- mental duties. Much of this will be used in the aggregated study on the information- seeking behavior of engineering faculty and was not analyzed for the purposes of the current study because it was as- sumed to be (1) institution specific and (2) readily accessible to librarians at an institution without having to conduct a formal survey. However, the researchers did per- form a chi-square analysis of associa- tion on the responses received to the question, “Which of the following are included in your departmental duties?” (see table 1). The results in- dicated that most of the responses did not vary by institution in a statistically significant way, with the exception of field research (p=0.041) and under- graduate instruction (p=0.018). While the association between institution and responses to this question were statistically insignificant for the most part, librarians at the institution are best poised to know the job require- ments of their institutions’ faculty and should interpret survey results with this knowledge in mind. TABLE 1 Departmental Duties Correlated to Respondents’ Institution Pearson χ2 Value df p Undergraduate instruction 28.585 15 0.018 Field research 25.767 15 0.041 Supervision of doctoral research 24.544 15 0.056 Graduate instruction 16.559 15 0.346 Commercial/proprietary research 13.834 15 0.538 Grant preparation 12.733 15 0.623 Lab research 10.857 15 0.763 522 College & Research Libraries November 2011 Information Use The chi-square analysis of the responses received to the question, “How many of the following have you completed within the past 5 years?” (see table 2) indicated that there was no statistically significant association between the responses and the respondents’ institution. The researchers asked faculty how frequently they sought information to complete a series of tasks common to engineering faculty members. The frequency with which faculty sought in- formation to complete three of the listed tasks was statistically significantly asso- ciated to institution, while the remaining four tasks did not indicate a statistically significant association to institution. The statistically significantly associated tasks include writing or researching for publication (p=0.017), professional devel- opment or staying current (p=0.021), and preparing for a conference presentation (p=0.034). To determine the possible implications for practitioners, the researchers com- bined responses to look at the range of faculty responses by institution indicating that they sought information to complete a given task at least monthly (see table 3). When looking at the data in this way, the only range that seems to indicate a mean- ingful difference would be preparing for a conference presentation, where only 26.6 percent of the engineering faculty at Oregon State indicated they sought information to prepare for a conference presentation at least monthly while 55.8 percent of the engineering faculty at Uni- versity of Washington indicated seeking information for this purpose at the same frequency. However, would this difference mat- ter to a practitioner? Probably not. By TABLE 2 Frequency of Completed Research Projects Correlated to Respondents’ Institution Pearson χ2 Value df p Conference proceedings 74.417 60 0.100 Refereed journal articles/book chapters 86.519 75 0.171 Grant applications 58.948 60 0.514 Nonrefereed journal articles/book chapters 54.624 60 0.672 Books 52.257 60 0.751 Patents/commercial 41.467 60 0.967 TABLE 3 Frequency of Information Seeking to Complete Tasks Correlated to Respondents’ Institution Pearson χ2 Value df p At Least Monthly (%) Write/research for publication 103.353 75 0.017 59.5–82.0 Professional development/stay current 102.010 75 0.021 66.7–93.0 Prepare for a conference presentation 98.936 75 0.034 26.6–55.8 Prepare new research proposal/grant application 82.022 75 0.271 34.5–56.6 Research patents 81.065 75 0.296 9.1–29.3 Determine protocols for laboratory procedures 68.415 75 0.691 19.5–41.9 Prepare for student lectures 63.855 75 0.817 62.5–97.3 How Unique Are Our Users? 523 and large, the types of information that faculty would require for writing for publication would be similar to the types of information needed for preparing a conference presentation. Knowing that engineering faculty are doing one or the other regularly is probably enough to inform decision-making in that area. Researching patents or determining pro- tocols for laboratory procedures could require different types of resources than those that would support the traditional publication or presentation, but a look at the ranges of faculty seeking informa- tion for those purposes at least monthly indicates that at no institution is this hap- pening at that frequency by the majority of the faculty. Finding Information Knowing the productivity level of faculty and how frequently they seek information to complete given tasks is fine, but librarians tend to be more interested in how their users discover needed information. To this end, the en- gineering faculty members were asked the importance of certain sources for helping with their research, how they kept abreast of current developments in their field, and how they became aware of older resources. TABLE 4 Ranked Importance of Sources for Aiding Research Correlated to Respondents’ Institution Pearson χ2 Value df p Very Important + Important (%) Face-to-face discussion with students 64.360 45 0.031 82.4–97.1 Attendance at conferences 62.318 45 0.044 72.1–91.7 E-mail discussion with students 62.173 45 0.046 54.6–86.9 Internet resources 61.466 45 0.052 88.3–100.0 Face-to-face discussion with colleagues 50.597 45 0.260 81.4–97.4 E-mail discussion with colleagues 47.498 45 0.371 72.1–89.1 Textbooks 42.685 45 0.570 48.0–76.2 Books 42.552 45 0.576 74.5–95.3 Scholarly journals (in print or online) 41.088 45 0.638 92.1–100.0 TABLE 5 Sources for Remaining Current in the Field Correlated with Respondents’ Institution Pearson χ2 Value df p Range of Responses (%) Personal communication 38.076 15 0.001 55.8–90.5 Scan abstracting & indexing tools 22.810 15 0.088 15.9–48.9 Conference attendance 19.150 15 0.207 72.1–95.2 Follow references from articles 18.813 15 0.222 76.5–94.9 Scan current issues journals 16.796 15 0.331 70.6–90.7 RSS feeds 15.089 15 0.445 2.0–15.8 E-discussion lists 11.189 15 0.739 5.0–19.2 Current awareness 8.580 15 0.898 0.0–6.3 524 College & Research Libraries November 2011 Of the nine resources listed as poten- tially helping faculty with their research, three were found to be statistically sig- nificantly associated to the respondent’s institution (see table 4). Those that were significantly associated include face-to- face discussion with students (p=0.031), attendance at conferences (p=0.044), and e-mail discussion with students (p=0.046). While this information is useful to librarians in that it provides a better understanding of the importance their engineering faculty members place on these resources, it has minimal impact on library operations, unless a library is actively involved in fostering relation- ships between engineering faculty and their students or providing some sort of assistance for their faculty to attend conference. The chi-square analysis of the re- sponses to “How do you keep abreast of current developments in your field?” provided a similar finding. Personal communication was the only response that was statistically significantly as- sociated to institution (p=0.001) (see table 5). Again, unless librarians are in TABLE 6 Tools Used to Discover Less Recent Journal Articles Correlated to Respondents’ Institution Pearson χ2 Value df p Range of Responses (%) Retrospective searching of indexing & ab- stracting tools 21.835 15 0.112 48.8–68.8 Personal communication 16.891 15 0.325 36.8–76.2 Browsing older volumes 16.397 15 0.356 6.7–35.3 Citations at the end of book chapters 16.673 15 0.339 33.3–69.8 Citations at end of journal article 5.569 15 0.986 93.0–100.0 TABLE 7 Ranked Importance of Library Services in Meeting Information Needs Correlated to Respondents’ Institution Pearson χ2 Value df p Very Important + Important (%) Interlibrary loan 127.392 60 0.000 52.7–88.1 Document delivery 108.881 60 0.000 31.0–71.1 Access to laboratory protocols 81.735 60 0.033 7.7–39.1 Print subscriptions to scholarly journals 73.601 60 0.112 22.0–50.0 Assistance from library personnel 72.878 60 0.123 23.8–63.4 Library databases (such as INSPEC) 72.005 60 0.138 59.7–81.0 Space to study/conduct research 70.777 60 0.161 26.8–47.6 Electronic access to archives of scholarly journals 57.777 60 0.557 86.5–100.0 Electronic access to current scholarly journals 57.171 60 0.580 88.1–100.0 Physical book collection 54.691 60 0.669 60.5–79.3 Electronic book collection 49.970 60 0.819 43.9–71.4 How Unique Are Our Users? 525 providing access to both print and elec- tronic journals and books. However, the value that faculty place on services such as document delivery and interlibrary loan is significantly tied to their home institution. To better understand this difference, figures 1, 2, and 3 show the percentage of faculty indicating the importance they place on interlibrary loan, assistance from library personnel, and space to study and conduct research by institution. Responses of important and very important were combined, as were responses of unim- portant and not applicable because the researchers felt these responses would have similar implications for practitioners. It is interesting to note that the importance faculty place on interlibrary loan was shown to be statistically significantly as- sociated to institution (p≤0.001). Yet, when you look at the responses by institution when the percentage that rates the ser- vice “very important” and “important” are combined, it appears to matter very little which institution you surveyed—the response would indicate that interlibrary the business of fostering personal com- munication between their faculty, this would have minimal implications for practitioners. None of the responses to how faculty members became aware of less recent journal articles were sta- tistically significantly associated to the respondents’ institutions (see table 6). Using the Library The frequency of in-person library vis- its reported by the faculty members is significantly related to their institution (Pearson χ2 Value=130.418; p≤0.000). This seems logical as many factors contribute to the willingness and/or need for fac- ulty members to visit a library in person, such as the robustness of the library’s online presence, availability of electronic resources, the location of the library in proximity to the faculty member’s office, and even the aesthetics of the library. Of the sixteen institutions included in this study, fourteen have an engineering or science library, which could also contrib- ute to in-person visitation differences. The findings suggest that librarians will need to rely on their own data when determining how frequently faculty visit their library in person. The engineering faculty were asked to rate eleven library services as very im- portant, important, neutral, unimportant, or not appli- cable to their needs. With regard to the importance that engineering faculty members place on vari- ous library services, there seems to be a distinction between physical space, the services themselves, and providing access to resources. Regardless of institution surveyed, re- searchers are likely to get similar results when asking about the importance that faculty place on libraries FIgURE 1 Importance Placed on Libraries Providing Interlibrary Loan by Institution 526 College & Research Libraries November 2011 loan is an important library service. The University of Texas faculty had the smallest percentage of faculty rating interlibrary loan as important, with 52.7 percent rating it as either “very important” or “important”; this is still a majority of the respondents indicating that the ser- vice is important to them. For planning purposes, in most institutions, a majority of respondents indicating that a service is important would likely be enough to inform decision-making. Thus, while the chi-square analysis of association would lead one to believe that institutions should not rely on others’ data regarding the im- portance of library services to its users, a closer look at the data itself suggests that the difference may not be significant in the realities of library work. With that said, it is important to take a closer look at figure 2, which depicts the percentage of faculty indicating the impor- tance they place on assistance from library personnel by institution. The importance that faculty place on assistance from library personnel was not shown to be statistically significantly associated to institution (p=0.123). This suggests that, regard- less of the institution surveyed, researchers could expect to receive similar results. However, a look at figure 2 tells a different story. If, for example, the study was conducted solely at UCLA, one might get the impression that more faculty find library assistance unimportant or not applicable than find it important (though 42.8 percent of the UCLA respondents remained neutral on the topic). This finding might call into question the necessity of reference departments at engineering librar- ies across the country. By contrast, if the study had been conducted at the University of Oklahoma, the results would suggest that assistance from library personnel was considered important to the majority of engi- neering faculty (63.4%) and have entirely different implications for practitioners. Figure 3 paints a similar picture: the importance that faculty place on libraries providing a space to study and conduct research was not shown to be statisti- cally significantly associated to institution (p=0.161). Again, while the chi-square analysis indicates that, regardless of the institution surveyed, researchers could expect to receive similar results, the re- sults themselves suggest otherwise. Of the sixteen institutions included in the study, seven had a greater percentage of faculty that indicated space to study or conduct research was important or very impor- tant than indicating it was unimportant or not applicable. UCLA had the most faculty members indicating that library space was important to them, with 47.6 percent of the faculty rating it as either important or very important; however, 33.3 percent of UCLA faculty rated it as either unimportant or not applicable. At Iowa State University, a mere 22 percent of faculty members rated library space to study or conduct research as important, while 58 percent rated it as unimport- fIgurE 2 Importance Placed on Libraries Providing Assistance from Library Personnel by Institution How Unique Are Our Users? 527 ant or not applicable. While the extent of importance varies by institution, the fact that no institution had a majority of faculty members who indicated that space for study and research was highly valued by them seems to indicate that the data found at one institution may indeed be “good enough” for other institutions. Other For each question where “Other (please describe)” was listed as an answer, a sta- tistically significant association between the response and the respondents’ institu- tions was found. However, for the pur- poses of analysis, the “other” answer was counted the same as any other answer and thus did not provide meaningful insight about the differences among institutions and were omitted from the tables present- ing the results for each question. Conclusion The findings in this study emphasize the importance of practitioner-researchers fully articulating the scope of their re- search, carefully defining and represent- ing the study’s population as well as the home institution, and honestly explaining the study’s limita- tions. Of course, practitioners must also be cautious consumers of information and fully under- stand research results before as- suming generalizability to their library population. While the data analysis suggests that survey data gathered at one institution about a defined population’s information- seeking behavior and use would yield similar re- sults to the data gathered from the same population at a similar institution, the value of this to practitioners may be limited. Yes, practitioners may safely be able to use the information, but what would they do with it? The researchers in this study crafted the survey instrument by taking questions from similar information-seeking behavior surveys and rephrasing and/or updat- ing the questions and answer choices. This was a deliberate choice, since the purpose of the study was to analyze the usefulness of single-institution studies and the potential for building a culture of shared assessment within the profes- sion. It was important for the survey to closely mirror those used by other practitioner-researchers. The responses to the frequently asked survey questions make it fairly easy to describe what types of information a given discipline values and how they discover source informa- tion as well as for what purposes they seek information; but, as practitioners, it left us asking, “So what?” If we had these data for any given institution, what could we do with it to inform decision- making, and our collective response was ultimately, “Very little.” For whatever reason, it seems that the questions asked in our profession’s admi- rable quest to better understand a specific user group have little, if any, practical implications. Perhaps qualitative stud- ies, such as focus groups or observations, FIgURE 3 Importance Placed on Libraries Providing Space to Study/Conduct Research 528 College & Research Libraries November 2011 Appendix: Information-Seeking Habits of Engineering Faculty Survey 1. What is your status within your university? q Professor q Associate Professor q Assistant Professor q Adjunct Professor q Instructor q Lecturer q Professor Emeritus q Other (please specify): 2. Which branch of Engineering is your area of emphasis? (Please select the answer most closely aligned with your area.) q Aerospace q Biomedical/Bioengineering q Chemical q Civil q Computer Science q Electrical q Environmental q Industrial q Mechanical q Petroleum q Other (please specify): 3. How long have you been a faculty member/researcher in your area of study? q 0–5 years q 6–10 years q 11–15 years q 16+ years are necessary to provide the “why” and “how” answers that practitioners need to inform their decision making, but we are hopeful this is not the case. Qualitative studies are often costly to conduct and require time-intensive analysis. Surveys are easily distributed and seem to be a preferred method of data gathering for many practitioner-researchers. As a pro- fession, we need to ensure that surveys yield results that are meaningful to the practitioner and not just simply interest- ing to know. This study is just the first of many needed to fully understand how to make survey results from information-seeking behavior studies relevant to practitio- ners. The authors intend to reconstruct the survey instrument to collect infor- mation that would be meaningful to practitioners and then conduct future multi-institutional studies that examine faculty and/or students in other disci- plines to see if the findings hold true in the social sciences, humanities, fine arts, and/or life sciences. How Unique Are Our Users? 529 4. Which of the following are included in your departmental duties? (Select all that apply.) q Undergraduate Instruction q Graduate Instruction q Laboratory Research q Field Research q Commercial/Proprietary Research q Supervision of Doctoral Research q Grant Preparation q Other (please specify): 5. How many of the following have you completed within the last 5 years? 0 1–3 4–7 8–11 12+ Refereed journal articles or book chapters Nonrefereed journal articles or book chapters Conference proceedings Grant applications Patents/commercial projects Books 6. How frequently do you seek and/or access information to complete the following tasks? Daily Weekly Monthly 1–2 times/ semester Annually N/A Prepare for student lectures Prepare for conference presenta- tion Determine protocols for labora- tory procedures Research patents Write/research for publication Prepare new research proposal/ grant application Professional development/remain current in field 7. How important are the following in helping you with your research? Very Important Important Neutral Unimportant N/A Scholarly journals Internet resources Books 530 College & Research Libraries November 2011 Textbooks Attendance at conference E-mail discussion with a colleague Face-to-face discussion with a colleague E-mail discussion with a student Face-to-face discussion with a student 8. How do you keep abreast of current developments in your field(s)? (Please check all that apply.) q Scanning current issues of journals q Scanning recent issues of abstracting/indexing tools q Personal communication q Attendance at conference q Follow references or leads from an article or item of interest (citation trail) q Electronic discussion lists q RSS feeds q Current Awareness service q Other (please describe): 9. How do you become aware of other less recent journal articles? (Please check all that apply.) q Citations at end of journal articles q Citations at end of book chapters q Retrospective searching of indexing/abstracting tools q Personal communication q Browsing through older volumes q Other (please describe): 10. How often did you visit the library in person in the last 12 months? q Never q 1–2 visits q 3–5 visits q 6–12 visits q 13–23 visits q 24 or more visits 11. How important are the following library services in meeting your information needs? Very Important Important Neutral Unimportant N/A Electronic access to current scholarly journals Electronic access to archives of scholarly journals Print subscriptions to schol- arly journals How Unique Are Our Users? 531 Physical book collection Electronic book collection Access to laboratory protocols Library databases (such as INSPEC) Interlibrary loan Document delivery Space to study/conduct research Assistance from library personnel Other (please specify) 12. Are there services your university library does not currently provide, but you wish they did? If so, please explain how these services would assist you in meeting your information needs. Notes 1. Rebecca Watson-Boone, “Academic Librarians as Practitioner-Researchers,” Journal of Academic Librarianship 26, no. 2 (2000): 85–93. 2. Ibid., 87. 3. Peter Jarvis, The Practitioner-Researcher: Developing Theory from Practice, (San Francisco: Jossey-Bass, 1999). 4. Ibid., 127. 5. Ibid., 4. 6. Ibid., 124. 7. Ibid., 122. 8. Ibid., 84–85. 9. Charles R. Hildreth and Selenay Aytac, “Recent Library Practitioner Research: A Method- ological Analysis and Critique,” Journal of Education for Library and Information Science 48, no. 3 (2007): 236–58. 10. Ibid., 249. 11. Ibid., 254. 12. Jeffrey Katzer, “ALA and the Status of Research in Library/Information Science,” Library and Information Science Research 11 (1989): 83–87; Thomas Childers, “Will the Cycle Be Unbroken? Research and Schools of Library and Information Studies.” Library Trends 38 (1984): 521–35; Mary Jo Lynch, “Research and Librarianship: An Uneasy Connection,” Library Trends 38 (1984): 367–83. 13. Charles R. McClure and Ann Bishop, “The Status of Research in Library/Information Sci- ence: Guarded Optimism,” College and Research Libraries 50, (1989):127–43. 14. Ibid., 138. 15. Nancy A. Van House, “Assessing the Quantity, Quality, and Impact of LIS Research,” in Library and Information Science Research: Perspectives and Strategies for Improvement, ed. Charles R. McClure and Peter Hernon (Norwood, NJ: Ablex Publishing, 1991), 85-100. 16. Ibid., 93. 17. Ibid., 96. 18. J. Michaels, “Library and Information Science Research: Are We Scholars or Parasites?” Library & Information Science Research 15, no. 4 (1993): 297–98. 19. Ibid., 298. 20. Peter Hernon, “Components of the Research Process: Where Do We Need to Focus Atten- tion?” The Journal of Academic Librarianship 27, no.2 (2001): 81–89. 21. Hildreth and Aytec, “Recent Library Practitioner Research,” 237. 532 College & Research Libraries November 2011 22. Rowena Cullen, “Perspectives on User Satisfaction Surveys,” Library Trends 49, no. 4 (2001): 662–86. 23. Peter Hernon and Candy Schwartz, “LIS Research—Multiple Stakeholders,” Library & Information Science Research 21, no. 4 (1999): 423–27. 24. Doris J. Schlichter and J. Michael Pemberton, “The Emperor’s New Clothes? Problems of the User Survey as a Planning Tool in Academic Libraries,” College & Research Libraries 53, no. 3 (1992): 257–65. 25. Steve Hiller, “Assessing User Needs, Satisfaction, and Library Performance at the University of Washington Libraries,” Library Trends 49, no. 4 (2001): 605–25. 26. Hildreth and Aytec, “Recent Library Practitioner Research,” 241. 27. Ibid., 253. 28. Watson-Boone, “Academic Librarians,” 90. 29. Steve Hiller, “How Different Are They? A Comparison by Academic Area of Library User, Priorities, and Information Needs at the University of Washington,” Issues in Science and Technology Librarianship 33 (2002), available online at www.istl.org/02-winter/index.html [accessed 3 March 2010]. 30. George R. Plosker, “Conducting User Surveys: An Ongoing Information Imperative,” Online 26, no. 5 (2002): 64–68. 31. Ibid., 68. 32. Jane Quigley, David R. Peck, Sara Rutter, and Elizabeth McKee Williams, “Making Choices: Factors in the Selection of Information Resources among Science Faculty at the University of Michigan,” Issues in Science and Technology Librarianship 34 (2002), available online at www.istl. org/02-spring/refereed.html [accessed 3 March 2010]. 33. Eileen G. Abels, Peter Liebscher, and Daniel W. Denman, “Factors that Influence the User of Electronic Networks by Science and Engineering Faculty at Small Institutions,” Journal of the American Society for Information Science 48, no. 6 (1997): 496–507. 34. Michael J. Aked, John C. Phillips, Karen Reiman-Sendi, Kevin Risner, Kathleen J. Voigt, and Judy Wiesler, “Faculty Use of an Academic Library Reference Collection,” Collection Building 17, no. 2 (1998): 56–64. 35. Ibid., 58. 36. John M. Budd and Lynn Silipigni Connaway, “University Faculty and Networked Infor- mation: Results of a Survey,” Journal of the American Society for Information Science 48, no. 9 (1997): 843–52. 37. Cecelia M. Brown, “Information Seeking Behavior of Scientists in the Electronic Information Age: Astronomers, Chemists, Mathematicians, and Physicists,” Journal of the American Society for Information Science 50, no. 10 (1999): 929–43. 38. Ibid., 937. 39. Nancy H. Dewald, “What Do They Tell Their Students? Business Faculty Acceptance of the Web and Library Databases for Student Research,” The Journal of Academic Librarianship 31, no. 3 (2005): 209–15. 40. University of Washington Libraries, UW Libraries Assessment (2009), available online at www.lib.washington.edu/assessment/ [accessed 3 May 2010]. 41. Lizabeth A. Wilson, “Local to Global: The Emerging Research Library,” Journal of Library Administration 48, no. 2 (2008): 127–39. 42. Xi Niu, Bradley M. Hemminger, Cory Lown, Stephanie Adams, Cecelia Brown, Allison Level, Merinda McLure, Audrey Powers, Michele R. Tennant, Tara Cataldo, “ National Study of Information Seeking Behavior of Academic Researchers in the United States,” Journal of the American Society for Information Science and Technology 61, no. 5 (2010): 869–90, available online at http://dx.doi.org/10.1002/asi.21307 [accessed 24 May 2010] 43. Brown, “Information Seeking Behavior”; Bradley M. Hemminger, Dihui Lu, K.T.L. Vaughn, and Stephanie J. Adams, “Information Seeking Behavior of Academic Scientists,” Journal of the American Society for Information Science and Technology 58, no. 4 (2007): 2205–25; Quigley et al., “Making Choices”; Margaret Stieg Dalton and Laurie Charnigo, “Historians and Their Informa- tion Sources,” College & Research Libraries 65, no. 5 (2004): 400–25.