Five Years Later: Predicting Student Use of Journals in a New Water Resources Graduate Program Previous Contents Next Issues in Science and Technology Librarianship Winter 2011 DOI: 10.5062/F40Z7152 Five Years Later: Predicting Student Use of Journals in a New Water Resources Graduate Program Andrea A. Wirth Geosciences & Environmental Sciences Librarian andrea.wirth@oregonstate.edu Margaret Mellinger Engineering Librarian margaret.mellinger@oregonstate.edu Oregon State University Libraries Corvallis, Oregon Copyright 2011, Andrea A. Wirth and Margaret Mellinger. Used with permission. Abstract Using citation analysis, the authors examined the journals cited in theses and dissertations over the first five years of the Water Resources Graduate Program at Oregon State University. These journal titles were compared to the titles predicted as being important in the 2003 Oregon State University Libraries new program (Category I) review. A preliminary core list of journals for this highly interdisciplinary program had been created for the Category I review from faculty publications, Journal Citation Reports, and Interlibrary Loan (ILL) requests. Through comparison of the citations in the students' works with the data compiled for the Category I, and a review of current ILL requests, the authors address whether the initial review predicted student use, which sources were most valuable for predicting use, and how analysis of student citations can add to the strength of the original core journal list. Introduction In 2003, Oregon State University (OSU) initiated a new Water Resources Graduate Program (WRGP), which awards M.S. and Ph.D degrees and brings together faculty and students from six colleges and multiple departments. Students take common core courses and do additional work in specific degree areas in this highly interdisciplinary program. Degree areas include Water Resources Engineering, Water Resources Science, and Water Resources Policy and Management (Water Resources Graduate Program 2008). New degree programs at OSU begin as Category I Proposals, which are used to designate major curricular changes requiring approval at the state level. Category I Proposals require an evaluation of the library collections expected to support the program. The library evaluation includes descriptive information about the subject areas, such as call number ranges, number of books and journals held, the rate of book purchase in the subject area, cost and coverage data to maintain the collection, and other important information sources for the discipline. Evaluations can include comparisons of the OSU Libraries (OSUL) collections to peer institutions identified by the authors of the Category I Proposal. Journal holdings evaluations can include comparisons against lists of core titles, lists of journals with high impact factors, lists of journals where faculty publish, or other relevant metrics. Recommendations for improvements to the collection and the implications for budgeting are also submitted with the library evaluation. A full description of this process can be found in Bobal et al. (2008). In the 2003 Water Resources Graduate Program Category I library evaluation (hereafter referred to as "2003 WRGP assessment"), librarians analyzed the collection, identified gaps and deficiencies, and made recommendations for improvements. Based on this evaluation, the University provided funding to supplement library collections for three years. This paper describes a comparison of the 2003 WRGP assessment and a citation analysis of the theses and dissertations published by graduate students in the WRGP. The comparison was used to determine if the titles identified as important and/or recommended for purchase in the 2003 WRGP assessment were predictive of the journals cited by WRGP students in their theses and dissertations between 2004 and 2009. Literature Review This literature review highlights selected articles addressing the use of citation analysis of theses and/or dissertations in three areas of interest to the current study; these articles were mined for methods, purpose, and terminology. Citation studies in water research and other interdisciplinary fields were read for approaches to core lists of journals and to determine the novelty of this study. Articles on evaluation of collections for new programs were reviewed to see if prior research employed citation analysis to retrospectively evaluate library collections developed in response to new degree programs. Citation analyses of theses and dissertations can be undertaken across institutions with large sets of citations as Zipp (1996) did with theses in geology and biology, or within institutions using a smaller data set, as Gooden (2001) did with chemistry doctoral dissertations. Research using citation analysis with graduate theses "tends to focus on collection management or evaluation issues and almost exclusively uses graduate theses to examine collection patterns at a particular institution" (Kushkowski et al. 2003). A common purpose for conducting citation analysis of these materials is for collection management decisions such as serials selection or cancellation (Edwards 1999; Haycock 2004; Kuruppu and Moore 2008; Sinn 2005; Smith 2003; Sylvia and Lesher 1995; Thomas 1993; Walcott 1994). The current study is an example of a small, local citation analysis used for the particular purpose of evaluating the use of journals supporting a new degree program. Although citation analyses can be found for related disciplines such as forestry (Brazzeal and Fowler 2005) or engineering (Kriz 1984; Williams and Fletcher 2006), few such studies exist in the library literature for water resources. Of the studies that do exist are two analyses of water resources publication-citation patterns coauthored by Richard Walker. Walker and Zuck (1976) reviewed the citation patterns of the publications of the Wisconsin Department of Natural Resources by analyzing the type of publications used, along with the subject areas and language of the cited materials. They found few journals were cited. The few journals that were cited represented 35 percent of the citations, and showed "widely scattered subject coverage." In a later study using the Selected Water Resources Abstracts database, Walker and Ahn (1994) analyzed a sample of water resources publications and their citations, to determine citation patterns for federally funded research. Their conclusion was that "it is fair to say that the source of support affects the use of the literature." These articles highlight the importance of consulting research that goes beyond strict disciplinary boundaries and illustrate that government works and grey literature contribute significantly to the body of knowledge in water resources, both of which are aspects that have relevance in today's water research environment. However, no study addressing the citation patterns of water research graduate students was identified during the review of the literature. The current study begins to address this gap. An underlying issue in this study and in conducting the original collection evaluation for the WRGP is the interdisciplinary nature of water resources. Quantifying interdisciplinarity is difficult, but tools to do so exist, as Dilevko and Dali (2004) described in their article on tourism studies scholarship. Dilevko and Dali recommended Library of Congress (LC) classification analysis of citations as a way "to get a better sense of the information scatter typically associated with an interdisciplinary field and to gain insight into how interdisciplinary scholars do their intellectual work." An illustration of the increasing level of interdisciplinarity in many fields is shown in Brazzeal and Fowler's (2005) review of forestry theses citations from Mississippi State University in order to ascertain which types of research materials the students were using (e.g., books, journal articles, etc.). Their examination of journal citations revealed that a large portion of the journals cited by forestry students were more closely associated with other departments on campus. They advise consideration of the impact of collections decisions on other departments, and note that a "wide range of indexes and databases should be consulted when assisting forestry researchers." There are several papers describing formal and informal processes for evaluating library collections and services for new programs. Johnson (2000) described a method for evaluating collections for new environmental sciences programs at Texas Tech University based on recent publications of the faculty who would be part of the new program. The citation analysis Johnson undertook was in addition to a collection assessment that included a description of the current collection, document delivery service options, and acquisition spending. In a brief look to possibilities for further analyses, Johnson suggested that a citation analysis of graduate theses in this new program could be beneficial as a follow-up review. Sinha and Tucker (2005) described a standardized process for assessing new programs at the University of Nevada, Las Vegas (UNLV) developed in response to a significant increase in the number of program proposals and requests for librarian input on them. A task force of three subject librarians created guidelines for things to consider in the assessment, such as carefully reading the new program proposal summary, looking for core lists of resources, and consulting accreditation standards. A template made the assessment product more consistent and complete, and prompted librarians to address the types of materials needed: books, journals and backfiles, electronic resources, government publications, and non-book materials. Additional aspects covered in the analysis were inflation, public services costs and expected document delivery. As the article described a new process, Sinha and Tucker did not include recommendations about later evaluation of programs. Bobal et al. (2008) described OSUL history and processes of evaluating new program proposals. The concept of using guidelines and templates to drive the analysis was similar to that used at UNLV. In addition, OSUL guidelines recommend the use of specific tools such as WorldCat Collection Analysis, Journal Citation Reports, the reporting feature of ILLiad for Interlibrary Loan statistics and database comparison tools such as Gold Rush. Because OSUL has a history of such assessments, the article also addressed available routes for doing follow-up evaluations. Two of those routes are the accreditation process and the 10-year review of programs, each including library assessments. The five-year review of new programs does not require a library component. The current study provides a way to evaluate the collection at the five-year point and builds on Johnson's (2000) recommendation to use citation analysis of graduate theses as a means of collection review for a new program. Methods There were few enough WRGP theses and dissertations that the entire set was analyzed. All citations were entered into a spreadsheet. Data collection included compiling data from the 2003 WRGP assessment and current Interlibrary Loans (described in detail below) in a way that facilitated data comparisons. To determine if the 2003 WRGP assessment predicted the use of these titles, the authors examined OSU faculty publishing, 2002 JCR, and 2002-2003 ILL lists. In addition, 2005-2009 ILL data was compared against the list of WRGP citations. Collectively, these comparisons provided a method to evaluate retrospectively whether the sources used at the time of the 2003 WRGP assessment were predictive of student use. Microsoft Excel was used for this purpose, as its features of sorting, filtering, and custom searching addressed the needs of the study. Data sources Data used for comparison in this study are as follows: Dataset 1: Data from the 2003 WRGP assessment appendices The following 3 sets of data were used during the 2003 WRGP assessment to identify titles not held by the library and make recommendations based on the data. Faculty publishing Faculty members who would be supporting the WRGP were identified in the text of the 2003 proposal. Working from the list of faculty, 136 journals were identified as sources of OSU water research faculty publications. The rationale for using the list of faculty publications was that they would constitute the pool of advisors available to the WRGP students, and therefore would either directly or indirectly influence student selection of research materials. 2002 Journal Citation Reports (2002 JCR) The 50 titles in the 2002 JCR Water Resources category comprised this dataset. As with the faculty publications titles, the list was compared to the OSUL holdings, with an emphasis on gaps in the collections, especially for the titles with higher impact factors. 2002-2003 Interlibrary Loan requests (2002-2003 ILL) ILL reports were run on the OCLC ILLiad system to determine the demand for water-related titles. The data were compiled from January 2003 through September 2003. Librarians reviewed the titles obviously relevant to water resources research that were requested from the colleges involved in WRGP. Next, titles were checked to verify that they were unavailable at OSUL. Finally, librarians made a projection of anticipated ILL requests for subsequent years with associated copyright fees, leading to a comparison of the fiscal support for beginning subscriptions versus continued reliance on ILL. Dataset 2: Citations from Water Resources Graduate Program theses (WRGP citations) The theses and dissertations produced by WRGP students in a 5-year period provided citation data used to compare to Dataset 1. A search of the OSUL catalog yielded a group of 15 theses and three dissertations (hereafter referred to collectively as "theses") submitted between August 2004 and July 2009. The references from all the theses were entered into a spreadsheet. The authors identified the following data elements for each citation: author, publication year, title (for article, web site, and book) journal title (with volume, issue, and page numbers) WRGP degree track (engineering, policy and management, or science) M.S. or Ph.D. thesis/dissertation author advisor Since the authors wished to quantify the total body of journals that were used by the students and the number of citations to each of the titles, the journal titles were normalized from the varied citation styles employed by the students. The normalized titles were then counted and organized into a separate spreadsheet for further analysis. The term "journal" was used to describe periodical publications that exclude proceedings and newspapers (Reitz 2007). Citations that did not refer to journals were categorized as "other," "book," or "web site." Each of these was entered into a separate column in the spreadsheet to enable sorting, filtering, and grouping by field. Dataset 3: 2005-2009 Interlibrary Loan requests (2005-2009 ILL) An ILL report was run on archived OCLC ILLiad data to obtain the numbers of requests from graduate students for water-related journals since the advent of the program. Complete data was available only for the years 2005-2009, rather than from 2004 when the first thesis was produced. Another shortcoming of this data set was that the report included the number of article requests by college, but not by program. Although the researchers limited the request data to that from the colleges in the WRGP, it is likely that some of the requests were from graduate students who were not in the WRGP. While it would have been helpful to have an LC call number field included in this report, it was not an option, so the authors looked at each journal title on the list to see if it was water-related. Results The 18 theses yielded 1,794 citations for analysis. The total number of journals cited in the WRGP theses (accounting for title changes), was 328. The results are outlined below, including an overview of the WRGP citations, and a comparison of WRGP citations against the faculty publishing, 2002 JCR, 2002-2003 ILL, and 2005-2009 ILL datasets. The majority of citations came from journals (Table 1). The number of citations per journal title ranged from 1 to 141. The second most commonly cited material type was the category of "other," which included materials such as U.S. Geological Survey publications. Table 1: WRGP theses citations by material type Material type Number of Citations (percent) Journal 1,054 (59%) Book 257 (14%) Web site (non-governmental only) 35 (2%) Other (includes print and online government documents) 448 (25%) total 1,794 The majority of journals were cited only once (Table 2). Journal titles cited five or more times were defined as highly cited. These 37 titles account for 60 percent of the total citations, whereas 215 titles were cited only once and make up 20 percent of the citations (Table 3). Table 2: Distribution of WRGP thesis journals and associated citations by the number of times cited Times cited Journals (percent of total)* Citations (percent of total)* 1 215 (66%) 215 (20%) 2 42 (13%) 84 (8%) 3 18 (6%) 54 (5%) 4 16 (5%) 64 (6%) 5+ 37 (11%) 637 (60%) Total 328 1,054 *percentages do not add to 100% due to rounding Table 3: Highly cited journals (cited 5 or more times) and their occurrence on the 2003 WRGP assessment datasets Journal Times Cited Faculty Publishing 2002 JCR 2002-2003 ILL Water Resources Research 141 Y Y Y Journal of Hydrology 71 Y Y N Hydrological Processes 41 Y Y Y Ground Water 34 Y Y N Advances in Water Resources 24 Y Y N Environmental Science & Technology 20 Y N N Journal of Contaminant Hydrology 20 Y Y N Journal of the American Water Resources Association 20 Y Y N Society & Natural Resources 18 N N N Science 16 N N N Soil Biology & Biochemistry 16 N N N Soil Science Society of America Journal 16 Y N N Environmental Management 15 Y N Y Journal of Environmental Quality 14 Y N N Journal of Geophysical Research 14 Y N N BioScience 12 Y N N Canadian Journal of Fisheries & Aquatic Sciences 11 N N N Biogeochemistry 10 Y N N Journal of Irrigation & Drainage Engineering 10 Y Y N Nature 10 N N N Transactions of the ASABE 10 Y N N Physical Review E 8 N N N Annals of Glaciology 7 Y N N Hydrological Sciences Journal 7 Y Y N Limnology & Oceanography 7 N N N Environmental Impact Assessment Review 6 N N N EOS 6 Y N N Impact Assessment & Project Appraisal 6 N N N Journal of Glaciology 6 N N N Proceedings of the National Academy of Sciences 6 N N N Agricultural Water Management 5 Y Y Y Geochimica et Cosmochimica Acta 5 N N N Geomorphology 5 Y N N Journal of Hydrology (New Zealand) 5 N N N Journal of the North American Benthological Society 5 N N N Soil Science 5 Y N N Vadose Zone Journal 5 N N N Total: citations 637 Y=22 Y=10 Y=4 Faculty publishing results Of the 136 titles on the faculty publishing list, the WRGP students cited 51 in the following five years. These 51 titles accounted for 568 (or 54 percent) of the total journal citations and included 22 highly cited journals, and 29 cited from one to four times. JCR and impact factor results The titles on the JCR Water Resources category list (2002) were reviewed for the 2003 WRGP assessment. For the current study, the authors compared the list of journals cited in WRGP theses to those on the 2002 JCR and found that 26 (52 percent) of the 50 JCR titles were used by the students and accounted for 403 (38 percent) of the total journal citations. These included 10 highly cited journals and 16 cited from one to four times. The journals on the 2002 JCR list had 2002 impact factors ranging from .012 to 1.717. Of the 10 titles in the 2002 Water Resources JCR category with the highest impact factors (1.081-1.717), nine were cited by the students (344 citations). Expanding to the titles with impact factors in the top 25 (.661-1.717), 16 were cited (377 citations). 2002-2003 ILL results Half of the 18 journals represented on the 2002 ILL list are titles that OSUL held in part or complete run at the time of the 2003 WRGP assessment. Collection gaps, lack of back files and issues unavailable at the time of the request (bindery, missing, etc.) may account for the number of requests for articles from journal titles owned by OSUL. Ten of the 18 titles from the 2002 ILL list were cited in the theses. Titles from the 2002 ILL list accounted for 214 (20 percent) of the total WRGP citations. Four of those journals were highly cited titles. In summary, the WRGP assessment showed that each list used in the 2003 library assessment included titles that were drawn upon extensively by WRGP students and a considerable number of other titles that were either used minimally or not used at all (Figure 1). Figure 1: Faculty publishing, 2002 JCR, and 2002-2003 ILL by percentage of titles highly cited, cited 1-4 times, and not cited Overlap between faculty publishing, 2002 JCR and 2002-2003 ILL A master list of journal titles was compiled from the 2003 WRGP assessment appendices: faculty publishing, 2002 JCR titles and 2002-2003 ILL titles. After de-duplicating the master list, 183 unique titles were identified. Some of these titles occurred on more than one dataset and of course, as shown in Figure 1, many titles on these lists were not cited at all. To help address the individual value of a particular list used to identify important titles, the authors reviewed the number of unique titles cited and shared titles cited on each dataset. The comparison is shown in Table 4. Only four titles appear on all three lists, and only 11 on any two datasets. Much more numerous are the titles cited by the WRGP students that did not appear on the 2003 WRGP assessment at all (260 titles). Table 4: Distribution of journals and associated percentage of citations across datasets Faculty publishing only 2002 JCR only 2002-03 ILL only On two datasets On all three datasets Unique to theses Total Journals (percent) 37 (11%) 12 (4%) 4 (1%) 11 (3%) 4 (1%) 260 (79%) 328 Citations (percent)* 177 (17%) 23 (2%) 7 (<1%) 207 (20%) 188 (18%) 452 (43%) 1054 *percentages do not add to 100% due to rounding Dataset 3: 2005-2009 ILL results When comparing WRGP citations to 2005-2009 ILL data, the authors found that several of the theses-cited journals were among the top 20 in ILL requests from 2005-2009 (Table 5). Of the titles cited over five times as well as requested over eight times, OSUL subscribed to all, at least partially. Hydrological Processes was the most cited journal in this comparison and Agricultural Water Management was the most requested journal (see bolded titles in Table 5). Librarians noted the importance of these two titles to the subject in the 2003 WRGP assessment. They also ran a projection of the number and costs of future ILL requests for these two journals and found that the costs for requests would be unlikely to outpace the rising subscription costs. This prediction turned out to be true. Subscribing to the full run of Hydrological Processes would have cost OSUL $2,830 in 2003-2004. Given a minimum of 5 percent inflation per year, OSUL would have spent over $20,000 on this subscription over five years. The 23 requests for this title over the same time cost the Libraries less than $600 (presuming a maximum ILL cost of $23 per article). Re-subscribing to the cancelled Agricultural Water Management would have cost $1,519 in 2003-2004, and more than $8,000 over the next five years. The 44 requests for articles from this journal over five years cost the libraries an estimated $1,000. As use increasingly drives library subscription decisions, it will become more important to weigh the benefits of subscription over access. One unknown variable to consider is how online availability drives use. While it is gratifying to see the cost savings from not subscribing to these journals, it is worth questioning if use would have been higher had the journal been readily available. Nature and Science were both held in full by OSUL. Their appearance in the 2005-2009 ILL data can possibly be attributed to changing practices in ILL rejections, missing issues or articles, or to a period of time when the online subscription to Nature was disconnected because of licensing issues. Table 5: Comparison of WRGP Citations to 2005-2009 ILL requests Journal Title Library Holdings 2002-2003 ILL WRGP citations 2005-2009 ILL Agricultural Water Management Until 1998 Y 5 33 Applied Biochemistry & Biotechnology Y begins in 1996 N 1 27 Epidemiology Y 1990 -2004 N 2 25 Hydrological Processes 2002-present Y 41 23 Oecologia Y N 3 21 Journal of Environmental Planning & Management N N 2 21 Nature Y N 10 18 Science Y N 16 13 Transport in Porous Media, Y N 4 12 Environmental Monitoring & Assessment Y 1997 - present Y 1 12 Environmental Toxicology & Chemistry N N 1 12 Water Science & Technology N Y 1 12 International Journal for Numerical & Analytical Methods in Geomechanics N N 1 11 Soil Science Society of America Journal Y N 16 9 Journal of the North American Benthological Society Y lack v. 1-4 N 5 9 Journal of the American Water Resources Association Y N 20 8 Soil Science Y N 5 8 Chemical Engineering Communications Y N 1 8 Journal of Dairy Science Y N 1 8 Journal of Great Lakes Research Until 1982 N 1 8 Discussion of Results Predicting future use with faculty publishing, 2002 JCR, and ILL data analysis Analysis of the 2003 WRGP assessment shows faculty publishing was most predictive of future graduate student citation when reviewing the three sources individually. This is true whether looking at the number of journals cited or the percentage of citations represented in the WRGP theses (Table 4). It is also clear that the 2002 JCR and 2002-2003 ILL lists each highlighted unique titles subsequently cited by the students, making each valuable for identifying potentially useful titles. In addition, journals appearing on two or three datasets garnered a greater number of citations than those appearing on any single dataset. Many titles identified in the 2003 WRGP assessment were not used at all by the students (85 from faculty publishing, 24 from 2002 JCR, and eight from the 2002-2003 ILL list). Accounting for overlap between these lists means that 115 of 183 total unique titles were not cited. The 2003 WRGP assessment assumed a connection between faculty publishing and future citation behavior, especially as Zipp (1996) found that faculty citation patterns are a predictor of student citation patterns. Although there was a connection, the authors of this study expected a stronger relationship between the journals in which faculty publish and the journals students cite. It may be that faculty publishing is not as strong a predictor of student citations as are faculty citation patterns. Reasons for the lack of citations to nearly half of the JCR titles are also unknown, but most JCR titles fall into the science and engineering aspects of water research. Students in the policy and management (PM) track of the WRGP may use social science titles more frequently and therefore their resource needs might not have been accurately reflected in the 2002 JCR Water Resources category. The highly cited journals used by PM students are: Society & Natural Resources (18 of 18 are PM citations) Environmental Management (13 of 15 are PM citations) Science (10 of 16 are PM citations) BioScience (9 of 12 are PM citations) Environmental Impact Assessment Review (6 of 6 are PM citations) Impact Assessment & Project Appraisal (6 of 6 are PM citations) Proceedings of the National Academy of Sciences (5 of 6 are PM citations) The nature of OSUL ILL data precludes the tracking of requests by university affiliation other than college and academic status (e.g., graduate student). Therefore, the ILL data provides a check against obvious omissions in serials collections, rather than identifying core titles that are lacking to support a particular program. Journals cited by the students alone (not identified in the 2003 WRGP assessment) are numerous (260). Although most of those journal titles (196) were cited only once, 15 of them were highly cited and account for 130 (12 percent) of the total citations. These titles lend themselves to further analysis. Although three of these titles, Nature, Science, and PNAS, are clearly important for the sciences generally and therefore part of the OSUL collection at the time of the 2003 WRGP assessment, the subject areas of the remaining 12 titles are quite varied, as would be expected in a interdisciplinary field (Table 6). Since the 2003 WRGP assessment, JCR has added one of the 12 titles (Vadose Zone Journal) to the JCR Water Resources Category. The fact that some of the most highly cited journal titles did not appear on the 2003 WRGP assessment underscores the value of local citation analysis. The combination of the faculty publishing, the 2002 JCR Water Resources category, and 2002-2003 ILL requests is a good start, but it did not include all the titles needed by the new program. The WRGP theses citation analysis provides additional information that could not be expected to be captured in the original library review. Table 6: Highly cited titles absent from the 2003 WRGP assessment, with 2008 JCR categories and LC classes Journal Times Cited JCR 2008 categories LC Classes (from call numbers assigned in OSU Libraries catalog) Society & Natural Resources 18 Environmental studies Planning & development Sociology Economic history and conditions (HC) Science 16 Multidisciplinary sciences General science (Q) Soil Biology & Biochemistry 16 Soil science Natural history (QH) Canadian Journal of Fisheries & Aquatic Sciences 11 Fisheries Marine & freshwater biology Fisheries (SH) Nature 10 Multidisciplinary sciences General science (Q) Physical review E 8 Physics, fluids & plasmas Physics, mathematical General physics (QC) Limnology & Oceanography 7 Limnology Oceanography Natural history (QH) Environmental Impact Assessment Review 6 Environmental studies Environmental effects of industries and plants (TD) Impact Assessment & Project Appraisal 6 Not in JCR Environmental effects of industries and plants (TD) Journal of Glaciology 6 Geography, physical Geosciences, multidisciplinary Hydrology (GB) Proceedings of the National Academy of Sciences 6 Multidisciplinary sciences General science (Q) Geochimica et Cosmochimica Acta 5 Geochemistry & geophysics Mineralogy (QE) Journal of Hydrology (New Zealand) 5 Not in JCR Hydrology (GB) Journal of the North American Benthological Society 5 Ecology Marine & freshwater biology Ecology (QH) Vadose Zone Journal* 5 Environmental sciences Water resources* Soils/Soil science (S)** *Not in 2002 edition of JCR Water Resources category **LC Class from WorldCat since call number not assigned in OSUL catalog The titles cited only once (whether or not they were captured on 2003 WRGP assessment), provide a snapshot of what might be described as the "long tail" of water resources research at OSU. While the long tail seems extreme at 215 titles, it represents only 20 percent of the total citations. None of the data sets used in the library evaluation could have predicted the long tail titles because they cover such a broad range of disciplines. Some of the titles were unexpected; e.g., Brain Research Bulletin and Czechoslovakian Journal of Food Science are unlikely to appear on a list of core water resources journals. However, most of the titles in the long tail made sense in the context of the three degree tracks of the WRGP: engineering, policy and management, and science. Further Research One area of potential study is to use a type of citation analysis called title dispersion analysis to show the degree to which water research is interdisciplinary. Several authors have evaluated the inter- or multi-disciplinary nature of some fields through title dispersion analysis (Edwards 1999; Kuruppu and Moore 2008; Sherriff 2010; Sinn 2005). Title dispersion analysis reveals how authors in an interdisciplinary field use research from across a range of disciplines. Further analyzing the dispersion of WRGP citations by degree track could show patterns of use that would be helpful to librarians in understanding the particular emphases of the program. Another opportunity lies within faculty citation patterns as some authors have opted to analyze the correlation between faculty citations and graduate students within departments (Pancheshnikov 2007; Zipp 1996). The unanalyzed portion of the data concerning monographs and "other" resources presents opportunities as well. For example, the importance of books or government documents in water resources research could be studied, or web sites cited by students could form the "core" of essential links on water resources subject guides. Beyond the collections aspect of the study, there is also a benefit to WRGP faculty and students. In addition to conducting further research, the authors can share highlights of this study with program faculty, offering them new insights into what their students cite at a programmatic level. The aggregated patterns of student citations may spark conversations on the types of sources students cite and how they discover, access, and incorporate them. Conclusion The analysis of the three parts of the 2003 WRGP assessment led to a good understanding of the collection gaps and strengths. The three data sources chosen for the assessment all helped identify titles that the students of the new program would be using. Overall, faculty publishing data were the most predictive of the student citations, but the combination of all three led to the development of a core list of 183 titles potentially important to WRGP students. The authors would recommend this or a similar multi-pronged approach to evaluate the serials collection for a new program -- particularly if it is highly interdisciplinary and no available core list of titles exists. The theses citations, in combination with the 2003 WGRP assessment and current ILL data, helped OSU librarians develop an improved core list for Water Resources to consult in future collection management activities. The WRGP study benefits the library by providing additional data for collection decisions. A thorough collection analysis aids allocation of funds for new programs, but it can only do so much to predict use several years forward. Retrospective review of the level of library support for the program after it is established is a valuable tool for librarians, as well as for the students and faculty who are part of the program. References Bobal, A.M., Mellinger, M., and Avery, B.E. 2008. Collection assessment and new academic programs. Collection Management, 33(4), 288-301. Brazzeal, B. and Fowler, R. 2005. Patterns of information use in graduate research in forestry: a citation analysis of master's theses at Mississippi State University. Science & Technology Libraries, 26(2), 91-106. Dilevko J. and Dali, K. 2004. Improving collection development and reference services for interdisciplinary fields through analysis of citation patterns: an example using tourism studies. College & Research Libraries, 65(3), 216. Edwards, S. 1999. Citation analysis as a collection development tool: a bibliometric study of polymer science theses and dissertations. Serials Review, 25(1), 11-20. Gooden, A.M. 2001. Citation analysis of chemistry doctoral dissertations: an Ohio State University case study. Issues in Science & Technology Librarianship 32 [Internet]. [Cited 2009 Jun 17]. Available from http://www.istl.org/01-fall/refereed.html Haycock, L.A. 2004. Citation analysis of education dissertations for collection development. Library Resources & Technical Services, 48(2), 102-106. Johnson, B. 2000. Environmental impact: a preliminary citation analysis of local faculty in a new academic program in environmental and human health applied to collection development in an academic library. Library Philosophy & Practice, 2(2) [Internet]. [Cited 2009 Jun 16]. Available from http://unllib.unl.edu/LPP/johnson.html Kriz, H.M. 1984. Library management implications of journal citation patterns in engineering doctoral dissertations. ERIC Document Reproduction Service No. ED243481. Kuruppu, P.U. and Moore, D.C. 2008. Information use by PhD students in agriculture and biology: a dissertation citation analysis. portal: Libraries & the Academy, 8(4), 387-405. Kushkowski, J.D., Parsons, K.A., and Wiese, W.H. 2003. Master's and doctoral thesis citations: analysis and trends of a longitudinal study. portal: Libraries & the Academy, 3(3), 459-479. Pancheshnikov, Y. 2007. A comparison of literature citations in faculty publications and student theses as indicators of collection use and a background for collection management at a university library. Journal of Academic Librarianship, 33(6), 674-683. Reitz, J.M. c2004-2007. ODLIS: Online Dictionary for Library and Information Science [Internet]. Santa Barbara: Libraries Unlimited; [updated 2009 Nov 19; cited 2009 Aug 2009]. Available from http://lu.com/odlis/index.cfm. Sherriff, G. 2010. Information use in history research: a citation analysis of master's level theses. portal: Libraries and the Academy, 10(2), 165-183. Sinha, R. and Tucker, C. 2005. New program growth and its impact on collection assessment at the UNLV Libraries. Library Hi Tech, 23(3), 362-371. Sinn, R.N. 2005. A local citation analysis of mathematical and statistical dissertations. Science & Technology Libraries, 25(4), 25-37. Smith, E.T. 2003. Assessing collection usefulness: an investigation of library ownership of the resources graduate students use. College & Research Libraries, 64(5), 344-355. Sylvia, M. and Lesher, M. 1995. What journals do psychology graduate students need? A citation analysis of thesis references. College & Research Libraries, 56(4), 313-318. Thomas, J. 1993. Graduate student use of journals: a bibliometric study of psychology theses. Behavioral & Social Sciences Librarian, 12(1), 1-7. Walcott, R. 1994. Local citation studies - a shortcut to local knowledge. Science & Technology Libraries,14 (3), 1-14. Walker, R.D. and Ahn, M.L. 1994. The literature cited by water resources researchers. In: Haner, B.E. and O'Donnell, J., editors. Changing gateways: the impact of technology on geoscience information exchange. Proceedings of the 29th Meeting of the Geoscience Information Society, 1994 Oct 24-27; Seattle. Alexandria (VA): The Society. p. 67-77. Walker, R.D. and Zuck, G.J. 1976. Water resources literature cited in Wisconsin Department of Natural Resources publications, 1964-1973. In: Retrieval of geoscience information. Proceedings of the 10th annual meeting of the Geoscience Information Society, 1975 October 21; Salt Lake City. Falls Church (VA): The Society. p. 99-122. Water Resources Graduate Program [Internet]. 2008. Corvallis (OR): Oregon State University; [cited 2009 Dec 11]. Available from http://oregonstate.edu/gradwater/ Williams, V.K. and Fletcher, C.L. 2006. Materials used by master's students in engineering. Issues in Science & Technology Librarianship, 45. [Internet]. [Cited 2009 Jun 17]. Available from http://www.istl.org/06-winter/refereed1.html Zipp, L.S. 1996. Thesis and dissertation citations as indicators of faculty research use of university library journal collections. Library Resources & Technical Services, 40(4), 335-42. Previous Contents Next