321 A Review of Citation Analysis Methodologies for Collection Management Kristin Hoffmann and Lise Doucette Kristin Hoffmann is Head, Research & Instructional Services, in the D.B. Weldon Library at the University of Western Ontario; e-mail: khoffma8@uwo.ca. Lise Doucette is Research & Instructional Services Librar- ian in the Allyn & Betty Taylor Library at the University of Western Ontario; e-mail: ldoucet@uwo.ca. © 2012 Kristin Hoffmann and Lise Doucette, Attribution-NonCommercial (http://creativecommons.org/ licenses/by-nc-sa/3.0/) CC BY-NC While there is a considerable body of literature that presents the results of citation analysis studies, most researchers do not provide enough detail in their methodology to reproduce the study, nor do they provide rationale for methodological decisions. In this paper, we review the methodologies used in 34 recent articles that present a “user study” citation analysis with a goal of informing collection management. We describe major themes and outliers in the methodologies and discuss factors that require careful thought and analysis. We also provide a guide to considerations for cita- tion analysis studies, so that researchers can make informed decisions. Introduction and Literature Review Citation analysis is a branch of bibliomet- rics that examines the citations found in publications such as journal articles and books to look for patterns of use. This type of study typically involves record- ing the details of the reference lists of a number of publications to determine what materials are being consulted and then analyzing those materials by type, frequency, age, local holdings, or other factors. It is an unobtrusive methodol- ogy that can often be carried out with readily available data. By conducting a citation analysis, a researcher can better understand scholarly communication trends within a discipline, assess the use or accuracy of citation indexes, create bibliographies, analyze citation trends of specific user groups, or determine the extent to which a library’s collection meets researchers’ needs.1 The combination of the latter two applications is of interest for this paper: user studies that inform collection management. With a “user study” citation analysis, researchers focus on the citation patterns of a specific user group and often also focus on a specific type of publication. Common approaches in the literature are to analyze faculty publications or gradu- ate student theses for a particular depart- ment at a particular institution. This type of analysis can yield valuable insights into local citing trends, such as types and number of documents cited, accuracy of citations, and changes in citation patterns over time. The information can help li- brarians design and improve services and resources, from instruction to collections to reference.2 While citation analysis stud- ies can be compared across institutions, crl-254 322 College & Research Libraries July 2012 local factors can preclude generalization, including variation among institutions in type and size, research focus and disci- plines, library budget, or library consortia and package deals. With a “collection management” cita- tion analysis, the underlying premise is that “the more frequently cited publica- tions are the more valuable, will continue to be used heavily, and, consequently, are more important to have in the library collections.”3 Collection-focused activities that make use of citation data include examining print versus electronic usage to determine the impact of electronic jour- nals, compiling lists of most- and least- cited journals and local holdings to make acquisition or cancellation decisions, and examining age of cited references to help develop storage and retention policies.4 There is a gap in the literature in that there is no consolidated set of guidelines or considerations on the methodology for citation analysis studies. While there is a considerable body of journal literature that presents the results of such studies, most researchers do not provide enough detail in their methodology to reproduce the study, and explanations are generally not provided for decisions related to their study’s objectives, scope, and citation re- trieval and analysis. Monographs provide information on citation analysis method- ologies5—but often at a broad level. All of this published material is useful for providing background information or context for librarians who are plan- ning to conduct a citation analysis, but it does not provide guidance or a thorough description of the components of such a study, particularly for researchers new to this methodology. Additionally, without adopting some consistent methodological practices, it can be difficult and inaccurate to make comparisons among citation analysis studies. In this paper, we analyze the method- ologies used in 34 citation analysis stud- ies published between 2005 and 2010, all of which are user studies conducted to inform collection management practices and policies. We describe major themes and outliers in the methodologies used in the studies and discuss factors that re- quire careful thought and analysis when conducting this type of study. We also provide a guide to considerations for the many components of a citation analysis study so that researchers can make in- formed decisions in their own studies. Methods We searched the literature for citation analysis studies that examined the cita- tion behaviour of a specific, local user group to assess the adequacy of the cur- rent library collection or to inform collec- tions practices and policies. We conducted the literature search in two library science databases (Library Literature and Library, Information Science & Technology Ab- stract: LISTA) and one interdisciplinary database (Scopus) to capture a full range of published literature. The concepts used in the search strategy were “citation analysis” and collection.* The search was limited to articles published from 2005 through 2010. While the desired type of study is most succinctly described as “user study,” terms such as “user study,” “patron,” or “user group” did not retrieve relevant re- sults. We therefore used a relatively broad search strategy and manually reviewed the search results. The publication date range of 2005–2010 was chosen for two reasons. First, it pro- vided a manageable sample size that would still be large enough to draw out themes and conclusions among the methodologies. Second, it is a time period during which the tools available for citation analysis studies, such as citation databases, generally re- mained the same. This allowed us to focus on methodological approaches rather than tools used. We were able to achieve satura- tion of themes with this sample size. The search was conducted in the fall of 2010 and repeated in January 2011 to capture articles published in late 2010. Applying the above search strategy and limits, we found 194 unique studies A Review of Citation Analysis Methodologies for Collection Management 323 in the three databases after removing duplicates. Non-English articles were also removed. We reviewed titles, subjects, and abstracts to determine that 36 of these articles appeared to match the desired type of citation analysis study. Further analysis and reading the text of the ar- ticles led to the exclusion of two articles. We therefore analyzed 34 studies; a list of these articles can be found in Appendix A: Citation Analysis Studies Reviewed and Variables Analyzed. In our analysis of these studies, we looked for descriptions of methodology to determine how the researchers defined the scope of their study (user group, date range, and types of publications), as well as how they retrieved, examined, and then analyzed the citations. A strength of our approach is the focus on practical, pub- lished examples of the implementation of a citation analysis. Our aim is to build on the successes of these studies by consolidating and analyzing recent practices in citation analysis. However, our review of these methodologies is necessarily limited by the extent to which researchers discussed their methodology in their studies. Our analysis is qualitative in nature, as our purpose is to synthesize the meth- ods used in citation analysis studies. In the results, we describe major themes found within the methodologies of the 34 citation analysis studies. Themes were developed through the careful review and manual categorization of all written material related to methodology within each study. Further detail on generation of themes is given within each major Results section below. Results All studies we reviewed looked at the citation behaviour of a specific user group to do at least one of the following: assess the adequacy of the current library collection; inform collections practices, such as acquisitions and cancellations; or inform collections policies, such as funding distribution, retention policies, and electronic access. We first present a summary of the studies reviewed, looking at the subject area, library setting, user group, type of publication, and geographic location. We then describe in detail three methodologi- cal themes that emerged in our analysis that were common among all 34 studies: defining the scope of the study, retrieving and refining publications and citations, and analyzing the citations. The amount of detail, data, and reasoning provided by researchers in each of these three themes varied widely. Summary of Studies Reviewed The citation analysis studies covered a range of subject areas, and most fell into the categories of science, medicine or health, social science, or engineering. All but one of the studies were conducted in an academic library setting (the out- lier was a government library), and they covered the range of user groups that would be expected in academic libraries: faculty, PhD students, master’s students, and undergraduate students. The types of publications examined were also consis- tent with those expected in an academic library setting: journal articles, theses or dissertations, and undergraduate student papers. Some studies looked at more than one subject area, more than one category of user group, or more than one type of publication. The majority of the studies were conducted in the United States, and the others were conducted in coun- tries spanning five continents: Australia, Belgium, Canada, China, Finland, Iran, Nigeria, Spain, and Turkey. Defining the Scope of the Study The purpose and objectives of the citation analysis studies often provided a broad description of the scope, but researchers also gave detail about the group of users and type and date range of publications. Defining the Group of Users. Selecting the user group for the citation analysis requires decisions about the category of user (such as undergraduate student, 324 College & Research Libraries July 2012 graduate student, faculty) and the subject area of interest (for example: dentistry, forestry, educational psychology). A small number of researchers provided reason- ing for their choice of user group, ranging from capturing interdisciplinary research by performing a large-scale analysis6 to selecting top-ranked research areas7 to selecting subject areas that were under- studied in citation analyses.8 Defining the Types of Publications. The deci- sion of publication type to analyze was tied to the category of user. For example, researchers who investigated doctoral students analyzed their dissertations; researchers who investigated faculty ana- lyzed their journal articles. Some of the researchers elaborated on their choice of publications, particularly when the study looked at “any publication” by the user group. Two articles specifically excluded types of publications such as books, or materials that do not normally contain references.9 When significant, researchers noted the inclusion (official publications) or exclusion (all nonrefereed articles) of certain types of publications.10 Defining the Date Range of Publications. Re- searchers specified a publication date range of one to 30 years over which they analyzed their user groups’ publications. Some used publications from “recent years,” with “re- cent” meaning anywhere from one to five years. Other researchers chose longer time periods to track changes over time.11 Lon- ger time periods were also used to capture the transition to online databases12 or the effect of a subscription to a new database.13 Two studies took a different approach and used the most recent publications by each member of their user group rather than publications over a given time period.14 Retrieving and Refining Publications and Citations For studies that examined journal articles, the most common method of gathering citations was to use the Web of Science citation indexes to conduct an affiliation search or an author search. In other stud- ies, databases were chosen that were more specific to the subject or the user group, such as PubMed or the VIP Database of Chinese Science and Technology Periodi- cals.15 Some researchers used internal in- stitutional databases created by research, administrative, or library units.16 For studies that examined student theses or dissertations, authors gathered citations from print copies in the library or from electronic copies retrieved from either an internal database or ProQuest Digital Dissertations. Much of the data- gathering process involved converting the citations into a usable electronic format. In two cases, the researchers automated the process by creating a Perl script to pull cita- tions from a searchable theses database.17 Most researchers chose to use all of the publications they retrieved in their citation analysis study. Some limited the amount of data they analyzed, using the strategies of saturation18 and sampling.19 Other issues that researchers faced were whether to include duplicates if publications had more than one author who was part of the study’s user group;20 how to consider author order in selecting papers;21 and whether to include publica- tions that fell outside the scope of the topic identified for the study.22 Analyzing the Citations Most researchers used spreadsheet or database software to analyze the cita- tion data gathered; programs mentioned most frequently were Microsoft Access, Microsoft Excel, and the Statistical Pack- age for the Social Sciences (SPSS). Several resources were used to verify or standard- ize citations, including WorldCat, Ulrich’s Serials Directory, CASSI, ISI’s Journal Citation Reports, PubMed, local catalogue records, publisher Web sites, and various article databases. Most researchers did not comment on who compiled and ana- lyzed the citations, but some mentioned assistance from students or interns.23 The most significant component of ana- lyzing citations is related to the variables A Review of Citation Analysis Methodologies for Collection Management 325 that each researcher chose to analyze. All variables that were analyzed by more than one researcher are presented in Ap- pendix A. We will focus on the four most frequently discussed variables: types of resource cited, age of cited resources, frequency of citation to journal titles, and library holdings. We provide specific examples related to these four variables below and refer readers to the list of stud- ies and associated variables in Appendix A for all examples of how researchers analyzed these variables. Types of Resources Cited. Almost all re- searchers noted the types of resource cited, with a great range in the number of categories used, from three24 to twenty-six format types.25 Researchers analyzed dif- ferent types of resources as appropriate to the topic of the study; for example, patents were a type used in some stud- ies of engineering user groups.26 Some researchers adopted categories that had been used in previously published stud- ies27 or that matched those already in use by their own library28 or by WorldCat.29 Age of Cited Resources. Most researchers analyzed the publication date of cited resources. Several also calculated the citation age, defined as the difference be- tween the date of the citation and the date of the publication in which it was cited. Frequency of Citation to Journal Titles. Most researchers compiled a list of journals ranked by citation frequency to show the top journals cited by their user group. One issue that researchers faced was how to count multiple citations to the same article from one individual (either in one publication or in multiple publications by one author). Two strategies were to count each occurrence of a citation toward the citation frequency,30 or to calculate a weighted rank, which takes into account the number of authors who cited a given citation or journal.31 Title changes or splits were addressed in various ways. One approach was to list title changes as separate journals;32 another strategy was to treat title changes as one journal and title splits as new and different journals.33 The majority of researchers identified the amount of scattering or dispersion of the citations. In doing so, some researchers tested variations of the “80–20 rule”: that is, the principle that 80 percent of citations would come from 20 percent of journals.34 Other researchers chose specific values to check: determining, for example, the per- centage of citations to the top 10 journals35 or the number of journals that satisfied 50 percent of the citations.36 Many authors also investigated whether their user group’s citation patterns fit with Bradford’s Law of Scattering, in which journals cited are listed in order of decreasing citation frequency and then grouped into three zones, with equal numbers of citations in each zone. The law states that a progressively larger number of journals will be needed to match the zones containing equal numbers of cita- tions, in the ratio 1:n:n2.37 Library Holdings. A majority of researchers were interested in knowing whether their library provided access to the materials cited by their user group. Holdings of journal titles were of greatest interest, and some researchers verified access to the specific journal issue in which the citation was found. The type of access was also of interest, and some researchers noted whether the subscription was electronic, print, or a combination,38 or whether the journal was an open access publication.39 Most researchers verified each cita- tion recorded for their study. Some were only interested in holdings for a subset of journals, such as the top 10 or 20 cited journals,40 any journal that had received a certain number of citations,41 or a group of journals as defined by the scattering or dispersion of citations (such as using the 80–20 rule or Bradford zones).42 Other Variables Analyzed. The other, less commonly analyzed variables in the studies include number of citations per 326 College & Research Libraries July 2012 publication, subject area of cited journals, language of citations, impact factor of cited journals, publishers of cited jour- nals, and cost per citation. Studies that analyzed these variables are identified in Appendix A. Discussion All of the studies we reviewed described aspects of their methodology, but some provided considerably more detail than others. Some studies we read could be easily reproduced by another researcher; others gave such broad descriptions that their methodology was unclear. While a wide range of variables can be analyzed in a citation analysis, researchers were not always clear about how the variables they chose to analyze contributed to the objectives of their study. Explanations and rationale for methodological choices were not consistently provided. All of these gaps in methodological descrip- tion make it difficult for readers to be able to compare and contrast studies and to think about how to conduct their own study. Citation analysis is a relatively straight- forward tool for librarians to use to bet- ter understand their users or analyze their collections, but its ease of use can perhaps lead researchers to believe that careful reflection on the methodology is not required. As well, it is easy to conduct extensive analysis with the retrieved citations, but not all analysis contributes to a study’s objectives. To ensure clarity and comparability, researchers must be deliberate about the choices that define the scope of their study, and they must also be selective about what is analyzed and presented. In the citation analysis studies we reviewed, many different variables were analyzed. For most variables, we conclude from our analysis that there is not one “right” way to approach the study (for ex- ample, choosing the types of publications to analyze). Rather, researchers should think carefully about what choices will best meet the objectives of their study. We provide a guide to these considerations in Appendix B: Guide to Considerations for Citation Analysis Methodologies and discuss four of the more significant con- siderations in more detail below: selecting citations through sampling or saturation, analyzing age of cited resources, analyz- ing frequency of citation to journal titles, and conducting complementary studies. Sampling and Saturation Rather than collecting, verifying, and ana- lyzing every cited reference, researchers should consider strategies for working with a more manageable number. Using a sample of citations is one approach; a proportional sample should be used, and citations randomly selected, to ensure that the sample is representative. Sampling is most effectively used when the total number of publications is large enough that a subset of those publications will still be representative. Saturation is another technique that can facilitate analysis; in this approach, researchers analyze cita- tions from increasing numbers of publica- tions until it is determined that analysis of additional citations does not significantly change or add to the results. Citation Age When analyzing the age of cited re- sources, researchers should calculate the citation age rather than simply use the publication date. This is a simple practice to adopt and will facilitate comparison among the results of various citation analysis studies, whether among institu- tions or at one institution over time. Frequency of Citation to Journal Titles In the studies we reviewed, there was little consensus in how to determine and present frequency of citation to journals. This can be complicated by the many options available for using a defined law of scatter or calculating a number or percentage of frequently cited titles. Researchers should therefore consider whether there are published citation analysis studies that can be used for A Review of Citation Analysis Methodologies for Collection Management 327 comparison and use an approach that is consistent with those studies. If the cita- tion analysis results are intended to be used for local understanding only, rather than to be compared with other institu- tions’ results, researchers should take the approach that fits best with local practice and carefully describe their decisions. Conducting Complementary Studies Finally, citation analysis studies have limi- tations to their applicability, as several re- searchers have noted.43 These limitations can be minimized by conducting a study with a complementary method, such as interviews with faculty about their read- ing habits, to obtain a fuller picture of the citation analysis study’s findings.44 Conclusions In reviewing the methodologies of cita- tion analysis studies, we found that the amount of detail provided by research- ers varied widely. However, common themes among the methodologies can provide guidance for researchers who are planning to conduct citation analysis studies, in areas that range from defining the scope of the study to retrieving and analyzing citations. In each of these areas, there are many factors to consider, and researchers should think carefully about the choices that will best meet the objectives of their study. If there are published studies that will be used for comparison, researchers should consider using an approach that is consistent with those studies. If the citation analysis results are intended to be used primarily to enhance local un- derstanding, researchers should choose an approach that fits with local practices. In all cases, researchers must be deliber- ate about methodological choices, be selective about what is analyzed and presented, and clearly describe their decisions and rationale. These practices will help ensure clarity and comparabil- ity, thereby increasing the value of each individual citation analysis study. 328 College & Research Libraries July 2012 Citation Analysis Study Variables Analyzed in Study Brazzeal, Bradley, and Robert Fowler. “Patterns of Infor- mation Use in Graduate Research in Forestry: A Citation Analysis of Master’s Theses at Mississippi State Univer- sity.” Science & Technology Libraries 26 (2005): 91–106. type,a citation age,b frequency,c holdings,d cita- tions per publication, subject areae Burright, Marian A., Trudi B. Hahn, and Margaret J. Antonisse. “Understanding Information Use in a Multi- disciplinary Field: A Local Citation Analysis of Neurosci- ence Research.” College & Research Libraries 66 (2005): 198–210. type, citation age, frequency, subject area Choinski, Elizabeth M. “Journal Use in Pharmacy.” Sci- ence & Technology Libraries 27, no. 3 (2007): 53–64. type, citation age, frequency, holdings, subject area, publishers Cox, Janice E. “Citation Analysis of Graduate Dental The- ses References: Implications for Collection Development.” Collection Management 33 (2008): 219–34. type, citation age, frequency, holdings De Groote, Sandra L., and Felicia A. Barrett. “Impact of Online Journals on Citation Patterns of Dentistry, Nursing, and Pharmacy Faculty.” Journal of the Medical Library Association 98 (2010): 305–08. holdings deVries, Susann, Robert Kelly, and Paula M. Storm. “Mov- ing Beyond Citation Analysis: How Surveys and Interviews Enhance, Enrich, and Expand Your Research Findings.” College & Research Libraries 71 (2010): 456–66. type, holdings, impact factor Feyereisen, Pierre, and Anne Spoiden. “Can Local Citation Analysis of Master’s and Doctoral Theses Help Decision- Making about the Management of the Collection of Periodicals? A Case Study in Psychology and Education Sciences.” The Journal of Academic Librarianship 35 (2009): 514–22. type, frequency, holdings, language, impact factor, cost per citationf Fuchs, Beth E., Cristina M. Thomsen, Randolph G. Bias, and Donald G. Davis. “Behavioral Citation Analysis: Toward Collection Enhancement for Users.” College & Research Libraries 67 (2006): 304–24. type, citation age, frequency, holdings, publishers Gao, Shi-Jian, Wang-Zhi Yu, and Berenika M. Webster. “A Longitudinal Investigation into the Changing Citing Behavior of Geomatics Postgraduate Students at Wuhan University, China, 1988–2004: Implications for Collec- tion Development.” Library Collections, Acquisitions, & Technical Services 31 (2007): 42–57. type, citation age, frequency, holdings, subject area Gao, Shi-Jian, and Wang-Zhi Yu. “A Local Citation Analy- sis in China: From Wuhan University Faculty in Surveying and Mapping.” The Journal of Academic Librarianship 31 (2005): 449–55. type, citation age, frequency, holdings, citations per publi- cation, language Appendix A: Citation Analysis Studies Reviewed and Variables Analyzed A Review of Citation Analysis Methodologies for Collection Management 329 Citation Analysis Study Variables Analyzed in Study Gao, Shi-Jian, Wang-Zhi Yu, and Feng-Ping Luo. “Cita- tion Analysis of PhD Thesis at Wuhan University, China.” Library Collections, Acquisitions, and Technical Services 33 (2009): 8–16. type, citation age, frequency, subject area, language Iivonen, Mirja, Ulla Nygrén, Anu Valtari, and Tanja Heikkilä. “Library Collections Contribute to Doctoral Studies: Citation Analysis of Dissertations in the Field of Economics and Ad- ministration.” Library Management 30 (2009): 185–203. citation year, holdings, cita- tions per publication Kayongo, Jessica, and Clarence Helm. “Citation Patterns of the Faculty of the Anthropology Department at the University of Notre Dame.” Behavioral & Social Sciences Librarian 28 (2009): 87–99. citation year, frequency, holdings Kirkwood, Patricia. “Using Engineering Theses and Dis- sertations to Inform Collection Development Decisions Especially in Civil Engineering.” American Society for Engineering Education Conference Proceedings (2009). Available online at www.asee.org/search/proceedings [ac- cessed 21 November 2010]. type, holdings Knight-Davis, Stacey, and Jan S. Sung. “Analysis of Citations in Undergraduate Papers.” College & Research Libraries 69 (2008): 447–58. type, citation year, holdings, citations per publication Kraus, Joseph R. “Comparing Journal Use between Biolo- gy Faculty and Undergraduate Students.” Issues in Science and Technology Librarianship No. 45 (Summer 2005). Available online at www.istl.org/05-summer/article2.html [accessed 21 November 2010]. type, frequency LaBonte, Kristen B. “Citation Analysis: A Method for Collection Development for a Rapidly Developing Field.” Issues in Science and Technology Librarianship No. 45 (Summer 2005). Available online at www.istl.org/05-sum- mer/refereed.html [accessed 21 November 2010]. frequency, holdings Leiding, Reba. “Using Citation Checking of Undergraduate Honors Thesis Bibliographies to Evaluate Library Collec- tions.” College & Research Libraries 66 (2005): 417–29. type, frequency, holdings, citations per publication Nabe, Jonathan, and Andrea Imre. “Dissertation Citations in Organismal Biology at Southern Illinois University at Carbondale: Implications for Collection Development.” Issues in Science & Technology Librarianship No. 55 (Fall 2008). Available online at www.istl.org/08-fall/refereed. html [accessed 21 November 2010]. type, citation year, citations per publication Olatokun, Wole M., and Olayinka Makinde. “Citation Analysis of Doctoral Works Submitted to the Department of Animal Science, University of Ibadan, Nigeria.” Library Philosophy and Practice (November 2009). Available online at www.webpages.uidaho.edu/~mbolin/olatokun- makinde.pdf [accessed 21 November 2010]. type, citation age, frequency, citations per publication 330 College & Research Libraries July 2012 Citation Analysis Study Variables Analyzed in Study Omoba, Florence A., and Beatrice A. Fabunmi. “Evalua- tion of References in Dissertations and Theses Against the Holdings in a University Library.” Library Philosophy and Practice (February 2010). Available online at www.web- pages.uidaho.edu/~mbolin/omoba-fabunmi.pdf [accessed 21 November 2010]. type, citation year, holdings Ortega, Lina. “Age of References in Chemistry Articles: A Study of Local Authors’ Publications from Selected Years, 1975–2005.” Science & Technology Libraries 28 (2008): 209–46. type, citation age, citations per publication Pancheshnikov, Yelena. “A Comparison of Literature Citations in Faculty Publications and Student Theses as Indicators of Collection Use and a Background for Collec- tion Management at a University Library.” The Journal of Academic Librarianship 33 (2007): 674–83. type, citation age, frequency, subject area Price, Jason. “How Many Journals Do We Have? An Alter- native Approach to Journal Collection Evaluation Through Local Cited Article Analysis.” Serials 20 (2007): 134–41. frequency, holdings Ralston, Rick, Carole Gall, and Frances A. Brahmi. “Do Local Citation Patterns Support Use of the Impact Factor for Collection Development?” Journal of the Medical Library Association 96 (2008): 374–78. frequency, holdings, impact factor Rashidi, Ali, Bob Gilchrist, and Farhi Marir. “An Investi- gation of International Journal Usage by Iranian Medical Researchers.” Libres 18, no. 2 (2008). Available online at http://libres.curtin.edu.au/libres18n2/Rashidi_FINAL.pdf [accessed 21 November 2010]. type, citation age, frequency Rethlefsen, Melissa L. “Citation Analysis of Minnesota Department of Health Official Publications and Journal Articles: A Needs Assessment for the RN Barr Library.” Journal of the Medical Library Association 95 (2007): 260–66. type, citation age, frequency, holdings Salisbury, Lutishoor, and Jeremy S. Smith. “The Use of Web of Knowledge to Study Publishing and Citation Use for Local Researchers at the Campus Level.” Collection Management 35 (2010): 69–82. type, frequency, holdings Sherriff, Graham. “Information Use in History Research: A Citation Analysis of Master’s Level Theses.” portal: Libraries & the Academy 10 (2010): 165–83. type, citation age, frequency, holdings, citations per publi- cation, subject area, language Sinn, Robin N. “A Local Citation Analysis of Mathemati- cal and Statistical Dissertations.” Science & Technology Libraries 25 (2005): 25–37. type, citation age, frequency, citations per publication Tonta, Yaşar, and Umut Al. “Scatter and Obsolescence of Journals Cited in Theses and Dissertations of Librari- anship.” Library and Information Science Research 28 (2006): 281–96. type, citation year, citation age, frequency, holdings, citations per publication, language, impact factor A Review of Citation Analysis Methodologies for Collection Management 331 Citation Analysis Study Variables Analyzed in Study Vallmitjana, Núria, and L.G. Sabaté. “Citation Analysis of Ph.D. Dissertation References as a Tool for Collection Management in an Academic Chemistry Library.” College & Research Libraries 69 (2008): 72–81. type, citation age, frequency, subject area, impact factor, publishers, cost per citation Williams, Virginia K., and Christine L. Fletcher. “Materials Used by Master’s Students in Engineering and Implications for Collection Development: A Citation Analysis.” Issues in Science & Technology Librarianship No. 45 (Winter 2006). Available online at www.istl.org/06-winter/refer- eed1.html [accessed 21 November 2010]. type, citation age, frequency, holdings, citations per publi- cation, subject area Wilson, Concepción S., and Carol Tenopir. “Local Citation Analysis, Publishing and Reading Patterns: Using Multiple Methods to Evaluate Faculty Use of an Academic Library’s Research Collection.” Journal of the American Society for Information Science and Technology 59 (2008): 1393–408. type, citation year, frequency, holdings, impact factor a type of material (for example: books, journals) b difference between the date of the citation and date of the publication in which it is cited c list of journals ranked by citation frequency or scattering/dispersion of the citations d local library’s access to cited publication (such as journal) or specifically to cited item (such as journal issue) e any scheme of categorizing citations by subject area (for instance, Library of Congress call numbers or subject headings) f cost per citation: ratio of subscription cost of journal to the number of citations to that journal 332 College & Research Libraries July 2012 General Guidelines Purpose and objectives • Clearly describe the study’s purpose and objectives; if applicable, —Explain your relationship to the user group being studied (for instance, liaison librarian for chemistry) to help the reader understand the context of the study —Explain any particular motivations for performing the study • Identify variables to analyze that will meet the purpose and objectives of your study Resources • Consider soliciting assistance from students or interns for gathering and analyzing data • Identify the software needed for analysis and ensure you have access to it Defining the Scope of the Study Overall considerations • Be deliberate in all of your choices and provide a rationale for all aspects of scope • Determine whether you want to gather data for all publications that meet your scope requirements or a sample of those publications Define group of users • Clearly define the people and the subject area(s) you are studying • Clearly define terms that could be interpreted in many ways, such as “researchers” and “publications” • Consider whether you want to analyze publications from a defined group of researchers or publications about a topic (examples: the De- partment of Chemistry faculty publications, or nanotechnology articles published by your institution’s researchers) • Consider whether the order of authors in a publication is important (for instance, must the local author be first, or could he or she be the 4th of 6 authors?) Define types of publications • Consider the range of publication types that could be studied and focus on those that best meet your objectives • Clearly state which types of publications are included and excluded Define time range • Choose a date range that meets your objectives —Clearly describe whether you’re interested in recent or long- term trends —Clearly describe whether you’re looking at publications from all years in the date range or a sampling of years within the range • Consider dates that correspond to major changes at your institution, such as moving to e-journals or beginning a subscription to a major database Retrieving and Refining Publications and Citations Gather data • When selecting sources of citation data, consider ease of retrieval and data formats that can be easily brought into a spreadsheet or database program • If analyzing theses, consider availability in print or electronic format from an internal or external database • If analyzing other publications, consider electronic availability from citation databases, subject-specific databases, or internal databases Appendix B: Guide to Considerations for Citation Analysis Methodologies A Review of Citation Analysis Methodologies for Collection Management 333 Analyzing the Citations Overall considerations and variables to analyze • Be specific in all of your choices; ensure that the variables you ana- lyze have a direct link back to the purpose and objectives of your study • Consider whether to analyze all citations or a subset; if working with a subset, —Consider using a representative sample with randomly selected citations —Consider analyzing citations until saturation is achieved (for instance, no new information is added by analyzing citations from additional publications) Type of resource cited • Use categories that align with the study’s objectives • Consider using categories that are either locally relevant or consistent with those used in other publications Age of cited resource • Consider calculating the citation age (difference between the date of the citation and the date of the publication in which it was cited) rather than simply analyzing publication date Frequency of citation to journals • Clearly describe how you will treat journal title changes and splits (as one title or as separate titles) • Consider whether to use a weighted rank of citation frequencies based on how many different users in your group have cited an item • Consider using a defined law of scatter such as Bradford’s Law or the “80–20 rule” to determine a number or percentage of frequently cited titles Library holdings • Make decisions based on what will be locally useful, as analysis of a library’s holdings of the cited journal titles is done primarily to improve local understanding of the library’s collection rather than for compari- son with other published studies • Consider how comprehensive to be in analyzing library holdings (for example, whether to check every journal or only those cited at least a certain number of times) • Consider the importance of identifying whether a print or electronic item was cited, keeping in mind that in many cases the citation may not provide format information • Consider the importance of determining whether the library had a sub- scription to a particular title at the time that it was cited (and whether that information would be available), versus assessing current holdings of a title Notes 1. Allen B. Ashman, “An Examination of the Research Objectives of Recent Citation Analysis Studies,” Collection Management 34 (2009): 112–28; Linda C. Smith, “Citation Analysis,” Library Trends 30 (1981): 83–106. 2. Ashman, “An Examination of Research Objectives,” 123–25; Smith, “Citation Analysis,” 95. 3. Peggy Johnson, Fundamentals of Collection Development and Management, 2nd ed. (Chicago: American Library Association, 2009), 247. 4. Ashman, “An Examination of Research Objectives,” 118–19; Smith, “Citation Analysis,” 97–98. 5. For examples, see Johnson, Fundamentals of Collection Development and Management, 247–48, and Joseph R. Matthews, The Evaluation and Measurement of Library Services (Westport, Conn.: Libraries Unlimited, 2007), 121–25. 6. Lutishoor Salisbury and Jeremy S. Smith, “The Use of Web of Knowledge to Study Publish- ing and Citation Use for Local Researchers at the Campus Level,” Collection Management 35 (2010): 69–82. 334 College & Research Libraries July 2012 7. Shi-Jian Gao, Wang-Zhi Yu, and Feng-Ping Luo, “Citation Analysis of PhD Thesis at Wuhan University, China,” Library Collections, Acquisitions, and Technical Services 33 (2009): 8–16. 8. Jonathan Nabe and Andrea Imre, “Dissertation Citations in Organismal Biology at Southern Illinois University at Carbondale: Implications for Collection Development,” Issues in Science & Technology Librarianship, no. 55 (Fall 2008), available online at www.istl.org/08-fall/refereed.html [accessed 21 November 2010]. 9. Elizabeth M. Choinski, “Journal Use in Pharmacy,” Science & Technology Libraries 27 (2007): 53–64; Jessica Kayongo and Clarence Helm, “Citation Patterns of the Faculty of the Anthropology Department at the University of Notre Dame,” Behavioral & Social Sciences Librarian 28 (2009): 87–99. 10. Melissa L. Rethlefsen, “Citation Analysis of Minnesota Department of Health Official Pub- lications and Journal Articles: A Needs Assessment for the RN Barr Library,” Journal of the Medical Library Association 95 (2007): 260–66; Concepción S. Wilson and Carol Tenopir, “Local Citation Analysis, Publishing and Reading Patterns: Using Multiple Methods to Evaluate Faculty Use of an Academic Library’s Research Collection,” Journal of the American Society for Information Science and Technology 59 (2008): 1393–408. 11. Sandra L. De Groote and Felicia A. Barrett, “Impact of Online Journals on Citation Patterns of Dentistry, Nursing, and Pharmacy Faculty,” Journal of the Medical Library Association 98 (2010): 305–08; Shi-Jian Gao, Wang-Zhi Yu, and Berenika M. Webster, “A Longitudinal Investigation into the Changing Citing Behavior of Geomatics Postgraduate Students at Wuhan University, China, 1988–2004: Implications for Collection Development,” Library Collections, Acquisitions, & Technical Services 31 (2007): 42–57; Reba Leiding, “Using Citation Checking of Undergraduate Honors Thesis Bibliographies to Evaluate Library Collections,” College & Research Libraries 66 (2005): 417–29. 12. Beth E. Fuchs, Cristina M. Thomsen, Randolph G. Bias, and Donald G. Davis, “Behavioral Citation Analysis: Toward Collection Enhancement for Users,” College & Research Libraries 67 (2006): 304–24; Robin N. Sinn, “A Local Citation Analysis of Mathematical and Statistical Dissertations,” Science & Technology Libraries 25 (2005): 25–37. 13. Lina Ortega, “Age of References in Chemistry Articles: A Study of Local Authors’ Publica- tions from Selected Years, 1975–2005,” Science & Technology Libraries 28 (2008): 209–46. 14. Kayongo and Helm, “Citation Patterns,” 89; Jason Price, “How Many Journals Do We Have? An Alternative Approach to Journal Collection Evaluation through Local Cited Article Analysis,” Serials 20 (2007): 134–41. 15. Rethlefsen, “Citation Analysis,” 261; Shi-Jian Gao and Wang-Zhi Yu, “A Local Citation Analysis in China: From Wuhan University Faculty in Surveying and Mapping,” The Journal of Academic Librarianship 31 (2005): 449–55. 16. Choinski, “Journal Use in Pharmacy,” 55; Kayongo and Helm, “Citation Patterns,” 90; Ali Rashidi, Bob Gilchrist, and Farhi Marir, “An Investigation of International Journal Usage by Iranian Medical Researchers,” Libres 18, no. 2 (2008), available online at http://libres.curtin.edu. au/libres18n2/Rashidi_FINAL.pdf [accessed 21 November 2010]; Wilson and Tenopir, “Local Citation Analysis,” 1394. 17. Susann deVries, Robert Kelly, and Paula M. Storm, “Moving beyond Citation Analysis: How Surveys and Interviews Enhance, Enrich, and Expand Your Research Findings,” College & Research Libraries 71 (2010): 456–66; Nabe and Imre, “Dissertation Citations in Organismal Biol- ogy,” under “Methods.” 18. Mirja Iivonen, Ulla Nygrén, Anu Valtari, and Tanja Heikkilä, “Library Collections Contrib- ute to Doctoral Studies: Citation Analysis of Dissertations in the Field of Economics and Admin- istration,” Library Management 30 (2009): 185–203; Patricia Kirkwood, “Using Engineering Theses and Dissertations to Inform Collection Development Decisions Especially in Civil Engineering,” American Society for Engineering Education Conference Proceedings (2009), available online at www. asee.org/search/proceedings [accessed 21 November 2010]. 19. De Groote and Barrett, “Impact of Online Journals,” 306; Gao, Yu, and Luo, “Citation Analysis of PhD Thesis,” 10; Leiding, “Using Citation Checking,” 420; Florence A. Omoba and Beatrice A. Fabunmi, “Evaluation of References in Dissertations and Theses against the Hold- ings in a University Library,” Library Philosophy and Practice (Feb. 2010), available online at www. webpages.uidaho.edu/~mbolin/omoba-fabunmi.pdf [accessed 21 November 2010]; Wilson and Tenopir, “Local Citation Analysis,” 1394. 20. Kristen B. LaBonte, “Citation Analysis: A Method for Collection Development for a Rapidly Developing Field,” Issues in Science and Technology Librarianship, no. 43 (Summer 2005), available online at www.istl.org/05-summer/refereed.html [accessed 21 November 2010]. 21. Rethlefsen, “Citation Analysis,” 261. 22. Marian A. Burright, Trudi B. Hahn, and Margaret J. Antonisse, “Understanding Information Use in a Multidisciplinary Field: A Local Citation Analysis of Neuroscience Research,” College & Research Libraries 66 (2005): 198–210. 23. Pierre Feyereisen and Anne Spoiden, “Can Local Citation Analysis of Master’s and Doctoral A Review of Citation Analysis Methodologies for Collection Management 335 Theses Help Decision-Making about the Management of the Collection of Periodicals? A Case Study in Psychology and Education Sciences,” The Journal of Academic Librarianship 35 (2009): 514–22; Kayongo and Helm, “Citation Patterns,” 89; Kirkwood, “Using Engineering Theses and Dissertations,” under “Methods”; Rick Ralston, Carole Gall, and Frances A. Brahmi, “Do Local Citation Patterns Support Use of the Impact Factor for Collection Development?” Journal of the Medical Library Association 96 (2008): 374–78. 24. Wilson and Tenopir, “Local Citation Analysis,” 1397. 25. Graham Sherriff, “Information Use in History Research: A Citation Analysis of Master’s Level Theses,” Portal: Libraries & the Academy 10 (2010): 165–83. 26. Virginia K. Williams and Christine L. Fletcher, “Materials Used by Master’s Students in Engineering and Implications for Collection Development: A Citation Analysis,” Issues in Sci- ence & Technology Librarianship, no. 45 (Winter 2006), available online at www.istl.org/06-winter/ refereed1.html [accessed 21 November 2010]. 27. Rethlefsen, “Citation Analysis,” 261; Yaşar Tonta and Umut Al, “Scatter and Obsolescence of Journals Cited in Theses and Dissertations of Librarianship,” Library and Information Science Research 28 (2006): 281–96. 28. Wilson and Tenopir, “Local Citation Analysis,” 1395. 29. Bradley Brazzeal and Robert Fowler, “Patterns of Information Use in Graduate Research in Forestry: A Citation Analysis of Master’s Theses at Mississippi State University,” Science & Technology Libraries 26 (2005): 91–106. 30. Burright, Hahn and Antonisse, “Understanding Information Use,” 202; Janice E. Cox, “Citation Analysis of Graduate Dental Theses References: Implications for Collection Develop- ment,” Collection Management 33 (2008): 219–34. 31. Kayongo and Helm, “Citation Patterns,” 92; Williams and Fletcher, “Materials Used by Master’s Students,” under “Journals.” 32. Brazzeal and Fowler, “Patterns of Information Use,” 98; Cox, “Citation Analysis of Gradu- ate Dental Theses,” 226; Sinn, “A Local Citation Analysis,” 30. 33. LaBonte, “Citation Analysis,” under “Methods.” 34. Fuchs et al., “Behavioral Citation Analysis,” 309; Sinn, “A Local Citation Analysis,” 33; Tonta and Al, “Scatter and Obsolescence,” 288. 35. Rethlefsen, “Citation Analysis,” 262. 36. Núria Vallmitjana and L.G. Sabaté, “Citation Analysis of Ph.D. Dissertation References as a Tool for Collection Management in an Academic Chemistry Library,” College & Research Libraries 69 (2008): 72–81. 37. Simon C. Bradford, “Sources of Information on Specific Subjects,” Engineering: An Illustrated Weekly Journal 137 (1934): 85–86; Thomas E. Nisonger, “Journals in the Core Collection: Definition, Identification, and Applications,” The Serials Librarian 51 (2007): 51–73. 38. For example, see Wilson and Tenopir, “Local Citation Analysis,” 1395. 39. Iivonen et al., “Library Collections Contribute to Doctoral Studies,” 189; Price, “How Many Journals Do We Have?” 136. 40. Brazzeal and Fowler, “Patterns of Information Use,” 99; Gao, Yu, and Luo, “Citation Analysis of PhD Thesis,” 12. 41. Gao and Yu, “A Local Citation Analysis in China,” 451. 42. LaBonte, “Citation Analysis,” under “Cited Journals.” 43. For example, see Smith, “Citation Analysis,” 86–93. 44. deVries, Kelly, and Storm, “Moving beyond Citation Analysis,” 457; Fuchs et al., “Behavioral Citation Analysis,” 305; Wilson and Tenopir, “Local Citation Analysis,” 1394.