The Open Access Citation Advantage: Does It Exist and What Does It Mean for Libraries? Colby Lewis INFORMATION TECHNOLOGY AND LIBRARIES | SEPTEMBER 2018 50 Colby Lewis (colbyllewis@gmail.com), a second year Master of Science in Information student at the University of Michigan School of Information, is winner of the 2018 LITA/Ex Libris Student Writing Award. ABSTRACT The last literature review of research on the existence of an open access citation advantage (OACA) was published in 2011 by Philip M. Davis and William H. Walters. This paper reexamines the conclusions reached by Davis and Walters by providing a critical review of OACA literature that has been published since 2011 and explores how increases in open access publication trends could serve as a leveraging tool for libraries against the high costs of journal subscriptions. INTRODUCTION Since 2001, when the term “open access” was first used in the context of scholarly literature, the debate over whether there is a citation advantage (CA) caused by making articles open access (OA) has plagued scholars and publishers alike.1 To date, there is still no conclusive answer to the question, or at least not one that the premier publishing companies have deemed worthy of acknowledging. There have been many empirical studies, but far fewer with randomized controls. The reasons for this range from data access to the numerous potential “methodological pitfalls” or confounding variables that might skew the data in favor of one argument or another. The most recent literature review of articles that explored the existence (or lack thereof) of an open access citation advantage (OACA) was published in 2011 by Philip M. Davis and William H. Walters. In that review, Davis and Walters ultimately concluded that “while free access leads to greater readership, its overall impact on citations is still under investigation. The large access -citation effects found in many early studies appear to be artifacts of improper analysis and not the result of a causal relationship.”2 This paper seeks to reexamine the conclusions reached by Davis and Walters in 2011 by providing a critical review of OACA literature that have been published since their 2011 literature review.3 This paper will examine the methods and conclusions provoking such criticisms and whether these criticisms are addressed in the studies. I will begin by identifying some of the top confounders in OACA studies, in particular the potential for self-archiving bias. I will then examine articles from July 2011, when Davis and Walters published their findings, to July 2017. There will be a few exceptions to this time frame, but the studies cited in figures 4 and 5 are entirely from this period. In addition to reviewing OACA studies since Davis and Walters’ March 2011 study, I will explore the implications of an OACA on the future of publishing and the role of librarians in the subscription process. As Antelman points out in her Association of College and Research Libraries conference paper, “Leveraging the Growth of Open Access in Library Collection Decision Making,” it is the responsibility of libraries to use the newest data and technology available to them in the interest of best serving their patrons and advancing scholarship.4 In connecting OACA mailto:colbyllewis@gmail.com THE OPEN ACCESS CITATION ADVANTAGE | LEWIS 51 https://doi.org/10.6017/ital.v37i3.10604 studies and the potential bargaining power an OACA could bring libraries, I assess the current roles that universities and university libraries play in promoting (or not) OA publications and the implications of an OACA for researchers, universities, and libraries, and I provide suggestions on how recent research could influence the present trajectory. I conclude by summarizing what my findings tell us about the existence (or lack thereof) of an OACA, and what these findings imp ly for the future of library journal subscriptions and the publish-or-perish model for tenure. Lastly, I will suggest some alternative metrics to citations that could be used by libraries in determining future journal subscriptions and general collection management. SELF-ARCHIVING BIAS AND WHY IT DOESN’T MATTER The idea of a self-archiving bias is based upon the concept that, if faced with a choice, authors will always opt to make their best work more widely available. Effectively, when open access is not mandated, these articles may be specifically chosen to be made open access to increase readership and, hypothetically, citations.5 This biased selection method has the potential to confound the results of OACA studies because of the intuitive notion that an author’s best work is much more likely to be cited than any of their other work. Its effect is amplified by making this work available OA, but it prevents studies in which articles were self-archived from being able to convincingly claim that the citation advantage these articles received was due to OA and not to its inherent quality and subsequent likelihood to be cited anyway. In a 2010 study, Gargouri et al. determined that articles by authors whose institutions mandated self-archiving (such as in an institutional repository [IR]) saw an OACA just as great for articles that were mandated to be OA as for articles that were self-selected to be OA.6 This by no means proves a causal relationship between OA and CA, but does counter the notion that self -archived articles are an uncontrollable confounder that automatically compromises the legitimacy of OACA studies.7 Ottaviani affirms this conclusion in a 2016 study in which he writes, “In the long run better articles gain more citations than expected by being made OA, adding weight to the results reported by Gargouri et al.”8 In short, claiming that articles self-selected for self-archiving irreparably confound OACA studies ignores the fact that these authors have accounted for the likelihood that articles of higher quality will inherently be cited more. As Gargouri et al. put it, “The OA advantage [to self-archived articles] is a quality advantage, rather than a quality bias” (italics in original).9 GOLD VERSUS GREEN AND THEIR EFFECT ON OACA ANALYSES Many critics of OACA studies have argued that such studies do not distinguish between Gold OA, Green OA, and Hybrid (subscription journals that offer the option for authors to opt-in to Gold OA) journals in their sample pool, thus skewing the results of their studies. In fact, there are many acknowledged subcategories of OA, but for the purposes of this paper, I will primarily focus on Gold, Green, and hybrid OA. Figure 1, provided by Elsevier as a guide for their clients, distinguishes between Gold and Green OA.10 While the chart provided applies specifically to those looking to publish with Elsevier, it highlights the overarching differences between Gold OA and Green OA. A comprehensive list of OA journals is available through the Directory of Open Access Journals (DOAJ) website (https://doaj.org/). https://doaj.org/ INFORMATION TECHNOLOGY AND LIBRARIES | SEPTEMBER 2018 52 Figure 1. Elsevier explains to potential clients their options for publishing OA with Elsevier and the differences between publishing with Gold OA versus Green OA. The argument that not distinguishing between Gold OA and Green OA in OACA studies distorts study results primarily stems from the potential for skew in Green OA journals. Green OA journals allow authors to self-archive their articles after publication, but the articles are often not made full OA until an embargo period has passed. This problem was addressed in a recent study conducted by Science-Metrix and 1science, who manually checked and coded approximatively 8,100 top-level domains (TLDs).11 It is important to note that this study was made available as a white paper on the 1science website and has not been published in a peer-reviewed journal. Additionally, 1science is a company built on providing OA solutions to libraries, which means they have a vested interest in proving the existence of an OACA. However, just as publishers such as Elsevier have a vested interest in a substantial OACA not existing, this should not prevent us from examining their data. For their study, 1science did not distinguish hybrid journals as being in a distinct journal category. Critics, such as the editorial director of journals policy for Oxford University Press, David Crotty, were quick to fixate on this lack of distinction as a means of discrediting the study.12 Employees of Elsevier were similarly inclined to criticize the study, declaring that it, “like many others [studies] on this topic, does not appear to be randomized and controlled.”13 However, Archambault et al., acknowledging that their study “does not examine the overlap between green and gold,” have provided an extremely comprehensive sample pool, examining 3,350,910 OA papers published between 2007 and 2009 in 12,000 journals.14 This paper examines the notion that “the advantage of OA is partly due to citations having a chance to arrive sooner . . . and concludes that the purported head start of OA papers is actually contrary to observed data.” 15 THE OPEN ACCESS CITATION ADVANTAGE | LEWIS 53 https://doi.org/10.6017/ital.v37i3.10604 In a more recent study published in February 2018, Piwowar et al. examine the prevalence of OA and average relative citation (ARC) based on three sample groups of one hundred thousand articles each: “(1) all journal articles assigned a Crossref DOI, (2) recent journal articles indexed in Web of Science, and (3) articles viewed by users of Unpaywall, an open-source browser extension that lets users find OA articles using oaDOI.”16 Unlike the 1science study, Piwowar et al. had a twofold purpose: to examine the prevalence of OA articles available on the web and whether an OACA exists based on their sample findings. I do not include their results in my literature review because of the dual focus of their study, although I do compare their results with those of Archambault et al. and analyze the implications of their findings. BRONZE: NEITHER GOLD NOR GREEN In their article, Piwowar et al. introduce a new category of OA publication: Bronze. If Gold OA refers to complete open access at the time of publication, and Green OA refers to articles published in a paywalled journal but ultimately made OA either after an embargo period or via an IR, Bronze OA refers to OA articles that somehow don’t fit into either of these categories. Piwowar et al. define Bronze OA articles as “free to read on the publisher page, but without any clearly identifiable license.”17 However, as Crotty points out in a Scholarly Kitchen article reflecting on the preprint version of Piwowar et al.’s article, “Bronze” already exists as an OA category, but has simply been called “public access.”18 While coining “Bronze” as a new term for “public access” is helpful in connecting it to OA terms such as “Green” and “Gold,” it is not quite the new phenomenon it is touted to be. ARC AS AN INDICATION OF AN OACA Both Archambault et al. and the authors of the 1science paper provide the ARC as a means of establishing a paper’s impact on the larger research community. 19 Within their ARC analyses, Archambault et al. distinguish between non-OA and OA, within which they differentiate between Gold and Green OA (figure 2). Piwowar et al. group papers by closed (non-OA) and OA, with the following OA subcategories: Bronze, hybrid, Gold, and Green OA (figure 3). An ARC of 1.0 is the expected amount of citations an article will receive “based on documents published in the same year and [National Science Foundation (NSF)] specialty.” 20 Based on this standard, articles with an ARC above or below 1.0 represent a citation impact that percentage above or below the expected citation impact of like articles. For example, an article with an ARC of 1.23 has received 23 percent more citations than expected for articles of similar content and quality. This scale can be incredibly useful in determining the presence of a citation advantage, and it can enable researchers to determine overall CA patterns. INFORMATION TECHNOLOGY AND LIBRARIES | SEPTEMBER 2018 54 Figure 2. Research impact of paywalled (not OA) versus open access (OA) papers “computed by Science-Metrix and 1science using oaIndx and the Web of Science.” Archambault et al., “Research Impact of Paywalled Versus Open Access Papers,” white paper, Science-Metrix and 1science, 2016, http://www.1science.com/1numbr/. Critics’ fixation on the “randomized and controlled” nature of the 1science study ignores the fact that the authors do not claim causation. Rather, their findings suggest the existence of an OACA when comparing OA (in all forms) and non-OA (in any form) articles (see figure 2). The authors ultimately conclude that “in all these fields, fostering open access (without distinguishing between gold and green) is always a better research impact maximization strategy than relying on strictly paywalled papers.”21 Unlike Archambault et al., Piwowar et al. found that Gold OA articles had a significantly lower ARC, and that the average ARC of all OA balances out to 1.18 because of the high ARCs of Bronze (1.22), hybrid (1.31), and Green (1.33). However, both studies fou nd that non-OA (referred to by Piwowar et al. as “closed”) articles had an ARC below 1.0, suggesting a definitive correlation between OA (without specifying type) and an increase in citations. http://www.1science.com/1numbr/ THE OPEN ACCESS CITATION ADVANTAGE | LEWIS 55 https://doi.org/10.6017/ital.v37i3.10604 Figure 3. “Average relative citations of different access types of a random sample of World of Science (WoS) articles and review with a Digital Object Identifier (DOI) published between 2009 and 2015.” Heather Piwowar et al., “The State of OA: A Large-Scale Analysis of the Prevalence and Impact of Open Access Articles,” PeerJ, February 13, 2018, https://doi.org/10.7717/peerj.4375. SIX YEARS AND WHAT HAS CHANGED IN OACA RESEARCH Between July 2011 and the publication of Piwowar et al.’s work in February 2018, nine new OACA studies have been published in peer-reviewed journals. Of these, five only look at the OACA in one field, such as cytology or dentistry. The other four are multidisciplinary studies, two of which are repository-specific and only use articles from Deep Blue and Academia.edu, respectively. This is important to note because of critics’ earlier stated objections to the use of studies that are not randomized controlled studies. However, the Deep Blue study can still be considered a randomized controlled sample group because the authors are not self-selecting articles to upload to the repository as they are with Academia.edu. Rather, articles were made accessible through Deep Blue “via blanket licensing agreements between the publishers and the [University of Michigan] library.”22 Some of the field-specific studies use sample sizes that may not reflect a general OACA, but rather one only for that field, and in certain cases, only for a single journal. FIELD-SPECIFIC STUDIES Between July 2011 and July 2017, five field-specific studies were conducted to determine whether an OACA existed in those fields. I summarize the scope and conclusions of these studies in table 1. As you can see from the table, the article sample size vastly varied between studies, but that can likely be accounted for by considering the specific fields studied since there are only five major cytopathology journals and nearly fifty major ecology journals. Piwowar et al. acknowledge this in their study, noting that the NSF assigns all science journals “exactly one ‘discipline’ (a high-level categorization) and exactly one ‘specialty’ (a finer-grained categorization).”23 The more deeply nested in an NSF discipline a subject is, the more specialized the field becomes and the fewer journals there are on the subject. This alone is reason not to extrapolate from the results of these studies and project their results on the existence of OACA across all fields. https://doi.org/10.7717/peerj.4375 INFORMATION TECHNOLOGY AND LIBRARIES | SEPTEMBER 2018 56 Only two of these studies, those focused on an OACA in dentistry and ecology, can be cons idered truly randomized controlled studies. Both the cytopathology and marine ecology studies chose a specific set of journals from which to draw their entire sample pool. While the dentistry and ecology studies can be considered randomized controlled in nature, they still only reflect the occurrence (or lack thereof) of an OACA in those specific fields. It would be irresponsible to allow the results from studies in a single field of a single discipline to represent OACA trends across all disciplines. Therefore, it is surprising that Elsevier employees use the dentistry study to make such a claim. Hersh and Plume write, “Another recent study by Hua et al (2016) looking at citations of open access articles in dentistry found no evidence to suggest that open access articles receive significantly more citations than non-open access articles.”24 The key phrase missing from the end of this analysis is in dentistry. One might question whether a claim about multidisciplinary OACA can effectively be extrapolated from a single-field analysis. The authors do, two sentences later, qualify their earlier statement by saying, “In dentistry at least, the type of article you publish seems to make a difference but not OA status.”25 That is indeed what this study seems to show, and is therefore a logical claim to make. Likewise, the three empirical studies in table 1 show that, for those respective fields, OA status does correlate to a citation advantage. In the case of the ecology study, the authors are confident enough in their randomized controlled methodology to claim causation. 26 The ecology study is the most recently published OACA study, and its authors were able to learn from similar past studies about the necessary controls and potential confounders in OACA studies. With this knowledge, Tang et al. determined that: By comparing OA and non-OA articles within hybrid journals, our estimate of the citation advantage of OA articles sets controls for many factors that could confound other comparisons. Numerous studies have compared articles published in OA journals to those in non-OA journals, but such comparison between different journals could not rule out the impacts of potentially confounding factors such as publication time (speed) and quality and impact (rank) of the journal. These factors are effectively controlled with our focus on hybrid journals, thereby providing robust and general estimates of citation advantages on which to base publication decisions. 27 THE OPEN ACCESS CITATION ADVANTAGE | LEWIS 57 https://doi.org/10.6017/ital.v37i3.10604 SUMMARY OF KEY FIELD-SPECIFIC STUDIES Author Study Design Content Number of Articles Controls Results, Interpretation, and Conclusion Clements 2017 Empirical 3 hybrid-OA marine ecology journals All articles published in these journals between 2009 and 2012; specific number not provided JIF; Article type; Self- citations “On average, open access articles received more peer-citations than non- open access articles.” OACA found. Frisch et al. 2014 Empirical 5 cytopathology journals; 1 OA and 4 non-OA 314 articles published between 2007 and 2011 JIF; Author frequency; Publisher neutrality “Overall, the averages of both CPP and Q values were higher for OA Cytopathology Journal (CytoJournal) than traditional non-OA journals.” OACA found. Gaulé and Maystre 2011 Empirical 1 major biology journal 4,388 articles published between 2004 and 2006 Last author; Characteristics; Article quality “We find no evidence for a causal effect of open access on citations. However, a quantitatively small causal effect cannot be statistically ruled out.” OACA not found. Hua et al. 2016 Randomized controlled Articles randomly selected from PubMed database, not specific dentistry journals 908 articles published in 2013 Randomized article selection; Exclusion of articles Unrelated to dentistry; Multi- database search to determine OA status “In the present study, there was no evidence to support the existence of OA ‘citation advantage’, or the idea that OA increases the citation of citable articles.” OACA not found. Tang et al. 2017 Randomized controlled 46 hybrid-OA ecology journals 3,534 articles published between 2009 and 2013 GNI of author country; Randomized article pairing; Article length “Overall, OA articles received significantly more citations than non-OA articles, and the citation advantage averaged approximately one citation per article per year and increased cumulatively over time after publication.” OACA found. Table 1. Scope, Controls, and Results of Field-Specific OACA Studies Since 2011. Based on a chart in Stephan Mertens, “Open Access: Unlimited Web Based Literature Searching,” Deutsches Ärzteblatt International 106, no. 43 (2009): 711. JIF, journal impact factor; CPP, citations per publication; Q, Q-value (see Frisch, Nora K., Romil Nathan, Yasin K. Ahmed, and Vinod B. Shidham. “Authors Attain Comparable or Slightly Higher Rates of Citation Publishing in an Open Access Journal (CytoJournal) Compared to Traditional Cytopathology Journals—A Five Year (2007–2011) Experience.” CytoJournal 11, no. 10 (April 2014). https://doi.org/10.4103/1742-6413.131739 for specific equation used.) https://doi.org/10.4103/1742-6413.131739 INFORMATION TECHNOLOGY AND LIBRARIES | SEPTEMBER 2018 58 SUMMARY OF KEY MULTIDISCIPLINARY STUDIES Author Study Design Content Number of Articles Controls Results, Interpretation, and Conclusion McCabe and Snyder 2014 Empirical 100 journals in ecology, botany, and multidisciplinary science All articles published in these journals between 1996 and 2005; specific number not provided JIF; Journal founding year “We found that open access only provided a significant increase for those volumes made openly accessible via the narrow channel of their own websites rather than the broader PubMed Central platform.” OACA found. Niyazov et al. 2016 Empirical Unspecified number of journals across 23 academic divisions 31,216 articles published between 2009 and 2012 Field; JIF; Publication vs. upload Date “We find a substantial increase in citations associated with posting an article to Academia.edu. . . . We find that a typical article that is also posted to Academia.edu has 49% more citations than one that is only available elsewhere online through a non-Academia.edu venue.” OACA found for Academia.edu. Ottaviani 2016 Randomized controlled Unspecified number of journals who have blanket licensing agreements between the publishers and the University of Michigan Library 93,745 articles published between 1990 and 2013 Self-selection “Even though effects found here are more modest than reported elsewhere, given the conservative treatments of the data and when viewed in conjunction with other OACA studies already done, the results lend support to the existence of a real, measurable, open access citation advantage with a lower bound of approximately 20%.” OACA found. Sotudeh et al. 2015 Empirical 633 APC-funded OA journals published by Springer and Elsevier 995,508 articles published between 2007 and 2011 Journals who adopted OA policies after 2007 Journals with non– article processing charge OA policies “The APC OA papers are, also, revealed to outperform the TA ones in their citation impacts in all the annual comparisons. This finding supports the previous results confirming the citation advantage of OA papers.” OACA found. Table 2. Scope, Controls, and Results of Multi-Disciplinary OACA Studies Since 2011. JIF, journal impact factor; APC, article processing charge; TA, toll access THE OPEN ACCESS CITATION ADVANTAGE | LEWIS 59 https://doi.org/10.6017/ital.v37i3.10604 Based on the randomized controlled methodology that Tang et al. found hybrid journals to provide, it is possible that this study may serve as an ideal model for future larger OACA studies across multiple disciplines. However, more field-specific hybrid journal studies will have to be conducted before determining if this model would be the most accurate method for measuring OACA across multiple disciplines in a single study. MULTIDISCIPLINARY STUDIES The multidisciplinary OACA studies conducted since 2011 include a single randomized control study and three empirical studies (table 2). All these studies found an OACA; in the case of Niyazov et al., an OACA was found specifically for articles posted to Academia.edu. I included this study because it is an important contribution to the premise that a relationship exists between self - selection and OACA. Niyazov et al. highlight this point in the section “Sources of Selection Bias in Academia.edu Citations,” explaining that “even if Academia.edu users were not systematically different than non-users, there might be a systematic difference between the papers they choose to post and those they do not. As [many] . . . have hypothesized, users may be more likely to post their most promising, ‘highest quality’ articles to the site, and not post articles they believe will be of more limited interest.”28 To underscore this point, I refer to Gargouri et al., who stated that “the OA advantage [to self - archived articles] is a quality advantage, rather than a quality bias” (italics in original).29 Again, it is unsurprising that articles of higher caliber are cited more and that making such articles more readily available increases the amount of citations they would likely already receive. Similar to my conclusion in the field-specific study section, we simply need more randomized controlled studies, such as Ottaviani’s, to determine the nature and extent of the relationship between OA and CA across multiple disciplines. CONCLUSIONS Critics of some of the most recent studies, specifically Archambault et al. and Ottaviani, have argued that authors of OACA studies are too quick to claim causation. While a claim of causation does indeed require strict adherence to statistical methodology and control of potential confounders, few of the authors I have examined actually claim causation. They recognize that the empirical nature of their studies is not enough to prove causation, but rather to provide insight into the correlation between open access and a citation advantage. In all their conclusions, these authors acknowledge that further studies are needed to prove a causal relationship between OA and CA. The recent work published by Piwowar et al. provides a potential model for replication by other researchers, and Ottaviani offers a replicable method for other large research institutions with non-self-selecting institutional repositories. Alternatively, field-specific studies conducted in the style of Tang et al. across all fields would serve to provide a wider array of evidence for the occurrence of field-specific OACA and therefore of a more widespread OACA. Recent developments in OA search engines have created alternative routes to many of the same articles offered by subscriptions, but at a fraction (if any) of the cost. Antelman proposed that libraries use an OA-adjusted cost per download (OA-adj CPD), a metric that “subtracts the downloads that could be met by OA copies of articles within subscription journals,” as a tool for negotiating the price of journal subscriptions.30 By calculating an OA-adj CPD, libraries could INFORMATION TECHNOLOGY AND LIBRARIES | SEPTEMBER 2018 60 potentially leverage their ability to access journal articles through means other than traditional subscription bundles to save money and encourage OA publication. While Antelman suggests using OA-adj CPD as a leveraging tool when making deals with publishers for journals subscriptions, I suggest that libraries use the data-gathering methods of Piwowar et al. via Unpaywall to determine whether enough articles from a specific journal can be found OA via Unpaywall. By using metrics such as those collected by Piwowar et al. through Unpaywall, the potential confounding variable of articles found through illegitimate means (such as SciHub) is alleviated. Instead, Piwowar et al.’s metrics focus on tracking the percentage of material searched by library patrons that can be found OA through the Unpaywall browser extension. According to Unpaywall’s “Libraries User Guide” page, libraries “can integrate Unpaywall into their SFX, 360 Link, or Primo link resolvers, so library users can read OA copies in cases where there's no subscription access. Over 1000 libraries worldwide are using this now. ”31 Ideally, scholars will also be more willing to publish papers OA, and institutions will be more supportive of providing the necessary costs for making publications OA. Though the publish-or- perish model still reigns in academia, there is great potential in encouraging tenured professors to publish OA by supplementing the costs through institutional grants and other incentives wrapped into a tenure agreement. Perhaps through this model, as Gargouri et al. have suggested, the longstanding publish-or-perish doctrine will give way to an era of “self-archive to flourish.”32 BIBLIOGRAPHY Antelman, Kristin. “Leveraging the Growth of Open Access in Library Collection Decision Making.” ACRL 2017 Proceedings: At the Helm, Leading the Transformation, March 22–25, Baltimore, Maryland, ed. Dawn M. Mueller (Chicago: Association of College and Research Libraries, 2017), 411–22. http://www.ala.org/acrl/sites/ala.org.acrl/files/content/conferences/confsandpreconfs/2017/L everagingtheGrowthofOpenAccess.pdf. Archambault, Éric, Grégoire Côté, Brooke Struck, and Matthieu Voorons. “Research Impact of Paywalled Versus Open Access Papers.” White Papers, Science-Metrix and 1science, 2016. http://www.1science.com/1numbr/. Calver, Michael C. and J. Stuart Bradley. “Patterns of Citations of Open Access and Non -Open Access Conservation Biology Journal Papers and Book Chapters.” Conservation Biology 24, no. 3 (May 2010): 872-80. https://doi.org/10.1111/j.1523-1739.2010.01509.x. Chua, S. K., Ahmad M. Qureshi, Vijay Krishnan, Dinker R. Pai, Laila B. Kamal, Sharmilla Gunasegaran, M. Z. Afzal, Lahri Ambawatta, J. Y. Gan, P. Y. Kew, et al. “The Impact Factor of an Open Access Journal Does Not Contribute to an Article’s Citations” [version 1; referees: 2 approved]. F1000 Research 6 (2017): 208. https://doi.org/10.12688/f1000research.10892.1. Clarivate Analytics. “InCites Journal Citation Reports.” Dataset updated September 9, 2017. https://jcr.incites.thomsonreuters.com/. Clements, Jeff C. “Open Access Articles Receive More Citations in Hybrid Marine Ecology Journals.” FACETS 2 (January 2017): 1–14. https://doi.org/10.1139/facets-2016-0032. http://www.ala.org/acrl/sites/ala.org.acrl/files/content/conferences/confsandpreconfs/2017/LeveragingtheGrowthofOpenAccess.pdf http://www.ala.org/acrl/sites/ala.org.acrl/files/content/conferences/confsandpreconfs/2017/LeveragingtheGrowthofOpenAccess.pdf http://www.1science.com/1numbr/ https://doi.org/10.1111/j.1523-1739.2010.01509.x https://doi.org/10.12688/f1000research.10892.1 https://jcr.incites.thomsonreuters.com/ https://doi.org/10.1139/facets-2016-0032 THE OPEN ACCESS CITATION ADVANTAGE | LEWIS 61 https://doi.org/10.6017/ital.v37i3.10604 Crotty, David. “Study Suggests Publisher Public Access Outpacing Open Access; Gold OA Decreases Citation Performance.” Scholarly Kitchen, October 4, 2017. https://scholarlykitchen.sspnet.org/2017/10/04/study-suggests-publisher-public-access- outpacing-open-access-gold-oa-decreases-citation-performance/. Crotty, David. “When Bad Science Wins, or ‘I’ll See It When I Believe It.’” Scholarly Kitchen, August 31, 2016. https://scholarlykitchen.sspnet.org/2016/08/31/when-bad-science-wins-or-ill-see-it- when-i-believe-it/. Davis, Philip M. “Open Access, Readership, Citations: A Randomized Controlled Trial of Scientific Journal Publishing.” FASEB Journal 25, no. 7 (July 2011): 2129–34. https://doi.org/10.1096/fj.11- 183988. Davis, Philip M., and William H. Walters. “The Impact of Free Access to the Scientific Literature: A Review of Recent Research.” Journal of the Medical Library Association 99, no. 3 (July 2011): 208– 17. https://doi.org/10.3163/1536-5050.99.3.008. Elsevier. “Your Guide to Publishing Open Access with Elsevier.” Amsterdam, Netherlands: Elsevier, 2015. https://www.elsevier.com/__data/assets/pdf_file/0020/181433/openaccessbooklet_May.pdf. Evans, James A. and Jacob Reimer. “Open Access and Global Participation in Science.” Science 323, no. 5917 (February 2009): 1025. https://doi.org/10.1126/science.1154562. Eysenbach, Gunther. “Citation Advantage of Open Access Articles.” PLoS Biology 4, no. 5 (May 2006): e157. https://doi.org/10.1371/journal.pbio.0040157. Fisher, Tim. “Top-Level Domain (TLD).” Lifewire, July 30, 2017. https://www.lifewire.com/top- level-domain-tld-2626029. Frisch, Nora K., Romil Nathan, Yasin K. Ahmed, and Vinod B. Shidham. “Authors Attain Comparable or Slightly Higher Rates of Citation Publishing in an Open Access Journal (CytoJournal) Compared to Traditional Cytopathology Journals—A Five Year (2007–2011) Experience.” CytoJournal 11, no. 10 (April 2014). https://doi.org/10.4103/1742-6413.131739. Gaulé, Patrick, and Nicolas Maystre. “Getting Cited: Does Open Access Help?” Research Policy 40, no. 10 (December 2011): 1332–38. https://doi.org/10.1016/j.respol.2011.05.025. Gargouri, Yassine, Chawki Hajjem, Vincent Larivière, Yves Gingras, Les Carr, Tim Brody, and Stevan Harnad. “Self-Selected or Mandated, Open Access Increases Citation Impact for Higher Quality Research.” PLoS ONE 5, no. 10 (October 2010). https://doi.org/10.1371/journal.pone.0013636. Hajjem, Chawki, Stevan Harnad, and Yves Gingras. “Ten-Year Cross-Disciplinary Comparison of the Growth of Open Access and How it Increases Research Citation Impact.” IEEE Data Engineering Bulletin 28, no. 4 (December 2005): 39-46. Hall, Martin. “Green or Gold? Open Access After Finch.” Insights 25, no. 3 (November 2012): 235– 40. https://doi.org/10.1629/2048-7754.25.3.235. https://scholarlykitchen.sspnet.org/2017/10/04/study-suggests-publisher-public-access-outpacing-open-access-gold-oa-decreases-citation-performance/ https://scholarlykitchen.sspnet.org/2017/10/04/study-suggests-publisher-public-access-outpacing-open-access-gold-oa-decreases-citation-performance/ https://scholarlykitchen.sspnet.org/2016/08/31/when-bad-science-wins-or-ill-see-it-when-i-believe-it/ https://scholarlykitchen.sspnet.org/2016/08/31/when-bad-science-wins-or-ill-see-it-when-i-believe-it/ https://doi.org/10.1096/fj.11-183988 https://doi.org/10.1096/fj.11-183988 https://doi.org/10.3163/1536-5050.99.3.008 https://www.elsevier.com/__data/assets/pdf_file/0020/181433/openaccessbooklet_May.pdf https://doi.org/10.1126/science.1154562 https://doi.org/10.1371/journal.pbio.0040157 https://www.lifewire.com/top-level-domain-tld-2626029 https://www.lifewire.com/top-level-domain-tld-2626029 https://doi.org/10.4103/1742-6413.131739 https://doi.org/10.1016/j.respol.2011.05.025 https://doi.org/10.1371/journal.pone.0013636 https://doi.org/10.1629/2048-7754.25.3.235 INFORMATION TECHNOLOGY AND LIBRARIES | SEPTEMBER 2018 62 Hersh, Gemma, and Andrew Plume. “Citation Metrics and Open Access: What Do We Know?” Elsevier Connect, September 14, 2016. https://www.elsevier.com/connect/citation-metrics-and- open-access-what-do-we-know. Houghton, John, and Alma Swan. “Planting the Green Seeds for a Golden Harvest: Comments and Clarifications on ‘Going for Gold.’” D-Lib Magazine 19, no. 1/2 (January/February 2013). https://doi.org/10.1045/january2013-houghton. Hua, Fang, Heyuan Sun, Tanya Walsh, Helen Worthington, and Anne-Marie Glenny. “Open Access to Journal Articles in Dentistry: Prevalence and Citation.” Journal of Dentistry 47 (April 2016): 41– 48. https://doi.org/10.1016/j.jdent.2016.02.005. Internet Corporation for Assigned Names and Numbers. “List of Top-Level Domains.” Last updated September 13, 2018. https://www.icann.org/resources/pages/tlds-2012-02-25-en. Jump, Paul. “Open Access Papers ‘Gain More Traffic and Citations.’” Times Higher Education, July 30, 2014. https://www.timeshighereducation.com/home/open-access-papers-gain-more-traffic- and-citations/2014850.article. McCabe, Mark J., and Christopher M. Snyder. “Identifying the Effect of Open Access on Citations Using a Panel of Science Journals.” Economic Inquiry 52, no. 4 (October 2014): 1284–1300. https://doi.org/10.11111/ecin.12064. McCabe, Mark J., and Christopher M. Snyder. “Does Online Availability Increase Citations? Theory and Evidence from a Panel of Economics and Business Journals.” Review of Economics and Statistics 97, no. 1 (March 2015): 144–65. https://doi.org/10.1162/REST_a_00437. Mertens, Stephan. “Open Access: Unlimited Web Based Literature Searching.” Deutsches Ärzteblatt International 106, no. 43 (2009): 710–12. https://doi.org/10.3238/arztebl.2009.0710. Moed, Hank. “Does Open Access Publishing Increase Citation or Download Rates?” Research Trends 28 (May 2012). https://www.researchtrends.com/issue28-may-2012/does-open-access- publishing-increase-citation-or-download-rates/. Niyazov, Yuri, Carl Vogel, Richard Price, Ben Lund, David Judd, Adnan Akil, Michael Mortonson, Josh Schwartzman, and Max Shron. “Open Access Meets Discoverability: Citations to Articles Posted to Academia.edu.” PLoS ONE 11, no. 2 (February 2016): e0148257. https://doi.org/10.1371/journal.pone.0148257. Ottaviani, Jim. “The Post-Embargo Open Access Citation Advantage: It Exists (Probably), It’s Modest (Usually), and the Rich Get Richer (of Course).” PLoS ONE 11, no. 8 (August 2016): e0159614. https://doi.org/10.1371/journal.pone.0159614. Pinfield, Stephen, Jennifer Salter, and Peter A. Bath. “A ‘Gold-Centric’ Implementation of Open Access: Hybrid Journals, the ‘Total Cost of Publication,’ and Policy Development in the UK and Beyond.” Journal of the Association for Information Science and Technology 68, no. 9 (September 2017): 2248–63. https://doi.org/10.1002/asi.23742. Piwowar, Heather, Jason Priem, Vincent Larivière, Juan Pablo Alperin, Lisa Matthias, Bree Norlander, Ashley Farley, Jevin West, and Stefanie Haustein. “The State of OA: A Large-Scale https://www.elsevier.com/connect/citation-metrics-and-open-access-what-do-we-know https://www.elsevier.com/connect/citation-metrics-and-open-access-what-do-we-know https://doi.org/10.1045/january2013-houghton https://doi.org/10.1016/j.jdent.2016.02.005 https://www.icann.org/resources/pages/tlds-2012-02-25-en https://www.timeshighereducation.com/home/open-access-papers-gain-more-traffic-and-citations/2014850.article https://www.timeshighereducation.com/home/open-access-papers-gain-more-traffic-and-citations/2014850.article https://doi.org/10.11111/ecin.12064 https://doi.org/10.1162/REST_a_00437 https://doi.org/10.3238/arztebl.2009.0710 https://www.researchtrends.com/issue28-may-2012/does-open-access-publishing-increase-citation-or-download-rates/ https://www.researchtrends.com/issue28-may-2012/does-open-access-publishing-increase-citation-or-download-rates/ https://doi.org/10.1371/journal.pone.0148257 https://doi.org/10.1371/journal.pone.0159614 https://doi.org/10.1002/asi.23742 THE OPEN ACCESS CITATION ADVANTAGE | LEWIS 63 https://doi.org/10.6017/ital.v37i3.10604 Analysis of the Prevalence and Impact of Open Access Articles.” PeerJ (February 13, 2018): 6:e4375. https://doi.org/10.7717/peerj.4375. Research Information Network. “Nature Communications: Citation Analysis.” Press release, 2014. https://www.nature.com/press_releases/ncomms-report2014.pdf. Riera, M. and E. Aibar. “¿Favorece la publicación en abierto el impacto de los artículos científicos? Un estudio empírico en el ámbito de la medicina intensive” [Does open access publishing increase the impact of scientific articles? An empirical study in the field of intensive care medicine]. Medicina Intensiva 37, no. 4 (May 2013): 232-40. http://doi.org/10.1016/j.medin.2012.04.002. Sotudeh, Hajar, Zahra Ghasempour, and Maryam Yaghtin. “The Citation Advantage of Author-Pays Model: The Case of Springer and Elsevier OA Journals.” Scientometrics 104 (June 2015): 581–608. https://doi.org/10.1007/s11192-015-1607-5. Swan, Alma, and John Houghton. “Going for Gold? The Costs and Benefits of Gold Open Access for UK Research Institutions: Further Economic Modelling.” Report to the UK Open Access Implementation Group, June 2012. http://wiki.lib.sun.ac.za/images/d/d3/Report-to-the-uk-open- access-implementation-group-final.pdf. Tang, Min, James D. Bever, and Fei-Hai Yu. “Open Access Increases Citations of Papers in Ecology.” Ecosphere 8, no. 7 (July 2017): 1–9. https://doi.org/10.1002/ecs2.1887. Unpaywall. “Libraries User Guide.” Accessed September 13, 2018. https://unpaywall.org/user- guides/libraries. Wray, K. Brad. “No New Evidence for a Citation Benefit for Author-Pay Open Access Publications in the Social Sciences and Humanities.” Scientometrics 106 (January 2016): 1031–35. https://doi.org/10.1007/s11192-016-1833-5. ENDNOTES 1 Elsevier, “Your Guide to Publishing Open Access with Elsevier” (Amsterdam, Netherlands: Elsevier, 2015), 2, https://www.elsevier.com/__data/assets/pdf_file/0020/181433/openaccessbooklet_May.pdf. 2 Philip M. Davis and William H. Walters, “The Impact of Free Access to the Scientific Literature: A Review of Recent Research,” Journal of the Medical Library Association 99, no. 3 (July 2011): 213, https://doi.org/10.3163/1536-5050.99.3.008. 3 David and Walters, “The Impact of Free Access,” 208. 4 Kristin Antelman, “Leveraging the Growth of Open Access in Library Collection Decision Making,” ACRL 2017 Proceedings: At the Helm, Leading the Transformation, March 22–25, Baltimore, Maryland, ed. Dawn M. Mueller (Chicago: Association of College and Research Libraries, 2017): 411, 413, http://www.ala.org/acrl/sites/ala.org.acrl/files/content/conferences/confsandpreconfs/201 7/LeveragingtheGrowthofOpenAccess.pdf. https://doi.org/10.7717/peerj.4375 https://www.nature.com/press_releases/ncomms-report2014.pdf http://doi.org/10.1016/j.medin.2012.04.002 https://doi.org/10.1007/s11192-015-1607-5 http://wiki.lib.sun.ac.za/images/d/d3/Report-to-the-uk-open-access-implementation-group-final.pdf http://wiki.lib.sun.ac.za/images/d/d3/Report-to-the-uk-open-access-implementation-group-final.pdf https://doi.org/10.1002/ecs2.1887 https://unpaywall.org/user-guides/libraries https://unpaywall.org/user-guides/libraries https://doi.org/10.1007/s11192-016-1833-5 https://www.elsevier.com/__data/assets/pdf_file/0020/181433/openaccessbooklet_May.pdf http://jmla.mlanet.org/ https://doi.org/10.3163/1536-5050.99.3.008 http://www.ala.org/acrl/sites/ala.org.acrl/files/content/conferences/confsandpreconfs/2017/LeveragingtheGrowthofOpenAccess.pdf http://www.ala.org/acrl/sites/ala.org.acrl/files/content/conferences/confsandpreconfs/2017/LeveragingtheGrowthofOpenAccess.pdf INFORMATION TECHNOLOGY AND LIBRARIES | SEPTEMBER 2018 64 5 Research Information Network, “Nature Communications: Citation Analysis,” press release, 2014, https://www.nature.com/press_releases/ncomms-report2014.pdf. 6 Gargouri et al., “Self-Selected or Mandated, Open Access Increases Citation Impact for Higher Quality Research,” PLoS ONE 5, no. 10 (October 2010): 17, https://doi.org/10.1371/journal.pone.0013636. 7 David Crotty, “When Bad Science Wins, or ‘I’ll See It When I Believe It’,” Scholarly Kitchen, August 31, 2016, https://scholarlykitchen.sspnet.org/2016/08/31/when-bad-science-wins-or-ill-see- it-when-i-believe-it/. 8 Jim Ottaviani, “The Post-Embargo Open Access Citation Advantage: It Exists (Probably), It’s Modest (Usually), and the Rich Get Richer (of Course),” PLoS ONE 11, no. 8 (August 2016): 9, https://doi.org/10.1371/journal.pone.0159614. 9 Gargouri et al., “Self-Selected or Mandated,” 18. 10 Elsevier, “Your Guide to Publishing,” 2. 11 Top-Level Domain (TLD) refers to the last string of letters in an internet domain name (i.e., the TLD of www.google.com is .com). For more information on TLDs, see Tim Fisher, “Top-Level Domain (TLD),” Lifewire, July 30, 2017, https://www.lifewire.com/top-level-domain-tld- 2626029. For a full list of TLDs, see “List of Top-Level Domains,” Internet Corporation for Assigned Names and Numbers, last updated September 13, 2018, https://www.icann.org/resources/pages/tlds-2012-02-25-en. 12 Crotty, “When Bad Science Wins.” 13 Hersh and Plume, “Citation Metrics and Open Access: What Do We Know?,” Elsevier Connect, September 14, 2016, https://www.elsevier.com/connect/citation-metrics-and-open-access- what-do-we-know. 14 Archambault et al., “Research Impact of Paywalled Versus Open Access Papers,” white paper, Science-Metrix and 1science, 2016, http://www.1science.com/1numbr/. 15 Archambault et al., “Research Impact.” 16 Heather Piwowar et al., “The State of OA: A Large-Scale Analysis of the Prevalence and Impact of Open Access Articles,” PeerJ, February 13, 2018, https://doi.org/10.7717/peerj.4375. 17 Piwowar et al., “The State of OA,” 5. 18 David Crotty, “Study Suggests Publisher Public Access Outpacing Open Access; Gold OA Decreases Citation Performance,” Scholarly Kitchen, October 4, 2017, https://scholarlykitchen.sspnet.org/2017/10/04/study-suggests-publisher-public-access- outpacing-open-access-gold-oa-decreases-citation-performance/. https://www.nature.com/press_releases/ncomms-report2014.pdf https://doi.org/10.1371/journal.pone.0013636 https://scholarlykitchen.sspnet.org/2016/08/31/when-bad-science-wins-or-ill-see-it-when-i-believe-it/ https://scholarlykitchen.sspnet.org/2016/08/31/when-bad-science-wins-or-ill-see-it-when-i-believe-it/ https://doi.org/10.1371/journal.pone.0159614 https://www.lifewire.com/top-level-domain-tld-2626029 https://www.lifewire.com/top-level-domain-tld-2626029 https://www.icann.org/resources/pages/tlds-2012-02-25-en https://www.elsevier.com/connect/citation-metrics-and-open-access-what-do-we-know https://www.elsevier.com/connect/citation-metrics-and-open-access-what-do-we-know http://www.1science.com/1numbr/ https://doi.org/10.7717/peerj.4375 https://scholarlykitchen.sspnet.org/2017/10/04/study-suggests-publisher-public-access-outpacing-open-access-gold-oa-decreases-citation-performance/ https://scholarlykitchen.sspnet.org/2017/10/04/study-suggests-publisher-public-access-outpacing-open-access-gold-oa-decreases-citation-performance/ THE OPEN ACCESS CITATION ADVANTAGE | LEWIS 65 https://doi.org/10.6017/ital.v37i3.10604 19 Archambault et al., “Research Impact”; Piwowar et al., “The state of OA,” 15. 20 Piwowar et al., “The State of OA,” 9–10. 21 Archambault et al., “Research Impact.” 22 Ottaviani, “The Post-Embargo Open Access Citation Advantage,” 2. 23 Piwowar et al., “The State of OA,” 9. 24 Hersh and Plume, “Citation Metrics and Open Access.” 25 Hersh and Plume, “Citation Metrics and Open Access.” 26 Tang et al., “Open Access Increases Citations of Papers in Ecology,” Ecosphere 8, no. 7 (July 2017): 8, https://doi.org/10.1002/ecs2.1887. 27 Tang et al., “Open Access Increases Citations,” 7. Tang et al. list the following as examples of the “numerous studies” as quoted above, which I did not include in the quote for the purpose of brevity: (Antelman 2004, Hajjem et al. 2005, Eysenbach 2006, Evans and Reimer 2009, Calver and Bradley 2010, Riera and Aibar 2013, Clements 2017). 28 Yuri Niyazov et al., “Open Access Meets Discoverability: Citations to Articles Posted to Academia.edu,” PLoS ONE 11, no. 2 (February 2016): e0148257, https://doi.org/10.1371/journal.pone.0148257. 29 Gargouri et al., “Self-Selected or Mandated,” 18. 30 Antelman, “Leveraging the Growth,” 414. 31 “Library User Guide,” Unpaywall, accessed September 13, 2018, https://unpaywall.org/user- guides/libraries.<> 32 Gargouri et al., “Self-Selected or Mandated,” 20. https://doi.org/10.1002/ecs2.1887 https://doi.org/10.1371/journal.pone.0148257 https://unpaywall.org/user-guides/libraries https://unpaywall.org/user-guides/libraries ABSTRACT Introduction Self-Archiving Bias and Why It Doesn’t Matter Gold versus Green and Their Effect on OACA Analyses Bronze: Neither Gold nor Green arc AS an indication of an oaca Six Years and What Has Changed in OACA Research Field-Specific Studies Summary of key field-Specific Studies Summary of key MultiDISCIPLINARY STUDIES MultiDisciplinary Studies Conclusions Bibliography ENDNOTES