dow.p65 146 College & Research Libraries March 2000 Editorial Gatekeepers Confronted by the Electronic Journal Ronald F. Dow Refereed journals perform a crucial role in the formal dissemination of new knowledge. Significant to the communication process are editors and editorial board gatekeepers. These gatekeepers were surveyed to learn their views on supplanting paper-formatted journals with journals in electronic formats. The survey showed that editorial gatekeepers, al­ though generally supportive of the creation of electronic journals, are still very enamored of the paper format as both a means of publishing and a medium for archiving academic writing. It is suggested that edito­ rial gatekeepers may view electronic journals as an extension of the informal communication system of the disciplines rather than as a me­ dium for supplanting the traditional paper format of academic journals. ntegral to the scholarly en­ deavor is the discovery and f o r m a l c o m m u n i c a t i o n o f new knowledge. The paper journal has served as the primary me­ dium of formal academic communica­ tion since the seventeenth century. At the close of the nineteenth century, as the academic disciplines became more structured and academic occupations became professionalized, acceptance for publication in an academic journal assumed an honorific aspect.1 The in­ dividuals who control the relationship between academic producers of ideas and their audience of readers and be­ tween academic obscurity and the pro­ fessional recognition earned by those who publish are the editors and refer­ ees of academic journals.2 In a seminal paper on gatekeeping, Kurt Lewin described a gatekeeper as an individual or group empowered to make the decision of “in” or “out.”3 Decisions are partly predicated on ideology—that is, a system of values and beliefs that de­ termines what the gatekeeper considers to be “good” or “bad”—and partly based on the way the gatekeeper perceives a particular situation. A similarity exists between Lewin’s discussion of The paper journal has served as the primary medium of formal academic communication since the seven­ teenth century. gatekeepers and the role that editors and referees perform in academic publishing. For example, Richard D. Whitley identi­ fied a relatively high degree of personal­ ization in the formal communication sys­ tem in British social sciences.4 Individual editors often operated independently of one another, tended to ignore the opera­ tion of other journals in the academic spe- Ronald F. Dow is Dean of the River Campus Libraries at the University of Rochester; e-mail: Rond@rcl.lib.rochester.edu. 146 mailto:Rond@rcl.lib.rochester.edu Editorial Gatekeepers Confronted by the Electronic Journal 147 cialization, and frequently relied on per­ sonal knowledge of immediate colleagues when evaluating manuscripts. Similarly, in a study of the gatekeeper role in edu­ cational journal publishing, Robert .J. Silverman and Erik L. Collins concluded, from an extensive survey of participants in the publication process, that editors assumed the decision-making role in scholarly and research journal publishing by using a combination of their own judg­ ment and the advice of reviewers when determining which manuscripts to let in and which ones to keep out.5 Editorial gatekeepers are empowered to control the quality of the journal by determining what ideas are let “in” and which ones are kept “out.” By their de­ cisions, the gatekeepers facilitate the awarding of status to professionals working within the discipline, maintain the intellectual boundaries of the disci­ pline, and serve the advancement of the academic discipline as an organization of professionals.6 The outcome of their decisions appears as literature in paper- formatted scholarly and research jour­ nals. As a consequence, journal editors and refer ees assume a significant gatekeeper role within their disciplines and their academic professions.7 In recent years, a number of initiatives have sought to advance the creation of electronically formatted journals and to forward this publication medium as a re­ placement for the traditional paper-for­ matted research journal. From the works cited above, editors, with the assistance of editorial reviewers, are important de­ cision makers and even gatekeepers within the context of formal academic publishing. The opinion that these edi­ torial gatekeepers hold toward the de­ velopment of e-journals and their per­ spective on supplanting paper-formatted journals with e-journals may be impor­ tant when assessing the fate of journal publishing formats. Therefore, a survey was undertaken to determine editorial gatekeeper attitudes toward a transition of paper-formatted journals to an elec­ tronic medium. The Study The purpose of this study was to deter­ mine whether editorial gatekeeper atti­ tudes might obstruct or assist the transi­ tion of paper-formatted academic journals to electronic formats. To ascertain gatekeeper attitudes, a survey of gatekeepers in academic journal publish­ ing was undertaken. The term attitude has been defined, for the purposes of mea­ surement, as a state of readiness or ten­ dency to respond in a certain manner when the object of the attitude is stimu­ lated.8 Attitudes, reinforced by personal beliefs and feelings, may precipitate or account for some future action. Assum­ ing that such is the case, gatekeeper atti­ tudes toward the ongoing role of journals may account for gatekeeper actions rela­ tive to the transition of paper-formatted journals to electronic formats. A population of editors and editorial board members in six disciplines, listed below by discipline and journal, were tar­ geted for this study: • Mechanical Engineering: Journal of Applied Mechanics and Journal of Fluid Me­ chanics • Botany: American Journal of Botany and International Journal of Plant Sciences • Agricultural Economics: Agribusiness and American Journal of Agricultural Econom­ ics • Anthropology: American Anthro­ pologist and Cultural Anthropology • Educational Administration: Educa­ tional Administration Quarterly and Jour­ nal of Educational Administration • Political Science: American Political Science Review and Political Science Quar­ terly The titles chosen are the more promi­ nent publications in a number of disci­ plines that fall along a continuum based on paradigm strength of the discipline.9 These titles were selected to improve the likelihood that editors and editorial board members associated with the journals would be academic leaders within their disciplines. Neither these journals nor their editorial board members can be viewed statistically as representative of other jour­ 148 College & Research Libraries March 2000 nals or editors, in these or other disciplines. However, there also is no reason to believe that their views would not reflect the views of editors and editorial board members of similar journals in other disciplines. It is equally conceivable that editors and board members of smaller or less traditional or more specialized journals might express different attitudes from those expressed by these gatekeepers. The total number of people surveyed was determined by the actual list of names identified on the journal’s title page as being on the editorial board or in an editorship position. Excluded from the survey were editors and editorial board members not affiliated with colleges or universities or whose institutional ad­ dresses were outside North America. The survey instrument was designed to measure respondent answers to a series of statements relevant to the study. A Likert­ like scale was chosen to transmit responses because it allowed subjects to place them­ selves on an attitude continuum ranging from “strongly agree” to “strongly dis­ agree” for each statement asked. From the outset, it was recognized that without extensive pretesting of the sur­ vey instrument, involving numbers greater than the ultimate population to be surveyed, it would be impossible to vali­ date the degree of measurement achieved by the survey instrument. In short, it was very likely that the various modifiers dif­ ferentiating among the alternative re­ sponses on the survey instrument (“strongly agree,” “agree,” “disagree,” and “strongly disagree”) would have dif­ ferent levels of meaning for each respon­ dent. The primary procedure for improv­ ing scalability in such cases is the com­ bining of categories. For this study, cases “strongly agree” and “agree” were com­ bined as were “disagree” and “strongly disagree” when the surveys were scored and the results tabulated.10 The scores produced ordinal values. The frequency of each value was tabu­ lated by response and the medians were calculated. Cross-tabulations of state­ ments compared percentages of re­ sponses, and a correlation between val­ ues was calculated using Kendall’s tau. Kendall’s tau is a measure of the extent to which one respondent’s rankings on one variable are associated with his or her rankings of a second variable when the rankings are collected through categories of responses, as was done in this study. Kendall’s rank correlation between ordi­ nal variables is identical to Pearson’s rho in that the computed coefficient varies between -1 and +1, providing information on the strength and direction of relation­ ships between variables. Kendall’s tau, although producing slightly lower corre­ lation coefficients than rho, is preferred here because it was suspected that a large proportion of individuals surveyed would have tied ranks on many of the same variables.11,12 The Survey In early 1997, 223 surveys were mailed to the targeted population of editors and TABLE 1 Survey Characterized by Discipline Discipline No. Targeted No. Responding % Responding Mechanical engineering Botany Anthropology Agricultural economics Political science Educational administration 15 31 50 32 51 44 5 19 25 19 32 29 33 61 50 59 63 66 Total 223 129 http:tabulated.10 Editorial Gatekeepers Confronted by the Electronic Journal 149 editorial board members. The names TABLE 2 of the targeted population were culled I favor the development of electronic from the title pages of current issues journals in the discipline (n = 128).of the journals to be surveyed. Every effort was made to identify specific Agreed Disagreed Uncertain Median campus addresses for each faculty sur­ 58.6% 14.1% 27.3% Agreeveyed. Campus directories available [75] [18] [35]over the Internet were canvassed, and “who’s who”–type guides and asso­ ciation membership rosters were con­ sulted to make the mailing as specific by local address as possible. A reminder was sent to nonrespondents thirty days after the initial mailing. A total of 129 usable survey instru­ ments was returned for a response rate of 58 percent. Table 1 characterizes the population surveyed by discipline, indi­ cating the number of editors and edito­ rial board members that were targeted with questionnaires and the count and percentage of respondents. The respon­ dents were associated with seventy-three different American colleges and univer­ sities. The research question focused on gatekeeper attitudes toward creating e- journals and supplanting the paper-for­ mat with an electronic format. To address the question, gatekeepers first were asked whether they favored the development of e-journals (see table 2). Overall, 58.6 per­ cent of respondents supported the devel­ opment of e-journals, compared to 14.1 percent who did not support their devel­ opment. A series of follow-up statements was offered to better understand why respon­ dents who supported the development of e-journals did so. Gatekeepers were asked TABLE 3 whether the use of refereed journals in electronic formats would make it easier to find articles or subsections of articles to support teaching and research. The belief that electronic formats enhance access to the research literature is one of the most frequently cited positive values associated with the development of e-journals. Table 3 shows how gatekeepers responded to this statement. Slightly fewer than half of respondents agreed with the statement. When responses were correlated with gatekeepers who indicated they supported the development of e-journals, a Kendall’s tau value of .261 was produced, indicat­ ing that a positive correlation between the two statements existed. Of those who fa­ vored e-journal development, 70.5 percent agreed that ease in finding literature would improve with electronic formats. Responses by those who opposed the de­ velopment of e-journals were more broadly distributed: 41.4 percent agreed that it would be easier to use the literature, 37.9 percent disagreed, and 20.7 percent were uncertain. A possible explanation for the variance between those who supported e-journal development and those who did not could relate to respondents’ overall atti­ tudes about the vastness of the literature of the discipline. Proponents of elec­ tronic publishing maintain these for­ mats provide easier access to the If refereed journals were available in growing vastness of the literature of electronic format, it would be easier to the disciplines. Gatekeepers who be- find articles or subsections of articles to lieved that the literature of their dis- support teaching and research (N = 129). cipline had become too vast might have supported the development of Agreed Disagreed Uncertain Median e-journals because they saw elec­ tronic formats as addressing the 48.4% 30.5% 21.1% Agree problem of accessing a growing mass of material. On the other hand, those [62] [39] [27] 150 College & Research Libraries March 2000 TABLE 4 Gatekeepers Favoring the Development of Electronic Journals by Discipline Discipline Favored Did not favor Uncertain Botany Mechanical engineering Agricultural economics Anthropology Educational administration Political science 47.4% 60.0% 68.4% 68.0% 65.5% 45.2% 26.3% 40.0% 21.1% 24.0% 27.6% 32.3% 26.3% 0.0% 10.5% 8.0% 6.9% 22.6% who disagreed that the literature had be­ come too vast might have been less sup­ portive of electronic formats because they did not see the new format as solving a problem. Gatekeepers were asked whether the literature important to their discipline had become too vast for many in the discipline to manage. Just over 63 per­ cent of all respondents agreed that it had. The correlation between vastness of literature and both the capabilities of e-journals to improve use and access to the literature and support for e-jour­ nal development produced Kendall’s tau values of .033 and .046, respectively. The low correlation values indicate that support for the development of e-jour­ nals does not appear to have arisen out of a shared belief that e-formats solve problems with the growing vastness of a specific discipline’s research litera­ ture. Researchers have found diversity by academic discipline in faculty orientation and productivity. Disciplinary cultures can vary greatly to a degree that can ac­ count for differences of gatekeeper per­ spective about the future of paper-format­ ted journals. It was speculated that atti­ tudes of editors and editorial board mem­ bers could vary based on the discipline of the respondent, explaining some of the variance in attitude toward e-journal de­ velopment. Gatekeeper attitudes toward e-journal development were correlated by respondents’ academic discipline. The correlation produced a Kendall’s tau value of -.111, indicating a slightly nega­ tive correlation between the respondent’s discipline and his or her attitude toward e-journal development. As table 4 shows, little of the variance between those sup­ porting and those not supporting the de­ velopment of e-journals can be accounted for by discipline. What does seem clear is that respondents, although from a con­ tinuum of disciplines based on paradigm strength, reflected shared attitudes on the subject of the creation of e-journals. Another possible explanation of vari­ ance among those who supported the de­ velopment of e-journals, those who did not, and those who were uncertain may be predicated on the respondents’ previ­ ous experience with journals in electronic formats. Gatekeepers were asked if they had ever used e-journals. Of those re­ sponding, 65.1 percent reported never hav­ ing used a journal in an electronic format. The correlation between those who sup­ ported e-journal development and previ­ ous experience with electronic journals produced a Kendall’s tau value of .171, in­ dicating a slightly positive correlation be­ tween the two statements. Although some­ what significant, such a low correlation indicates that previous experience with e- journals was a not a driving factor in un­ derstanding gatekeeper responses to the development of e-journals. Gatekeepers were asked whether they had ever formally participated in discus­ sions about e-journals. Overall, 38.6 per­ cent of respondents had participated in such discussions and 36 percent of those favoring development of e-journals had done so. Interestingly, 61.1 percent of those not favoring the development of e- journals had participated in formal dis­ Editorial Gatekeepers Confronted by the Electronic Journal 151 of support for e-journal development.TABLE 5 Only editorial board members who Of those supporting the development of supported the development of e-jour­electronic journals at least into the near nals were asked whether, at least in the future for an electronic journal in the near future, electronic journals in the discipline to be important it must be discipline would need to be archived archived in paper format (N = 129). in paper format in order to be consid­ ered important (see table 5). Those who Agreed Disagreed Uncertain Median disagreed with the development of e­80% 13 7% 6 7% Agree journals were asked to respond to the [60] [10] [.] cussions about electronic formats at pro­ fessional meetings, in editorial broad meetings, or on campus. It is possible to speculate that those not favoring the de­ velopment of e-journals are either more willing to articulate their point of view or are more frequently asked to partici­ pate in discussions on this topic. Overall, 58.6 percent of respondents fa­ vored the development of electronic jour­ nals. A significant percentage of gate­ keepers who supported e-journal develop­ ment agreed that the format would im­ prove access to the content of journals, and almost half of the respondents who indi­ cated that they did not support e-journal development also concluded that the elec­ tronic format would improve access to the literature of their discipline. Based on the survey, no strong relationship appears to exist between respondents’ support for the development of e-journals and their disci­ pline, previous experience with e-journals, or sense of the vastness of the literature of the discipline. Attitudes Toward the Paper Format of the Journal The journal is the key element of the formal publication system in same statement (see table 6). Responses from the group who were uncertain about the need for paper archiving of e-jour­ nals similarly reported, with 88.6 percent agreement, that archiving in paper would be important. Respondents who indicated support for e-journal development were then asked whether e-journals needed to be published in both paper and electronic formats (see table 7). Those not support­ ing e-journal development were similarly asked if they thought e-journals needed to be published in both paper and in elec­ tronic formats (see table 8). A significant percentage of those not supporting the creation of e-journals agreed that, if developed, e-journals would need to be published with a paper equivalent. Statements supporting the paper archiving of e-journals and the parallel publishing of e-journals in paper formats were equally supported by those gatekeepers who agreed with the devel­ opment of e-journals and those who did not agree or were uncertain as to their position. Only four respondents (3% of the total) agreed that neither paper print- TABLE 6 Of those not supporting the develop­academia. A characteristic of formal publication has been that published ment of electronic journals at least into information is permanently archived. the near future for an electronic journal To determine whether issues related in the discipline to be important it must to the paper archiving of e-journals, be archived in paper format (N , 129). or the desire to publish e-journals in parallel with paper editions, could Agreed Disagreed Uncertain Median explain why some gatekeepers sup­ 94 4% 0% 5 6% Agree ported the development of e-journals, [1.] [0] [1] respondents were categorized by level 152 College & Research Libraries March 2000 ing nor archiving of journals was im- TABLE 7 portant in an environment of e-jour- Of those supporting the development ofnal publishing. Another three respon­ electronic journals, an electronicdents (2%) were equally uncertain that paper was not required some- journal, to be successful, also must be where in the publishing process, ei- published in paper format (N = 129). ther in the format of the journal or as Agreed Disagreed Uncertain Mediana device for archiving. Academic communication takes 66.7% 6.7% 26.7% Agree two forms: the presentation of papers [50] [5] [20] and discussion of research results, and the publication of the same in pro­ fessional journals.13 The delivering of pa- for informal communication with mem­ pers, the distribution of preprints, and the bers and the intellectual leaders of the dis- face-to-face discussion of research results cipline, while excluding nonmembers that occur at professional meetings define from the dialogue. the informal communication system.14,15 The surveyed gatekeepers served on The informal communication system pro- the editorial boards of the most presti­ vides the researcher great freedom in the gious journals within their disciplines; selection of both the media and the audi- thus, it would seem unlikely that these ence addressed. It allows all work that scholars would choose to publish the re- reaches even minimum scientific stan- sults of their research in anything other dards to be disseminated, even though than a journal with a reputation for the that work may be unfinished or its claims highest quality. Yet, when asked whether unsubstantiated. The information con- they would publish in an e-journal, only veyed is commonly abstracted, usually 19.7 percent of the gatekeeper respon­ colloquial, frequently incomplete, and dents believed they would not do so at often vague. The results may even turn some point in their careers (see table 9). out to be false. Equally important, the in- Because it is counterintuitive to believe formation is often archived only tempo- that gatekeepers would publish research rarily and, ultimately, is difficult to re- results in journals that were other than of trieve. Indeed, an author is likely to have the highest quality or reputation, the data disseminated his or her manuscript support the supposition that gatekeepers through the informal communication sys- did not consider e-journals as replace- tem prior to submitting it to a journal for ments for the paper-formatted publica- publication for the purpose of obtaining tions that make up the formal communi­ feedback on his or her findings and in- cation mechanism of the disciplines. terpretations from relatively friendly au- However, it is perfectly conceivable that diences.16 Meetings organized by profes- gatekeepers would continue to explore all sional associations are excellent forums aspects of the informal communication mechanisms of the discipline to test research results. That could explain TABLE 8 why so many gatekeepers agreed, or at least were uncertain as to their will-Of those not supporting the ingness, to publish in an e-journal thatdevelopment of electronic journals, would be of the informal communi­an electronic journal, to be successful, cation mechanism of the disciplinesalso must be published in paper format and why only forty-one respondents (N = 129). (32.3%) wished to see their own jour-Agreed Disagreed Uncertain Median nal available in electronic format. 88.9% 5.6% 5.6% Agree Gatekeepers also were asked whether they thought e-journals were http:diences.16 http:journals.13 Editorial Gatekeepers Confronted by the Electronic Journal 153 a publishing fad (see table 10). When TABLE 9 responses to this statement were cor- At some point in my professional career, related with the responses to the I will publish in an electronic journalstatement asking whether the (n = 127).gatekeeper supported e-journal de­ velopment, a Kendall’s tau value of Agreed Disagreed Uncertain Median .526 was produced, indicating a strong correlation between the two 38.6% 19.7% 41.7% Uncertain statements. In reviewing the correla- [49] tion, those who supported e-journal development strongly disagreed that e- journal publishing was a fad and those who disagreed with e-journal develop­ ment strongly agreed that it was a fad (38.9%), although responses also were dis­ tributed in other categories (33.3% were uncertain and 27.8% disagreed). Again, those who were uncertain of their sup­ port for the development of e-journals were just as uncertain about whether e- journals were a fad (57.1%, compared to 5.7% who believed they were a fad). Conclusion It has been maintained that editors and referees of academic journals perform a gatekeeper function for their disciplines and their professions. The mechanism they use for making “in” or “out” deci­ sions is publication in refereed academic or scholarly journals. It would appear im­ portant to understand the attitudes that gatekeepers have toward the device that is their mechanism of control because at­ titudes may precipitate or account for TABLE 10 The electronic journal is just another publishing fad (N = 128). [25] [53] some future action these stakeholders ini­ tiate. In this case, it would appear signifi­ cant to understand editorial gatekeeper attitudes toward the paper format of the journal and toward proposed replace­ ments to that medium in order to better overcome possible objections by gatekeepers to attempts at change. This survey showed that editorial gatekeepers, although generally support­ ive of the creation of e-journals, are still very enamored of the paper format. One conclusion supported by the data is that many of the respondents see the e-jour­ nal as supportive of the informal commu­ nication processes of the discipline and not as a replacement for the paper-format­ ted journal, which is the backbone of the formal communication processes of the disciplines. Change agents who are aware of gatekeeper attitudes may wish to affirm, when experimenting with the establish­ ment of e-journals as devices for sup­ planting the paper-formatted journal, that their experiments are directed at the formal communication mechanisms of the discipline. Such an affirmation will force gatekeepers to view alter­ native mediums as affecting the so- Agreed Disagreed Uncertain Median cial processes they control and, as a result, to begin to address the real 7.8% [10] 64.1% [82] 28.1% [36] Disagree hurdles to be overcome in success­ fully implementing an alternate for­ mat for the paper journal. Notes 1. Talcott Parsons and Gerald M. Platt, The American University (Cambridge, Mass.: Harvard Univ. Pr., 1973), 112. 2. Lewis A. Coser, “Publishers As Gatekeepers of Ideas,” Annals of the American Academy of Political and Social Sciences 421 (Sept. 1975): 14–31. 3. Kurt Lewin, “Psychological Ecology,” in Field Theory in Social Science: Selected Theoretical 154 College & Research Libraries March 2000 Papers by Kurt Lewin, ed. D. Cartwright (New York: Harper & Brothers, 1951), 170–87. 4. Richard D. Whitley, “The Formal Communication System of Science. A Study of the Organisation of British Social Science Journals,” in The Sociology of Sociology, ed. P. Halmos (Keele, Staffordshire: Univ. of Keele, Sociological Review Monograph No. 16, 1970), 163–79. 5. Robert J. Silverman and Erik L Collins, The “Gatekeeper” Role in Educational Journal Publish­ ing. Final Report (Washington, D.C.: National Institute of Education, 1975). ERIC Document Re­ production Service No.ED 111 427. 6. Ronald F. Dow, “Gatekeeper Attitudes toward Supplanting Paper Journals with Electronic Alternatives” (Ph.D. diss, Pennsylvania State Univ., 1998). 7. Harriett Zuckerman and Robert K. Merton, “Patterns of Evaluation in Science: Institu­ tionalization, Structure and Functions of the Referee System,” Minerva 9 (Jan. 1971): 66–100. 8. Abraham N. Oppenheim, Questionnaire Design, Interviewing and Attitude Measurement (New York: Pinter Publishers, 1992), 174–95. 9. Anthony Biglan, “The Characteristics of Subject Matter in Different Academic Areas,” Journal of Applied Psychology 57 (June 1973): 195–203. 10. Oppenheim, Questionnaire Design, Interviewing and Attitude Measurement. 11. Alan Agresti, Statistical Methods for the Social Sciences (San Francisco: Dellen Publishing, 1986), 219–20. 12. Alan Bryman and Duncan Cramer, Quantitative Data Analysis for Social Scientists (New York: Routledge, 1990), 173. 13. Talcott and Platt, The American University, 12. 14. William D. Garvey and Belver C. Griffith, “Scientific Communication: Its Role in the Con­ duct of Research and Creation of Knowledge,” American Psychologist 26 (Apr. 1971): 359–60. 15. William D. Garvey, Communication: The Essence of Science (New York: Pergamon Pr., 1979), 154. 16. Garvey and Griffith, “Scientific Communication,” 358.