College and Research Libraries Research Notes An Investigation of Collection Support for Doctoral Research Marion L. Buzzard and Doris E. New A major concern of bibliographers and other academic librarians involved in de- veloping the collections of research li- braries is, or should be, the amount and quality of support that is provided for the graduate programs of their institutions. In carrying out their assignments, these librarians tend to rely chiefly on contact with the faculty for information. Since it is the faculty who shapes the curriculum for both undergraduate and graduate stu- dents and who also carries out most of the postdoctoral research, this seems a rea- sonable approach. It may, however, be less responsive to the needs of graduate research. ·some doctoral research is re- lated to the research interests of faculty advisers, but it would be unwise to as- sume the existence of such a connection without substantiation. And although there are faculty members who are consci- entious in articulating the needs of their graduate students to librarians, as well as bibliographers who actively seek out these users, the latter are seldom identified as an essential contact point for selectors of library materials. It is possible, therefore, for a collection to support instructional and faculty research programs to a greater extent than the graduate programs that may strongly influence its scope and fund- ing. At the University of California, Irvine (UCI), collection development generally follows the traditional method of faculty contact to obtain most of the information used in making selection decisions. An additional source of information is pro- vided through the use of interlibrary loan requests, which are forwarded to selectors after each transaction is completed. In or- der to evaluate the effectiveness of this ap- proach, a study was devised to determine the actual level of support for doctoral re- search. A random sample of twelve disser- tations completed at UCI in the humani- ties, sciences, and social sciences was selected for the study. The citations from these dissertations were divided into monographs and serials and were checked against the library's holdings. The results were expected to indicate where greater effort should be made to as- certain the needs of doctoral candidates at UCI and to support these needs with are- allocation of resources, if necessary. BACKGROUND Since the sixties, numerous attempts have been made to describe the relation- ship between the various types of aca- demic programs and library resources in order to arrive at an understanding of what constitutes optimum or at least ade- Marion L. Buzzard is head, Collection Development and Acquisitions, and Doris E. New is head, Serials De- partment, both at the University of California, Irvine. 469 470 College & Research Libraries quate library support. 1 Although these studies utilize both qualitative and quanti- tative methods of assessment, they have generally accepted the number of volumes owned as the most important indicator of support. This tendency was further codi- fied by the development of the Clapp- Jordan formula, which has had a lasting effect on standards for college and univer- sity libraries and the way in which re- search collections are funded. 2 Like Clapp-Jordan, the revised stan- dards for college libraries provided a for- mula for collection size that was weighted very heavily in favor of doctoral pro- grams.3 Several years later the newly drafted standards for university libraries made a conscious attempt to deemphasize quantitative approaches but concluded that ''nevertheless, quantitative measures are increasingly important in guiding the qualitative judgment that must ultimately be applied to university libraries and their collections. ''4 · Whatever their faults, quantitative ap- proaches have given libraries a convenient means of determining funding for their collections. As a case in point, since 1979 the University of California libraries have had their acquisitions funds allocated on the basis of a formula that bears more than a passing resemblance to its predeces- sors.5 Like them it awards volumes (which are then translated into dollars) largely ac- cording to the number and type of doc- toral and graduate professional programs offered by each of the nine campuses. Be- cause of the emphasis placed on these pro- grams, it is highly desirable to be able to determine the support actually provided for doctoral research at UCI. RELATED RESEARCH Relatively few studies to date have fo- cused on graduate students as a separate user group, and in almost all cases these have been limited to candidates for the master's degree and to a single discipline or field. A simple and fairly straightfor- ward means of measuring the existing level of collection support for doctoral re- search is available by examining dissertations-the products of the re- search effort-to learn whether the refer- ences cited are held by the library. Citation November 1983 analysis for this purpose, although recom- mended in library literature, has seldom been applied. A study by Popovich of thirty-one dissertations in the field of busi- ness/management had as a secondary as- pect the determination of the extent to which the library owned the materials cited. However, less than half of the dis- sertations examined had been completed at that institution, thus precluding any di- rect correlation between the results ob- tained and the success of the collection in meeting the needs of its own doctoral stu- dents.6 Citation analysis has long been used as a means of examining the characteristics of the literature cited by scientists and other scholars, and of singling out the most im- portant journals in any particular field, usually the sciences. Peat proposes it as an acceptable technique for measuring re- search use of a research library, to comple- ment circulation studies. 7 Other writers, such as Baughman and Voos, consider ci- tation analysis a viable means, and per- haps the best objective approach, to col- lection evaluation. 8 As with any quantitative measure, cita- tion analysis must be applied with a recog- nition of its limitations; these have been well documented by Smith. 9 The greatest weaknesses applicable to the study here described are (1) the assumption that all works cited were actually used by the re- searcher, and (2) the likelihood that there may have been a tendency to cite works that were accessible and to omit those that were not. HYPOTHESIS AND METHODOLOGY User studies published in the literature indicate that, on an average and regard- less of size, libraries have acquired 90 per- cent of the materials requested by users. 10 Without attempting to argue that this per- centage constitutes either adequate or op- timum support, we would nevertheless expect that doctoral students, as a discrete group, should have a success rate similar to that of other users. The hypothesis tested in this study is that the UCI Library collection includes 90 percent of the mate- rials required by UCI doctoral candidates for their research. A random sample was selected from all dissertations completed at UCI during the last two quarters of the previous academic year. These were the most recent disserta- tions available for the study and provided a sufficiently large population from which to draw the sample. References cited in the selected dissertations were checked against the library's holdings. An as- sumption was made that the collection would not have changed significantly in the relatively short period since these par- ticular students had completed their re- search. The 36 dissertations from which the sample was drawn were divided into three broad fields of knowledge: the sci- ences, social sciences, and humanities. Within each group the dissertations were listed alphabetically by author. A system- atic random sample was obtained by se- lecting every third dissertation listed, since the population was less than 100. This yielded 12 for the study, with a total of 1,384 citations. Of the 12 dissertations, 3 fell into the humanities, 5 into the sci- ences, and 4 belonged in the social sci- ences. The citations appeared either in the form of a bibliography, where each title was cited one time only, or as a list of refer- ences, in which some titles were repeated one or more times. For each dissertation the citations were divided into mono- graphs, serials, or "other." Monographic titles, which included monographic se- ries, were counted only the first time cited. Some. citations were for editions other than those owned by UCI. Because the use of variant editions by the research- ers could not be readily understood, a de- cision was made to count as held only those titles where the library's edition was the one cited or a later printing of the one cited, or else a simultaneous edition. Se- rial titles were determined according to the AACR2 definition, except for mono- graphic series, and included journals, pro- ceedings, annuals, and other materials is- sued periodically. These were considered to be duplicates and disregarded when the same volume for a particular title was cited more than once; when a different volume of that title was listed, it was counted as a separate citation, since many journals at Research Notes 471 UCI have interrupted rtins, with missing volumes. The ''other'' category included a variety of unpublished materials, such as manu- scripts, technical reports, and papers de- livered orally or submitted for publication to unspecified journals. Titles falling into this group were ignored in the study be- cause the library would not normally ex- pect to collect this type of material; how- ever, all dissertations cited were treated as monographs. Only 33 "other" citations were found, amounting to less than 3 per- cent of the total. Of the remainder, 207 were found to be duplicate citations, leav- ing 1,144 monographs and serials to be checked against the library's holdings. Table 1 shows the distribution of cita- tions broken down by field and by type of publication. The average number of cita- tions per dissertation is 95.3, while the range extends from 11 to 162. Each citation was checked against the card catalog and the order file, or the seri- als records. Those not located in the collec- tion were reviewed for potential prob- lems, such as works incorrectly cited, and where necessary, were rechecked. Fi- nally, all of the data were reviewed for consistency and accuracy. RESULTS AND ANALYSIS The data obtained in the study were considered in two different ways. Table 2 treats all citations as equal and classifies them by field and by type of publication. It shows the percentage of monographic or serial citations held by the library in each of the three broad fields of knowledge. The table also shows the results for all monographs and all serials, as well as for each field irrespective of the type of publi- cation. The percentage of successes for the entire list of citations checked is 90.4, as shown in the ''Complete Survey'' line. The standard deviation has been calcu- lated for the sample sizes and observed percentages, assuming a binomial distri- bution. The error ratio column shows the difference between the observed percent- age and the hypothesized 90 percent di- vided by the standard deviation of the sample. If the ratio is less than ±1.96, then there is no statistically significant differ- ence at the .05 probability level. In two 472 College & Research Libraries November 1983 TABLE 1 DISTRIBUTION OF CITATIONS BY FIELD AND TYPE OF PUBLICATION Field ~~bllc~tion Humanities Sciences Social Sciences Totals No. % No. % No. % No. % Monographs 249 91 53 14 130 27 432 38 Serials 26 9 337 86 349 73 712 62 Totals 275 100 390 100 479 100 1,144 100 TABLE2 ANALYSIS OF CITATIONS BY FIELD AND TYPE OF PUBLICATION No. of GrouE Citations Monographs only 432 Serials orily 712 Humanities 275 Sciences 390 Social sciences 479 ComElete survey 1,144 cases a significant ratio was obtained, as indicated by the asterisks. Before analyzing these results, an alter- nate approach was considered. A poten- tially more useful way of looking at the results is to examine the mean or average of the percentage for students in each group, instead of the percentage of all cita- tions in each group. This second approach is oriented toward the user and reflects the effectiveness of the library in meeting the needs of the individual student. Table 3 shows the mean of the percent- ages of references that students in each group found in the library by type of pub- lication and by field. It is clear that the data differ from that in table 2. These differ- ences occur because each dissertation does not have the same number of cita- tions. In those cases where only a few ref- erences were cited, each of these has a Standard Error %Found Deviation Ratio 88.2 1.55 1.163 91.7 1.03 1.658 92.0 1.64 1.223 95.1 1.09 4.704* 85.6 1.60 -2.746* 90.4 .87 .441 greater weight in establishing the mean probability that the individual student's citations will or will not be found in the li- brary than in the case of dissertations with a large number. of citations. The data in table 3 were analyzed to de- termine if there is any statistical basis for rejecting the 90 percent support hypothe- sis based on the experience of the individ- ual students who formed the various sam- ple groups. For each group the mean and standard deviations of the sample per- centages were calculated, with the results shown in table 3. Because of the small number of cases in each set of samples, at- test was used to determine if there was a statistically significant difference from the hypothesized 90 percent support level. A .05 or less probability level was selected as significant. The t-test score and .05 t value are also shown in table 3. In only one case TABLE 3 GrouE Monographs only Serials only Humanities Sciences Social sciences ComElete survey ANALYSIS OF DISSERTATIONS BY FIELD AND TYPE OF PUBLICATION %of No . of Citations Standard Dissertations Found Deviation 12 80.6 27.5 12 90.2 7.3 3 91.6 5 .3 5 93.7 3 .2 4 85.9 8.3 12 90.6 6.4 .05 t t Score Value 1.185 2.20 .100 2.20 .427 2.57 2.352* 2.26 .849 2.44 .306 2.20 was a significant deviation observed, as indicated by the asterisk. In two cases in table 2 (sciences and so- cial sciences) and one in table 3 (sciences), the observed percentages lie outside the range of expected values. Since the rest of the results confirm the hypothesis of 90 percent support, the possibility of other factors not considered by this study should be examined to determine why the observed support level for these two areas deviates from the expected norm, one above and the other below. These differ- ences can be explained in two ways: first, as due to the sample size and the expected random distribution of results for the given size of sample; or second, because the actual support level for particular groups of users (such as graduate stu- dents in a given field) differs from the 90 percent hypothesized for the collection as a whole. The data for each dissertation are given in table 4. It shows that the 90 percent level is exceeded by all five of the dissertations in the sciences and by two out of three in the humanities, while three out of four in the social sciences fall below that level. The average number of citations found fell within ±7 percentage points of 90 percent except for dissertation no. 10 in the social sciences. In this exceptional case, the dif- ference of 14 percent suggests the exis- tence of special factors. A review of the dissertation indicates that it relied heavily on law journals for its references. Since UCI has no law school, these materials are collected on a very selective basis; thus many of the journal titles or specific vol- umes of these titles that were cited were not held by the library. Possible causes for the deviation in the science dissertations might be either the selection procedures for these areas, faculty influence in these- lection of dissertation topics, or other fac- tors not identified in this study. CONCLUSIONS The hypothesis that the UCI Library col- lection includes 90 percent of the materials required by UCI doctoral candidates for their research is accepted when the entire list of citations is considered. However, when the data are subdivided by field Research Notes 473 ]..o] \ONOO\Ot-..('1")0\0~~('f") oo-- ::l \CiN\Ci\Cic)\C).q;c)oc)\C)\C)C1') E-< J:: 0\0\000\0\0\0\0\00t-..O\OO '0.{§ OII)('f")O\t-..0~0\0\NII)'-0 '*-Jl& g .fx3~~~~~~~~~~ 474 College & Research Libraries and/or type of publication, the hypothesis can be accepted only if all the cases pass the statistical test used. Since in three in- stances they failed the test, the hypothesis must be rejected unless special circum- stances can be shown to exist. The one case in the social sciences represents an abnormal situation, where the disserta- tion involved relied heavily on a type of material collected very selectively at UCI due to the lack of a strong program in that particular field. Moreover, the test failure was true only for the analysis of the cita- tions (table 2) and not when the disserta- tions were considered (table 3). The other two instances of statistical test failure occurred for the sciences, both in the analysis of citations and of disserta- tions. The conclusion seems to be that the hypothesis does not hold for the sciences but is accepted for the humanities and so- cial sciences. Since the sample mean for the sciences is above 90 percent, a further conclusion to be drawn is that the sciences receive a higher level of support than the other fields . This finding could be used as Heritage on Microfilnt Rare and out-of-print titles and documents on 3Smm silver halide microfilm. • French Books before 1601 • Scandinavian Culture • 18th Century English Literature • Victorian Fiction • Literature of Folklore • Hispanic Culture Send for catalog and title information today . ~~r~~[M ~ COV\PA'-JY 70 Coolidge Hill Road Watertown, MA 02172 (617) 926-5557 November 1983 a factor in determining future allocation of library materials funding. The data for each dissertation (table 4) showing that three of the four samples in the social sciences fall below the 90 per- cent level suggests that it might be useful to examine all of the social sciences disser- tations in the original population of thirty- six to determine whether this pattern is typical. If it is, an effort could then be made to strengthen collection support in this particular field by increasing contacts with graduate students or by other appro- priate means. . . As one would expect, the humanities dissertations rely mostly on monographic references (91 percent), and the vast ma- jority of the citations in the science disser- tations are from journals (86 percent). The social sciences lie somewhere between those two fields, but with a much greater reliance on serials (73 percent) than on monographs. The data indicate that the level of support for serials at UCI is higher than for monographic materials. Al- though the difference does not seem par- ticularly significant in the analysis of the citations, the analysis of dissertations shows a difference nf 10 percent. The higher level for serials could be explained in a number of ways, including the fact that the universe of monographic titles is much larger and consequently more diffi- cult to identify and to acquire, especially when retrospective materials are consid- ered. It appears that citation analysis can pro- vide a valuable tool for evaluating collec- tion support for doctoral research and that it can be applied to all fields in measuring research use of a library. Future studies of this type might include publication dates of materials cited when data are gathered. An analysis of the age of the publications cited in each field or in specific disciplines could provide helpful information for col- lection development decisions and could be useful as a predictor for establishing cri- teria for deselection in each field or disci- pline. It might also be interesting to devise a study that would compare the level of collection support for doctoral research with the level provided for faculty re- search, since selection information is gen- erally obtained from the faculty. The findings of the UCI study tend to confirm other user studies indicating that libraries, on an average, acquire 90 per- cent of the materials requested by users. Research Notes 475 Had the hypothesis been worded to state that the collection includes at least 90 per- cent of these materials, the correlation would have been even stronger. REFERENCES 1. Sam E. Ifidon, "Qualitative/Quantitative Evaluation of Academic Library Collections," Interna- tional Library Review 8:299-308 (1976) . 2. Vernon W. Clapp and Robert T. Jordan, "Quantitative Criteria for Adequacy of Academic Library Collections," College & Research Libraries 26:371-80 (Sept. 1965). 3. "Standards for College Libraries," College & Research Libraries News 36:277- 79 (Oct. 1975). 4. "Standards for University Libraries," College & Research Libraries News 40:101-10 (Apr . 1979). 5. Known as the Voigt-Susskind formula and published in The University of California Libraries: A Plan for Development (Berkeley, Calif.: University of California, Office of the Executive Director of Uni- versitywide Library Planning, 1977), p .118-22. 6. Charles J. Popovich, ''The Characteristics of a Collection for Research in Business/Management,'' College & Research Libraries, 39:110-17 (Mar. 1978). 7. W. Leslie Peat, "The Use of Research Libraries: A Comment about the Pittsburgh Study and Its Critics," Journal of Academic Librarianship 7:229-31 (Sept. 1981). 8. James C. Baughman, "The Library Ecology Problem" inK. Nyren, ed., Library Space Planning (LJ Special Report, no .1 [New York: Bowker, 1976]), p.46; HenryVoos, "Collection Evaluation," Col- lection Building 3, no.1:6-11 (1981). · 9. Linda C. Smith, "Citation Analysis," Library Trends 30:83-106 (Summer 1981). 10. T. Saracevic, W. M. Shaw, Jr., andP. B. Kantor, ''Causes and Dynamics of User Frustration in an Academic Library," College & Research Libraries 38:7-18 Qan. 1977); Ben-Ami Lipetz, "Catalog Use in a Large Research Library," Library Quarterly 42:129-39 Qan. 1972); Daniel Gore, "Curbing the Growth of Academic Libraries," Library Joumal106:2183-87 (Nov. 15, 1981). 1~ y f A . ·t· L.b . ~ ~ cqutst ton 1 rartans ••••• t I I I j I I i I Now is the time to try the best source for: • any book in print • accurate invoicing • meaningful reports • 1 • rush order service I I,· • competitive discounts • PLUS MANY OTHER SERVICES ' 1-800-248-.1146 OTHER SERVICES .. .include binding of paperbacks upon request, returning slips in books, standing orders, staff orders, and up to date information on open orders. the BOOK HOlJSE I f CALLTOLL-FREETODAY I J08BERS SERVING LIBRARIES WITH I 1 : In Canada & Michigan :c:v.,:E~Iic~~c":~~T5~~~~~ 1M2 I ~I(" CALL COLLECf (517) 849-2117 ~'"'-'-'- "'C"'G" .,., SAN 169-3~~