College and Research Libraries Material Availability: A Study of Academic Library Performance Anne C. Ciliberti, Mary F. Casserly, Judith L. Hegg, and Eugene S. Mitchell This article reports the findings of a study modeled after Saracevic, Shaw, and Kantor's efforts to identify and quantify the causes of users' failures to identify and locate library materials. The researchers analyzed patron-reported and librarian-observed subject and known-item searches and found an overall success rate of only 54 percent. The problems that led to the 46 percent failure rate were analyzed by source and type of failure, and subjective observations concerning problems encountered by patrons were recorded. Recommendations are made for reducing li- brary malfunctions and circulation, patron, and acquisition errors. his article reports the findings of an empirical self-study un- dertaken at the William Pater- son College Library during the fall semester, 1985. The college is a state- supported New Jersey institution award- ing baccalaureate and master's degrees. It enrolls 7,000 full-time students and em- ploys a teaching faculty of 350 full-time professors. The library, with a profes- sional staff of 20, contains approximately 300,000 items of print and nonprint mate- rial. The primary purpose of the study was to determine what needed to be done to im- prove library services. Several important ancillary benefits were anticipated; these included involving staff, particularly those new to the organization, in aspects of the library (and perhaps the college) that were unfamiliar to them and intro- clueing them to the techniques and com- plexities of evaluating library operations. The potential for a positive political im- pact, particularly in terms of funding, was also noted. A steering committee com- prised of four staff members planned the study, analyzed the data, and prepared the following report. All staff, however, were involved in various aspects of the ac- tual data collection. Four criteria were used to select a type of self-study that would (1) identify the im- pact of library weaknesses on users, (2) evaluate functions used by patrons, (3) be feasible, and (4) serve as a management tool, not as an academic exercise. The selection of a self-study model fol- lowed an intensive review of the advan- tages and disadvantages associated with various library research methodologies. User surveys, document availability tests, Anne C. Ciliberti and Eugene S. Mitchell are Associate Directors, and Judith L. Hegg is Head of Collection Development at the Sarah Byrd Askew Library, William Paterson College, Wayne, New Jersey 07470. Mary S. Casserly is Head of Collection Development at Fogler Library, University of Maine, Orono, Maine 04469. 513 514 College & Research Libraries and catalog use studies were all examined and judged against the four selection crite- ria. From this review it became clear that a shelf availability study developed by Paul Kantor and described in an article by Sara- cevic, Shaw, and Kantor offered the most advantages. 1 This instrument provides a measure of performance for a library's acquisitions program, circulation policies, internal op- erations, and users' capabilities. A branching analysis, used to calculate probabilities, requires that the outcome of each sequential step in the search process be placed into one of several independent categories representing the obstacles to a successful search that must be overcome. In known-item searches, for example, the Kantor model suggests four steps: 1. Has the library acquired the desired title? 2. If acquired, is it in circulation? 3. If not in circulation, is it available on the shelf? 4. If available on the shelf, can the user retrieve it successfully? According to the branching technique, the proportion of searches that overcomes November 1987 each of these obstacles represents the probability of success for a category. When multiplied together, these individ- ual success probabilities determine the overall probability of availability. The Kantor evaluation model was modi- fied by Ciliberti for use at William Pater- son College. 2 The principal modifications were an expansion of the steps or branches involved in known-item searches and the addition of a parallel se- ries of branches involved in the successful completion of subject searches. These branches, represented in figures 1 and 2, are defined below. ACQUISITION ERROR Acquisition errors occur only in known- item searches when the desired material is not a part of the library's collection or is not fully represented in the card catalog. APPROPRIATE TITLE ERROR Appropriate title errors occur only in subject searches when patrons fail to se- lect call numbers for titles found in the cat- alog or when, after examination of se- lected titles, patrons fail to borrow (or use FIGURE 1 Branching Analysis of Known-Item Searches Material Availability 515 Success FIGURE2 Branching Analysis for Subject Searches in the library) materials found on the shelf. These errors occur when patrons choose not to consult items found on their topics because the material has already been read, is written in the wrong lan- guage, is too old or too new, is not at the correct reading level, or is in some other way unsuitable to the information need at hand. All such decisions and judgments are made by the patron; the researcher can only infer from patron actions and note that particular titles were in some way deemed inappropriate. BIBLIOGRAPHIC ERROR Bibliographic errors occur only in known-item searches when the desired material is not found by the patron be- cause the bibliographic clue or citation ( ei- ther remembered or written) is incorrect and the document can be verified from an- other source and is correctly represented in the card catalog. CATALOG USE ERROR Catalog use errors occur in either known-item or subject searches when the desired material is not found by the patron as a result of one of the following situa- tions: 1. No call number was identified, and the book had been acquired. 2. An incorrect or incomplete call num- ber was identified. 3. Special location symbols, such as Fo-. lio or Ref., printed adjacent to the call number, were not noted. CIRCULATION ERROR Circulation errors occur in either known-item or subject searches when the desired material cannot be found by the patron for one of the following reasons: 1. Item is located on a "hold" shelf waiting to be charged out. 2. Item has been borrowed for use out- side the library and record of the loan transaction is available. LIBRARY MALFUNCTION ERROR Library malfunction errors occur in ei- ther known-item or subject searches when the desired material cannot be found by the patron due to shortcomings in the pol- icies or routines of the library or its staff. 516 College & Research Libraries Shortcomings occur when items are (1) missing; (2) misshelved; (3) located on sorting shelves; (4) waiting to be shelved; (5) being recataloged, reprocessed, or re- paired. MATCHED QUERY ERROR Errors in matching query terms occur only in subject searches when patrons fail to discover a subject heading in the card catalog that partially or fully matches their query terms. Matching failures may be of two kinds, however. Type A errors occur when no match can be made from the ini- tial query to a Library of Congress subject heading and, therefore, represent patron errors: failing to find the appropriate Li- brary of Congress subject heading. Type B errors occur when no match can be made from the initial query term to the appropriate Library of Congress subject heading because the library does not own books on that subject; such errors, there- fore, represent library acquisition failures. RETRIEVAL ERROR Retrieval errors occur in either known- item or subject searches when the desired material cannot be found by the patron de- spite the fact that the correct and complete call number has been noted and the book is in its proper shelf location. RELATED LITERATURE The historical antecedents of shelf avail- ability research are diverse. The begin- nings of performance measurement re- search have been traced to the 1930s by Ciliberti. 3 Mansbridge4 also cites an exam- ple of availability research from 1934. It was not until the 1960s and 1970s, how- ever, that strong interest in performance measurement began. During those years many seminal investigations were under- taken, including work by Meier, 5 Rzasa and Baker, 6 and Hamburg, Ramist, and Bommer. 7 Whereas the early studies often endeav- ored to assess library service in its broad- est sense, later research was aimed at eval- uating intralibrary document delivery. Within this subfield of study two strains of empirical investigations developed: docu- November 1987 ''In contrast to document delivery tests, shelf availability studies mea- sure the degree of accessibility for ti- tles actually sought by library pa- trons." ment delivery tests and shelf availability research. The works of DeProspo, Altman, and Beasley8 and of Orr and others9 are repre- sentative of document delivery tests. In these studies availability was typically measured by determining the degree of availability for items listed in a bibliogra- phy. These citations were obtained through a variety of methods. In the De- Prospo study, for example, the citations were drawn randomly from editions of American Book Publishing Record, while they were culled from a broad range of re- cently published biomedical literature in the Orr study. In contrast to document delivery tests, shelf availability studies measure the de- gree of accessibility for titles actually sought by library patrons. In this manner, such variables as the competition for high- demand titles are viewed realistically, rather than in the artificial structure of document delivery testing. The research presented here is a true shelf availability study and follows the ba- sic methodology first proposed by Kan- tor10 and by Saracevic, 1 as described later in the article. As such, this study comple- ments an impressive group of studies in which the Kantor design was used; this group includes a longitudinal investiga- tion conducted at Case Western Reserve University. 12 Several other examples in- clude work by Whitlatch and Kieffer, 13 Wulff, 14 Smith and Granade, 15 Palais, 16 Kochtanek, 17 Radford, 18 Ciliberti, 19 and Ferland Robinson. 20 It is important to note that the research reported here differs from all of those studies except the Cili- berti work, 21 in that it investigates avail- ability rates for subject as well as known- item searches. METHODOLOGY Background During the summer of 1984 the Steering Committee discussed how and when li- brary users would be surveyed for self- study purposes. Because it was to be based on the outcome of card catalog searches, it was agreed that the self-study would rely on data obtained from a ran- domly selected group of catalog users dis- tributed throughout the day and week in the same proportion as all users of the card catalog. Towards this end, a preliminary study of card catalog use was planned and im- plemented throughout the fall semester. During each weekday hour, library staff observed and recorded each use of the card catalog in half-hour intervals; week- end observations were not economically feasible. Decisions on the sample size, variables to be observed, and methods of observa- tion were made by the Steering Commit- tee during the summer of 1985. It chose to follow the cell-size method developed by Gal tung for calculating sample sizes. 22 A sample size of 600 observations was used, and half-hour periods by day of week and week of semester were selected randomly. A second issue addressed by the com- mittee pertained to the methods of obser- vation to be used for collecting data from the 600 card catalog users. Previous re- search relied mainly on patron self-reports for data collection. In recent research, Cili- berti23 studied library performance on the basis of data collected through both pa- tron self-report and librarian observation. After reviewing the Ciliberti study, which found that self-reported data showed sig- nificantly higher levels of library success than data collected by librarian observa- tion, the committee elected to use both methods. Specifically, it agreed to collect data from 600 randomly selected users, by distributing survey forms on which they could record their search successes and failures (self-reported measurement), and from 40 randomly selected users by di- rectly observing their search efforts and recording their successes and failures ( ob- Material Availability 517 served measurement). The smaller set of librarian observations would serve as a check on the accuracy of the patron self-reports. At the conclusion of the study, the measures derived from the two samples would be compared to determine if the findings differed. Patron self-report forms were distrib- uteddailybetween8a.m. and 10p.m., ex- cluding weekends. Written scripts and di- rections to the staff on how to survey patrons were distributed and reviewed at a staff meeting. Distribution schedules were also generated and sent to staff throughout the semester. Data collectors were instructed to request cooperation from the first patron they observed ap- proaching the card catalog during their as- signed half-hour interval. Patrons agree- ing to participate were given work sheets upon which to record the titles and/or call numbers of the materials they desired and were asked to return them as they exited. If the first person declined to cooperate, the data collector approached another cat- alog user. The librarian observations were made by committee members, who accompa- nied participating patrons during their consultation of the card catalog and subse- quent search of the book stacks. DATA COLLECTION AND RETURN RATES Data collection began the first day of the fall1985 semester and continued through- out it. Rates of distribution and return were closely monitored by the committee, and steps were taken to insure that these rates remained acceptable. In addition to conducting the librarian observations of patron searches, the com- mittee was also responsible for analyzing the self-reported data. Each day, members of the committee collected the self-reports that had been returned in the preceding twenty-four hours and randomly selected one title for analysis. If this title had not been found or used, the cause of the fail- ure was determined. This required verify- ing the call number, subject heading, or bibliographic reference; checking the card catalog; searching the book stacks and 518 College & Research Libraries 11 • • • some patrons who had indi- cated that they were conducting known-item searches had in fact con- ducted subject searches and vice versa." sorting shelves; and examining the circu- lation files and reserve book shelves. Thirty-four observations were com- pleted, and 401 self-reports were received. The intention to observe forty library pa- trons was not met, either because of re- searcher error (the data collector missed the collection time) or because no patrons willing to participate in the study ap- proached the card catalog during the ap- pointed half-hour interval. Likewise, the committee intended to gather self-reports from 600 patrons, but only 560 were dis- tributed for the same reasons. Of the forms distributed, however, 401 were re- turned and usable, an overall return rate of 72 percent. RESULTS Types of Searches Conducted and Academic Status of Catalog Users The data indicate that 53 percent of the patrons who completed self-reports con- ducted known-item searches and 47 per- cent, subject searches. The breakdown of librarian-observed, known-item, and sub- ject searches was slightly different, but a chi-square test indicated that this differ- ence is not significant at the . 95 level of confidence. The process of determining which were November 1987 subject and which known-item searches was not as straightforward as had been anticipated. Despite the fact that patrons were asked whether they were looking for materials by author, title, or subject and were given the appropriate form, it was apparent from examining these forms that some patrons who had indicated that they were conducting known-item searches had in fact conducted subject searches and vice versa. As a result, six unsuccessful searches recorded on subject forms were counted as known-item searches, and five unsuccessful searches recorded on known-item forms were counted as sub- ject searches. As would be expected at a predomi- nantly undergraduate institution, the ma- jority of the participants were undergrad- uates. This group conducted 69 percent of the total searches observed and 66 percent and 73 percent, respectively, of all known- item and subject searches (see table 1). Graduate students, the second largest group of subjects, conducted 15 percent of the total searches. While undergraduates carried out an equal number of known- item and subject searches, graduate stu- dents conducted significantly more known-item than subject searches. These findings sup:gort Palmer24 and Tagliacozzo . and Koch en, who found that the propen- sity for conducting known-item searches increases with educational level. It should also be observed that the faculty consti- tuted the smallest category of users (3 per- cent). Analysis of Success Rates The overall success rate for the 401 self- reported searches was 54 percent-215 pa- TABLE 1 ANALYSIS OF TYPE OF SEARCH BY ACADEMIC STATUS Academic Status Undergraduate Graduate Faculty Other No Answer Total Known-Item 139 ( 66%) 40 ( 19%) 8 ( 4%) 15 ( 7%) 9 ( 4%) 211 (100%) Note: Due to rounding, columns do not total 100%. Type of Search Subject 139 ( 73%) 22 ( 12%) 6 ( 3%) 14 ( 7%) 9 ( 5%) 190 (100%) Total 278 ( 69%) 62 ( 15%) 14 ( 3%) 29 ( 7%) 18 ( 4%) 401 (100%) Material Availability 519 TABLE2 ANALYSIS OF SUCCESS IN KNOWN-ITEM AND SUBJECT SEARCHES BY METHOD OF DATA COLLECTION Method of Data Collection Observed Self-reported Known-Item 8 ( 47%) 107 ( 50%) trons found the material they were seek- ing. This approximates the overall success rates of similar studies reported in the li- brary literature. 26 A comparison of the success rates be- tween known-item searchers and subject searchers using both types of data collec- tion methods, is presented in table 2. As previously explained, the data collected by librarian observation was intended to be a check on the success rates derived from the patron self-reports. This check was needed because the Ciliberti study as noted above, indicated that self-reporting resulted in artificially high success rates in both known-item and subject searches. The data in table 2 fail to support this finding of the Ciliberti study. Chi-square tests indicate that there are no significant differences in performance due to the methods of data collection. Consequently, the self-reported success rates were not ar- tificially high, as had been anticipated. Analysis of Search Failures: A Macro Look Patrons' failures to locate the books be- ing sought can be divided into six catego- ries for both subject and known-item searches. In tables 3 and 4, these error cat- egories are listed in the order encountered and the success rate at each step of the search process. Thus, table 3 shows that 5 of the 211 pa- trons conducting known-item searches had erroneous bibliographic citations. Of the 206 patrons who had correct informa- tion, 21 were searching for titles the library had not purchased. Of the 185 who had accurate citations and were looking for books the library owned, 15 were unable to use the card catalog correctly, i.e., to lo- cate the appropriate cards and identify in- formation necessary for finding the books. Another 15 failed to find the books be- cause they were in circulation. At this Success Subject 9 ( 53%) 108 ( 50%) Total 17 (100%) 215 (100%) point, 155 persons were looking for titles that ostensibly should have been on the shelves; however, 40 of them were unsuc- cessful in locating these books because of some library malfunction, i.e., the books were not where they were supposed to be. Another 8 were unable to retrieve vol- umes that were shelved in their correct lo- cations. The total failure rate was 49 per- cent. When these errors are placed in the or- der of their relative negative impact on the search process they indicate where the greatest efforts toward future planning should be directed. The success ratio of only 74 percent at the library-malfunction stage of a patron's search should be of first concern, followed by acquisition, circula- tion, catalog use, retrieval, and biblio- graphic considerations. The subject-search errors shown in table 4 are listed in the order patrons encoun- tered them. Twelve of the 190 patrons con- ducting subject searches were either seek- ing titles that had not been purchased or were unable to select subject terms that matched their needs, i.e., were unsuc- cessful in locating a Library of Congress subject heading that would have been used in the card catalog. Of the 178 per- sons remaining, 11 had difficulty using the card catalog. Either they could not cor- rectly identify the call number, or they left out the location symbol, e.g., Folio or Ref. Twelve of the 167 patrons who success- fully reached this point were looking forti- tles that were subsequently determined to be in circulation and therefore inaccessi- ble. Twenty-one of the remaining 155 pa- trons were unable to locate their materials because of a shortcoming in either the pol- icies or procedures of the library that caused the book to be unavailable to them. Examples of library malfunction include 520 College & Research Libraries November 1987 TABLE3 KNOWN-ITEM SEARCH PERFORMANCE BY TYPE OF ERROR AND SUCCESS RATIO Bibliographic Acquisition Catalog use Circulation Library malfunction Retrieval Total Errors: 104 Total Known-Item Searches : 211 % Errors: 49% Number of Errors 5 21 15 15 40 8 missing books, volumes waiting to be shelved, and materials awaiting catalog- ing or repair. One hundred thirty-four searchers successfully negotiated these problem categories, but 12 more failures occurred because patrons were unable to find books that were correctly shelved. The sixth type of error, appropriateness, was committed by 14 patrons who found materials on the shelf but decided that they were inappropriate for their needs . The books might have been previously read, too old, too advanced, etc. Thus, only 108 patrons performing subject Total Patrons Searching 211 206 185 170 155 115 Success Ratio 98% 90% 92% 91% 74% 93% searches located material appropriate for their needs: 82 were unable to do so, re- sulting in a failure rate of 43 percent. It is possible to place the patron errors in the order of their negative impact on the search process: library malfunction is first, followed by appropriateness, retrieval, circulation, matching and acquisition, and catalog use errors. Analysis of Search Failures: A Micro Look The search failures encountered in this study can be further analyzed in three dif- ferent ways: (1) the origin of the failure- Material Availability 521 TABLE4 SUBJECT SEARCH PERFORMANCE BY TYPE OF ERROR AND SUCCESS RATIO Matching & acquisition Catalog use Circulation Library malfunction Retrieval Appropriateness Total Errors : 82 Total Subject Searches: 190 % Errors : 43% Number of Errors 12 11 12 21 12 14 library, patron, and other; (2) the status of the user; and (3) the longitudinal changes over the course of the semester. ORIGIN OF FAILURES Library Errors Sixty-three percent of all search failures can be considered library errors, i.e ., shortcomings in library routines. As table 5 indicates, sixty-one (56 percent) of these searches failed because the titles sought could not be located on the shelves or in the circulation records. An additional Total Patrons Searching 190 178 167 155 134 122 Success Ratio 94% 94% 93% 87% 91% 89% twenty-seven (25 percent) of these failures were due to the fact that the desired titles were already on loan, while the remaining twenty-one (19 percent) represent titles desired by patrons but not owned by the library. Library Malfunction Errors These sixty-one errors constituted 56 percent of all library errors. In more than one-half of these cases, patrons consulted the card catalog and found titles that they believed would be useful; however, these titles were unavailable. A closer look at 522 College & Research Libraries ''Sixty-three percent of all search fail- ures can be considered library errors, i.e., shortcomings in library rou- tines.'' these errors indicates that four were a result of books located on sorting shelves or trucks, two were overdue in circulation, one was on reserve in Lending Services, five were declared lost, and the remaining forty-nine could not be located by library staff Circulation Errors Twenty-seven (25 percent) of the fail- ures were the result of titles already being on loan when the patron searched for them. Acquisition Errors Twenty-one of the library errors were considered acquisition errors, i.e., pa- trons were searching for specific titles which the library did not own. A further analysis of these titles in terms of their suitability for an academic library collec- tion indicated that at least nine, but not more than fifteen, could be fairly judged to have been acquisition errors. Patron Errors More than one-third of all search fail- ures were errors committed by the pa- November 1987 trons. Of these, thirty-eight (60 percent) failed to use the card catalog correctly or interpret its contents accurately. An addi- tional twenty (32 percent) occurred be- cause patrons were unable to locate a title on the shelf when, in fact, it was there. A small proportion, only five (8 percent), re- sulted from erroneous bibliographic infor- mation brought to the catalog by the pa- trons. Matching and Catalog Use Errors A total of thirty-eight patrons made matching and catalog-use errors. Note that these failures represented 60 percent of all patron errors-patrons thus ap- peared to be ineffective users of the card catalog. They experienced difficulty in gaining subject access, in understanding the use of the call number, and in differen- tiating between the various sections of the divided catalog. Retrieval Errors A surprising twenty (32 percent) of all sixty-three errors observed were made by card catalog users who, though having correct bibliographic information and cor- rect card catalog information, were unable to locate books that were correctly shelved. Bibliographic Errors Judging from the small proportion (8 percent) of patrons who committed biblio- graphic errors, most of them were using TABLE 5 ORIGINS OF LffiRARY AND PATRON ERRORS Origin of Error Number(%) of Errors Library errors Malfunction Circulation Acquistion Total Patron errors Matching & catalog use Retrievaf Bibliographic Total *n = 172 Library Errors All Errors* ~- % % 61 27 21 109 56 25 19 100 Patron Errors No. % 38 20 5 63 60 32 8 100 35 16 12 63 All Errors* % 22 12 3 37 adequate. bibliographic information. Other Sources of E"or In addition to library and patron errors, appropriateness errors are a third source of failure in subject searches. Whereas fail- ures in the initial two categories typically represent titles not available at the time of need, appropriateness failures occur when patrons either fail to select call num- bers from titles found in the card catalog, or decide not to borrow (or use in the li- brary) the materials found after examining selected titles at the shelf. These decisions are usually made because the patron has already read the material or because, in his or her judgment, the information is too old or too new, not relevant, written in the wrong language or is in some other way not suitable to his or her information need. In contrast to the large numbers of li- brary and patron errors surveyed, only 8 percent of all failures observed were ap- propriateness errors. Unfortunately it is not possible from the available data to doc- ument the reasons patrons failed to select or use these materials. However, if pa- trons had had a better understanding of the information contained on the catalog card, it is possible that they would have been better able to distinguish early in their searches which titles were not appro- priate to their needs. Academic Status of User Undergraduates conducted 69 percent of all searches surveyed and committed 71 percent of the known-item and 73 percent of the subject errors. In addition, as shown in table 6, they were responsible for the majority of failures in each error category with the exception of matching errors in subject searches. A chi-square test was used to determine if the number of known-item and subject failures for undergraduates differed sig- nificantly from the number for other groups. The test indicated that under- graduates do not differ significantly in their search performance from all others. While undergraduates were responsible for 71 percent of the known-item search failures, they committed a disproportion- Material Availability 523 ''While undergraduates were respon- sible for 71 °/o of the known-item search failures, they committed a dis- proportionately large percentage (93°/o) of the catalog use errors.'' ately large percentage (93 percent) of the catalog use errors. In the subject search category, where they were responsible for 73 percent of the failures, their share of catalog use (82 percent), and appropriate- ness (86 percent) errors was again high. They also fell victim to a disproportion- ately high number of circulation errors. Graduate students and faculty commit- ted relatively fewer catalog use errors than did undergraduates. The data indicate that the faculty more frequently encoun- tered acquisition failures than other types of failures, while graduate students were more likely to commit matching errors. Longitudinal Changes All patron and library errors were ana- lyzed to determine if there was a signifi- cant difference in performance between the first and second half of the semester. Two statistically significant findings should be noted. Circulation errors increased from the first to the second half of the semester. Circulation failures represented 7 percent of all subject search failures in the first half of the semester and 18 percent during the second half. For known-item searches the failures increased from 2 percent to 22 per- cent. The most obvious reason for this in- crease in failure rates is that the chance that an item, particularly a specific known-item, will be in circulation in- creases as the semester progresses. While circulation errors increased dur- ing the semester, catalog use errors de- creased. Catalog use errors represented 21 percent of all subject search errors during the first half of the semester and 10 per- cent during the second half. For known- item searches the corresponding statistics are 22 percent and 9 percent, respectively. The decrease in catalog use errors may be the result of increased sophistication 524 College & Research Libraries November 1987 TABLE 6 KNOWN-ITEM AND SUBJECT SEARCH FAILURES BY ACADEMIC STATUS Academic Status ~rK!~f Under~ad Graduate Facul!r Other No Answer Total Known-Item Bib 60% Acq 62% Cat 93% Circ 67% Lib mal 73% Ret 63% Match 25% Appro 85% Cat use 82% Circ 83% Lib mal 76% Ret 83% Key: Bib = Bibliographic Acq = Acquisition Cat = Cataloging Circ = Circulation Ret = Retrieval Match = Matching Cat use = Catalog use Lib mal = Library malfunction Appro = Appropriate 20% 14% 0 13% 10% 38% 42% 7% 0 0 10% 8% Note : Due to rounding, all rows do not totallOO% . among patrons, exposure to bibliographic instruction, more experience in using the library, or a greater willingness to ask for help from reference librarians. SUBJECTIVE OBSERVATIONS In the course of observing patron searches, several unanticipated patterns of user behavior were noted. Although these subjective impressions cannot be quantified or measured, they provide im- portant information regarding user be- havior and performance. • Patron cooperation was excellent, even though the process of being watched as one searched may have impinged upon patron behavior and privacy. • Many patrons did not bring writing ma- terials with them to the catalog. It is pos- sible, therefore, that the retrieval rate of the patrons who were observed by li- brarians was higher than it might have been because patrons were given a form on which to note call numbers and other pertinent information. • Some patrons lacked persistence when they did not find their materials on the shelves. Nor did they seem to be aware of additional assistance available to 0 20% 0 100% 14% 10% 0 100% 0 0 7% 100% 7% 7% 7% 100% 3% 8% 8% 100% 0 0 0 100% Subject 0 0 33% 100% 0 7% 0 100% 9 9% 0 100% 8% 8% 0 100% 5% 5% 5% 100% 0 8% 0 100% them such as reference help, interli- brary loan, and traces. • The divided card catalog was the source of many problems. Patrons wasted time looking for the correct section; some never did use the appropriate file. • The name section of the card catalog provided further difficulties for patrons who searched for authors who were ei- ther prolific or who were the subject of many critical works, such as Shake- speare. Patrons failed to examine either the preceding or succeeding drawer when each contained appropriate en- tries. • Patrons also failed to note the signifi- cance of location symbols such as Ref. and Folio. "None of the observed patrons used the Library of Congress Subject Head- ings even though its use was dis- cussed in bibliographic instruction classes and a copy was prominently displayed at the catalog.'' • None of the observed patrons used the Library of Congress Subject Headings even though its use was discussed in biblio- graphic instruction classes and a copy was prominently displayed at the cata- log. • Many patrons were not able to follow the range identifiers on the stack ends or the arrangement of books in call number order on the shelves. Some ti- tles were not found because they were very thin and their classification num- bers were not visible to the patrons. • Few, if any patrons, checked the sorting area. • During the process of following up on materials not found by patrons, it be- came clear that certain idiosyncracies in OCLC records and those of the library's automated circulation system were not universally understood by staff. RECOMMENDATIONS Although the study indicated the suc- cess rates of the card catalog users at the William Paterson College Library were similar to those reported in comparable studies at other institutions, the commit- tee members believed them to be unacceptably low. Therefore, an extensive list of recommendations was submitted to the library administration. Those that ad- dressed library malfunctions, the largest cause of patron failure, were given prior- ity. Among the recommendations for im- proving this area of library operations were initiation of inventory and regular- ization of shelf-reading programs. Recom- mendations for remedying circulation, pa- tron and acquisition errors included improving signage, purchasing duplicate copies of high demand items, and incor- porating discussions of patron retrieval and card catalog use problems into biblio- graphic instruction classes. CONCLUSIONS It is apparent that a library's policy deci- sions, organizational structure, and phys- ical plant idiosyncracies influence patron success rates. At the William Paterson College Library these peculiarities in- cluded a building design and window placement that almost invited theft, a long Material Availability 525 period between inventories, and staff shortages that precluded sending overdue notices. Although this study was designed to be an in-depth examination of patrons' search successes and failures in a single li- brary, the authors believe that analysis of these findings and the local circumstances that influenced them have implications for library administrators in other academic settings. These implications are presented as suggestions for those who are inter- ested in improving the likelihood that their library patrons will find the materials they need. An initial step is to consider the impact of lost and stolen materials on patron suc- cess. The ease with which materials can be taken from the library without being checked out can be assessed and past poli- cies on replacements and overdue materi- als studied. The development of an appro- priate and realistic inventory program, changes in the physical plant, and/or the installation of an electronic detection sys- tem are some methods of ameliorating the situation. Another factor that may affect patron search success is signage. Directional signs that have become ''invisible'' to staff members who "see" them all the time may be woefully inadequate. An assess- ment of signage by an outside party may help improve patron access to materials. This study has some additional implica- tions for those involved in the design or selection of on-line public access catalogs. In order to minimize catalog use errors li- brary planners should endeavor to design or select on-line systems with self- explanatory screen displays. Patron con- fusion and errors resulting from location abbreviations would be reduced if loca- tions within the library (e.g., Ref., Doc, etc.) were clearly spelled out. Further, li- . brary jargon need not be incorporated into these systems. Classification numbers, for example, could be labeled "location num- bers" or "shelf location numbers" rather than "call numbers." In addition, administrators may want to emphasize the inclusion of status informa- tion (i.e., whether a title is in circulation, at the bindery, on the shelf, etc.) in their 526 College & Research Libraries evaluations of public access systems. Er- rors stemming from patron failure to find materials on the shelf might be reduced if patrons searched more tenaciously. It seems reasonable to assume that if pa- trons knew that the material they want should be on the shelf their resolve to find it would be strengthened, and the likeli- hood that they would continue their searches by availing themselves of trace services offered by the library's circulation department would be increased. Finally, it is clear that a self-study such November 1987 as the one described in this report requires a substantial investment of staff time and effort. However, this report also indicates that the rewards for this investment can be the collection of data which accurately re- flect patron success and failure in obtain- ing needed library materials. Further, when the study is properly designed, the sample carefully drawn, and the return rate high, such an effort may enable li- brary administrators to identify and mea- sure the relative magnitude of the barriers to patron success. REFERENCES 1. T. Saracevic, W. M. Shaw, and P . B. Kantor, "Causes and Dynamics of User Frustration in an Academic Library," College and Research Libraries 38:7-18 (Jan. 1977). 2. A. C. Ciliberti, "The Development and Methodological Study of an Instrument for Measuring Material Availability in Libraries" (Ph.D. diss., Rutgers Univ., 1985) . 3. Ibid. 4 . J. Mansbridge, "Availability Studies in Libraries," Library and Information Science Review 8:299-314 (Oct.-Dec. 1986) . 5. R. L. Meier, "Efficiency Criteria for the Operation of Large Libraries," Library Quarterly 31 :215-34 (July 1961). 6. P . V. Rzasa and N. R. Baker, "Measures of Effectiveness for a University Library," Journal of the American Society for Information Science 23:248-53 (July-Aug . 1972). 7. M. Hamburg, L. E. Ramist, and M. R. Bommer, "Library Objectives and Performance Measures and Their Use in Decision Making," Library Quarterly 42:107-28 (Jan. 1982). 8. E. R. DeProspo, E. Altman, and K. E. Beasley, Performance Measures for Public Libraries (Chicago: American Library Assn., 1973). 9. R. H. Orr and others, "Development of Methodological Tools for Planning and Managing Library Services: II. Measuring a Library's Capability for Providing Documents," Bulletin of the Medical Library Association 56:241-67 (July 1968) . 10. P. B. Kantor, "Availability Analysis," Journal of the American Society for Information Science 27:311-19 (Sept.-Oct. 1976). 11. Saracevic, "Causes and Dynamics of User Frustration." 12. W . M. Shaw, Jr., "Longitudinal Studies of Book Availability," in Library Effectiveness: A State of the Art, ed. N . K. Kaske and W. G. Jones. (Chicago: American Library Assn., 1980), 338-49. 13. J. B. Whitlatch and K. Kieffer, "Service at San Jose State University: Survey of Document Avail- ability," Journal of Academic Librarianship 4:196-99 (Sept . 1978). 14. Y. W. Wulff, "Book Availability in the University of Minnesota Bio-Medical Library," Bulletin of the Medical Library Association 66:349-50 (July 1978). 15. R. H . Smith and W. Granade, ''Userand Library Failures in an Undergraduate Library,'' College& Research Libraries 39:467-73 (Nov. 1978). 16. E. S. Palais, "Availability Analysis Report," Association of Research Libraries SPECKit71:73-82 (Feb. 1981). - 17. T. R. Kochtanek, "User Satisfaction in the Hugh Stevens College Library," (Arlington, Va.: ERIC Document Reproduction Service, 1978), microfiche ED 190 164. 18 . N. A. Radford, "Failure in the Library-A Case Study," Library Quarterly 53:328-39 (July 1983) . 19. Ciliberti, ''The Development and Methodological Study.'' 20. T. E. Ferland M.G. Robinson, "Book Availability at the University of California, Santa Cruz," College & Research Libraries 47:501-8 (Sept. 1986). Material Availability 527 21. Ciliberti, "The Development and Methodological Study." 22. J. Galtung, Theory and Methods of Social Research (New York : Columbia Univ. Pr ., 1967). 23. Ciliberti, "The Development and Methodological Study." 24. R. P. Palmer, Computerizing the Card Catalog in the University Library (Littleton, Colo.: Libraries Un- limited, 1972). 25. R. Tagliacozzo and M. Kochen, "Information-Seeking Behavior of Catalog Users," Information Storage and Retrieval6:363-81 (Dec. 1970). 26. Shaw, ''Longitudinal Studies of Book Availability.''