College and Research Libraries T. SARACEVIC, W. M. SHAW, JR., and P. B. KANTOR Causes and Dynamics of User Frustration in an Academic Library Traditionally library effectiveness has been considered in relation to "Public Knowledge" and the associated literature. More recently it has been considered in relation to a given set of users~ to use~ and availability. A number of studies dealing with user satisfaction and frustration in obtaining books from a library are reviewed. Observa- tions on the causes of user frustration at Sears Library~ Case Western Reserve University (CWRU)~ were done on two occasions: In 1972~ during the time when a "sem.ester loan" policy was in effect and in 1974 when the loan period was changed to a "four week" policy. A method of analysis was developed that allows for .the calculation of four independent probabilities indicating measures of perfo_rmance of acquisitions policy~ circulation policy~ library operations~ and users. It is argued that the branching analysis for the combination of effects and the particular measures derived are universally applicable for studying these aspects of library performance. " ... a well-stocked, well supported, and allegedly well balanced library can routinely thwart its patrons on nearly half of their quests "1 IN THE LATE 1960s a new type of li- brary study emerged, oriented toward ~ibrary effectiveness and involving mod- els and measures of · performance found in operations research, systems engineering, and related fields. Consid- ered by some to be of only academic in- terest, such studies are nevertheless root- ed in real .and practical library problems T. Saracevic is professor of library sci- ence, W. M. Shaw, ]r.~ is research and sys- tems librarian and assistant professor of li- brary science, and P. B. Kantor is associate professor of library science and operat-ions research at Case Western Reserve Univer- ' sity, Cleveland, Ohio. exemplified by the quotation above. A number of studies have found that about 40 to 50 percent of the patrons trying to obtain a book in a library eventually leave without having ob- tained satisfaction. There .are too many such findings to be ignored or dismissed as statistical variation. Since it is also known that on the av- erage the libraries have acquired about 90 percent of the materials sought, it seems imperative to analyze and disen- tangle the factors contributing to the observed user frustration. Can anything more be done, or is the present situation in some peculiar sense a natural or "op- timal" one? Can we better the situation by improving library procedures? To what extent are the users causing their own frustration, and can anything be "'done about it? ·Our study seeks answers to these ques- I 7 8 I College & Research Librarie$ • January 1977 tions by means of library effectiveness studies. In particular, we seek to mea- sure the performance of the library and the user separately, but in ways which can be sensibly combined to provide a single overall measure. The pressures to undertake such a study result from a variety of adminis- trative and societal forces calling for increased effectiveness, .accountability, and broader bases and services, all with- in relatively static budgetary constraints. Numerous solutions have been pro- posed and/ or attempted in order to in- crease effectiveness. For instance, the no- tion that information systems should be organized and regarded as utilities re- flects user orientation: increased effec- tiveness, increased availability, broaden- ing of the population served, easier ac- cess, etc. Studies of library performance are rooted in .and related to both present and anticipated problems. The studies by themselves do not provide solutions. They do, however, provide background information and a basis for rational de- cision and policy and thus for better li- brary performance. LIBRARY PERFORMANCE The performance of any system can be very roughly divided into "effective- ness" (how well it does what it is in- tended to) and "efficiency" (how much it costs in time or money per unit of performance). A complete examination of effectiveness includes: 1. Specification of a purpose or goal of the system and of the parts studied. 2. Selection of a measure or measures reflecting this purpose and specifi- cation of the units of measure. 3. Specification or construction of measuring instruments .and the im- plementation of a measuring meth- od. In general, the assessment of effective- ness is a complex problem. In particu- lar, for social systems (and to a certain extent libraries are social systems ) the difficulty is heightened because there .are unresolved problems at each of the three steps. As yet, there are no universally ac- cepted measures, measuring units, or methods for the study of library ef- fectiveness. In part, this is because there is little agreement as to what is to be measured. Which of the purposes of a library should the measures reflect? Libraries can be thought of, quite generally, as systems which acquire, or- ganize, and store recorded knowledge for the purpose of communicating knowledge to their users. Libraries are imbedded in the larger process of com- munication of "public knowledge," a term used by Ziman for the "rational consensus of ideas and information."2 The input to the library is recorded knowledge, i.e., literature; on the output side we find the users. Libraries seek to enable and enhance the process by which the users receive the knowledge contained in the literature. Effectiveness might be measured at a number of points in the process. Tradi- tionally, library effectiveness was consid- ered in relation to "public knowledge" and the associated literature. Thus at- tention was directed to completeness and balance of the collection (in rela- tion to subject and subject literature), adequacy of material selection and .as- sociated policies, flexibility of classifica- tion schemes, adequacy of cataloging, etc. For each of these a set of more or less arbitrary standards can be and has been devised. More recently library effectiveness has been considered in relation to the other end of the knowledge communication process, the given set of users and po- tential users. This directs our attention to types and proportions of satisfied re- quests, proportion of frustrated users, effect of loan policies and duplication policies on this satisfaction, etc. Since the problem is a difficult one, most stud- ies have isolated one or another factor of the problem and stayed away from considering the interplay between these factors. The study reported here is of the use type. It includes four factors which contribute to user satisfaction. We can easily see the w.ay in whi_ch they are com- bined with each other. We also report on a successful modification of one of these factors, producing exactly the de- gree of improvement to be expected, and showing that we have indeed re- solved the basic performance measure into a set of independent components. PREvious WoRK Although there have been several . studies addressing similar problems our work was primarily inspired by the work of Morse and Buckland. Morse proposed several models involv- ing probability distributions of tasks and users in a library, classes of users, arrivals, departures, circulation, queues, age dependence of demand, etc. For the MIT science library he inferred that at least 30 percent of the requests were un- satisfied. The factors he considered in- cluded loan period, book duplication, reserve collection, and reshelving. 3 Buckland and his colleagues studied usage at the University of Lancaster li- brary, concentrating on what they called the "pathology of library provision." They began with a "frustration survey," observing immediate availability of books and the factors that prevent it. The performance measure, called "sat- isfaction level," was taken to be the fraction of requests which were promptly satisfied, and was found to be 5.3 percent. 4 The earliest frustration study of the present type that we have found was done by Meier in 1957. "Loan perfor- mance" w.as studied from a closed stack library over a two~week period. Of the requests received 58 percent were de- User Frustration I 9 livered immediately. 5 A number of other studies concen- trated on the causes of readers' failure to find a book either at a catalog, on the shelves, or both. Tagliacozzo and Ko- chen studied various .aspects of the be- havior of catalog users in interviews at three libraries of the University of Michi- gan and the Ann Arbor Public Library. 6 Combining the failures at the catalog and at the shelves, they found that 50 to 60 percent of the searches were un- productive. Urquhart and Schofield conducted the largest survey to date on the causes of "failures of users to find a book on the shelf." Their work cannot be expressed as a satisfaction ratio, since successes were not recorded. 7 Orr and his colleagues extended the concept of user frustration to include the delay associated with retrieving doc" uments from a library. Their studies were conducted in a series of biomedi- cal libraries. The results of these studies are expressed in terms of a "capability index" -which has a maximal value of 100 only if all documents .are found on the shel£.8 Ben-Ami Lipetz studied the catalog use of the Yale University Library. He 'found that 84 percent of documents sought were located by the users, and some 5 percent were not located by the users but found in the _ catalog later by research staff of the study. The library possessed approximately 90 percent of items requested by users. 9 Not .all use-type ·studies of library ef- fectiveness have concentrated on de- mands, satisfaction, and ftustration. Other important measures have been suggested. Hamburg suggested that the overall measures of library performance should be based on document ex- posure since the "exposure of individ- uals to documents of recorded human existence ... is the most important as- pect of all public and university library objectives." The suggested measures 10 I College & Research Libraries~ January 1977 were: exposure counts, item-use-days, and exposure time. (Hamburg's book also provides the most complete review and synthesis of studies on library per- formance measures a'nd models con- ducted up to 1972.) Hamburg recog- nized that .alternative measures (such as Morse's and Buckland's) which concen- trate on the proportion of user demands satisfied may be preferable to document exposure measures, although the con- cepts of "demand" and "satisfaction" are not purely objective.l0 In another study Kantor has provided a synthesis of these two approaches in which exposure time is related to avail- ability (satisfaction) by means of "av- erage use times." These factors are sep- arately measurable and are presently un- der investigation. 11 SETTING AND PRocEDURES The study was conducted at Case Western Reserve University ( CWR U ) , a private university with about 4,700 un- dergraduate and 4,400 graduate and pro- fessional students, 650 faculty ( exclud- ing clinical faculty), and a staff of 1,400. The university has two major li- braries: 1. Freiberger Library: Approximately 1,000,000 volumes in humanities, arts, and social sciences. About 50,000 volumes circulate annually. 2. Sears Library: Approximately 200,000 volumes in science, tech- nology, and management. About 30,000 volumes ·circulate annually. Four other libraries at CWRU are as- sociated with professional schools: health, library science, law, and social science. There are also several smaller collections housed in individual depart- ments. The data reported in this study were collected at the Sears Library only. We collected data on the requests for books of a specific title/ author ("known item searches") and on the reasons for frus- tration of these requests, i.e., data were collected on satisfied and unsatisfied de- mands for known books. Data were col- lected at two different times: ( 1) For six days in April 1972 when 423 re- quests from 183 users were analyzed, and ( 2) for nine days in November 1974 when 437 requests from 189 users were analyzed. Loan policy had been changed during the intervening time, and thus we are able to isolate its effect on overall satis- faction. In 1972 there was a "semester loan" policy: Books borrowed at any time dur- ing the semester could be kept, without penalty, until the general recall date at the end of the semester. However, after two weeks from the original loan date a book could be recalled by any library user. Upon receipt of a recall notifica- tion, the borrower had to return the book within five days in order to avoid paying a penalty. In 1973 a study was undertaken to evaluate the "semester loan" policy. Ex- perimental data were gathered, a loan policy model was formulated and veri- fied through experimental observation, and simulations were carried out in which the effect on book availability and recall delays of differing loan pol- icies were studied. 12 As a result of that evaluation a "four week loan" policy was instituted in 1974. All books are due four weeks af- ter the check out date. As in the "semes- ter loan" policy, a book may be recalled by any library user two weeks after the original loan date. Therefore, we are comparing in this study, the effects of the "semester loan" policy with the ef- fects of the "four week" loan policy. Data were collected as follows: 1. Research staff were stationed at the catalog. 2. Users of the catalog were briefly interviewed and asked about par- ticipation. 3. A questionnaire was filled out, in- cluding the list of books sought. 4. Upon leaving the library, the user was reinterviewed as to satisfaction of requests. 5. Research staff determined the rea- sons for unsatisfied demands and, where necessary, extensively re- checked the problematic items and verified the decisions. Users finding books in ways that did not include use of the catalog were not interviewed. OBSERVATIONS Table 1 presents the results of the survey of satisfied and dissatisfied ( frus- trated) requests for ~books from Sears Library in 1972 (during the "semester loan" policy) and in 197 4 (during the "four week loan" policy). The notation is as follows: User Frustration I 11 W = total number of requests S = number of ,immediately satisfied requests D = number of dissatisfied requests The dissatisfied requests were further classified into four general categories: Da = number of dissatisfied requests attributable to library Acquisitions-the title was not in the library collection. De = number of dissatisfied requests due to the book being in Circulation (on loan or in-house use). D1 = number of dissatisfied requests attributable to Library malfunction. Du = number of dissatisfied requests attributable to tJ ser error. Within some dissatisfaction categories the reasons for failures were further elaborated and enumerated. Explanations are given with Table 1. TABLE 1 RESULTS OF THE _1972 AND 1974 SuRVEYS OF SATISFIED AND UNSATISFIED (FRUSTRATED) REQUESTs FOR BooKS AT SEARs LIBRARY, CASE WESTERN REsERVE UNIVERSITY Semester Loan Four Week Loan Policy,1972 Policy, 1974 Notation I. Total number of requests 423 437 w II. Number immediately satisfied 203 245 s III. Number not immediately satisfied 220 192 D IV. Distribution of unsatisfied requests A. Number of books not acquired by the library 52 38 Da B. Number circulating (on loan or in house use) 81 48 De 1. circulating on loan 70 43 2. located on the hold shelf• 3 3. in use in the library 11 2 c. Number of library malfunctions 29 45 Dt 1. missing and known by the library 2 18 2. missing and not known ·by the library 21 22 3. misshelved 6 2 4. located on the pre-shelving shelvest 2 5. book repair area within the library 1 D. Number of user errors 49 50 Du 1. incorrect call number 18 3 2. book properly shelved 12 23 3. book on reserve 10 2 4. book located elsewhere* 9 22 E. Other§ 9 11 o A book which has been recalled is held at the circulation desk "hold shelf" until checked out by the recaller. t Books which are picked up from the tables each day are brought to the preshelving area to be arranged by call number prior to being reshelved. t Books located in special areas of the library or in another campus library are identified by a location sym- bol on the catalog card. · §"For the 1974 study, the "Other" category contains all .titles for which there are multiple copies. It can be assumed that the status of these copies is distributed over cases De, D 1, and Du identified above. For the purpose of later calculation of probabilities the titles in category "Other" for both studies a:re allocated to De, Dp Du in the approximate ratio of number of titles in De, Dp Du• Thus for the 1972 ( 1974) data the 9 ( 11) items in category ~'Other" are allocated as follows: 3 ( 4) to Du, 2 ( 3) to D 1, and 4 ( 4) to De.'' 12 I College & Research Libraries • January 1977 METHOD OF ANALYSIS The data were analyzed using tech- niques13 developed by one of the authors as part of a larger program in a study of library effectiveness. The outcome of any specific request for a book can be placed in one of the following inde- pendent categories already defined: In other words, the outcome can be re- duced to these five independent com- ponent effects. The five independent outcomes can be represented on a branching diagram that follows the particular order in which an outcome can occur. This is shown in Figure 1. The values for S, Du, D~, D e, and Da are given from the data collection as pre- sented in Table 1. The values forT, U, V, and W are calculated as follows: T = S + Pu U = T + D1 V = U + D e W = V + Da Of course, the value of W obtained in this way is equal to the total number of requests. Independent probabilities which describe this branching process can now be calculated according to the following formulas: Pu = S/T Pr T/U Pe = U/V Pa = V/W We can interpret these probabilities as follows: Pn is a measure of user performance. P1 is a measure of library performance. P e is a measure of circulation policy performance (including internal li- brary use of the books ) . Pa is a measure of acquisition policy performance. s - ( Sa t il tied reques ts ) W • V + Da ( Tota l requests) Du • (Dissa t is fa c t ion d ue. to us er error) Fig. 1 Branching Diagram of the Outcomes of Requests for Books from a Library The four probabilities allow us to iso- late the various obstacles to satisfaction of requests and to explore effects of alter- native "fixes" of given hindrances. Buck- land's "satisfaction level," p 8 , is calculated as follows: Ps S/W, or in terms of the branch- ing probabilities: Ps Pu · PI · Pe · Pa The particular sequence of branching can be understood in the following way: 1. The first thing that matters in seek- ing a given book from a given li- brary is that the library has acquired it; if it has not, nothing else matters. The search must be moved to some other source. If it has acquired the book, still other barriers must be passed before one gets the book; therefore, Pa comes first. It repre- sents the fraction of sought books which have been acquired. 2. If a library has a book, the second thing that matters is that the book is not in circulation or in some other use; if it's "in," still other hindrances remain; thus, Pe follows Pa· 3. If the book is "in," the third thing that matters is that the book is in its proper place (not stolen, not mis- shelved, not at the bindery, etc.), i.e., if the library functions properly, the book should be in a known place. If it isn't, nothing else mat- ters. If it is, still another barrier re- mains; thus, p 1 follows Pe· It repre- sents the fraction of "in" books which are in proper locations. 4. If a book is "acquired," "in," and in its "proper place," then the book is available, but that does not neces- sarily mean that the user will get it. The user has to perform without error in catalog or shelf lookup, etc. , If the user commits an error, then even though the book is available the request is not satisfied; Pu fol- lows other probabilities and is placed at the last fork of the branch- ing diagram. The branching diagram represents the process of "staying on the right track" to achieve immediate satisfaction in a search for a known title. The branches must be arranged in this order because the im- plied conjunctions ("not circulating and in its proper place") make sense. If we arranged them so that library perfor- mance came first, we would have a fork in which we should ask: "How many of the books which are in their proper place are not in the library after all?" This pa- rameter (if it has any meaning at all) was surely not measured in this survey. We believe that it is reasonable to as- sume that the causes of the four kinds of branching act independently. Thus, if a book is returned from circulation, we assume that the probability that it will be correctly placed on the shelves is the same as the probability p 1 observed for those books in the library at the time of the study. It is this independence which makes these four probabilities important analytical tools. User Frustration I 13 ANALYSIS OF OBSERVATIONS "Satisfaction level," p 8 , for: 203 1972 was 423 == .48 and for 245 197 4 was 437 == .56 Figure 2 presents the branching dia- gram for the 1972 observations during the "semester loan" policy and Figure 3 for the 197 4 observations during the "four week" loan policy. The data are taken from Table 1. (As explained in the notes to Table 1, the dissatisfied re- quests from category "Other" are dis- tributed to the appropriate three cate- gories in the same ratio as the . number of items in each of the three categories; thus the nine items in category "Other" in the 1972 study were allocated as fol- lows: 3 to Du, 2 to Db and 4 to De. The eleven items in this category in the 1974 study were allocated in this way: 4 to Du, 3 to D~, and 4 to De.) Table 2 presents the probabilities, i.e., measures of performance for Sears Li- brary for both 1972 and 197 4 surveys calculated from the appropriate values from Figure 2 and Figure 3. The per- formance measures are presented sepa- rately from the branching diagrams in order to provide for easier comparison between the two surveys. Each of the performance measures is calculated from a sample and is, there- fore, subject to statistical errors. These errors may be estimated in terms of the standard deviation, assuming that the sample events represent Bernoulli trials. The formula expressing the standard deviation, CT, in terms of the performance measure p, and the size of the sample, N, is: u == v'p(1-p)/N For all the branching points in our analysis this error is less than ± 3 per- cent. The standard deviation for each probability is given in Table 2. 14 I College & Research Libraries • January 1977 TABLE 2 MEAsuREs OF PERFORMANCE FOR THE SEARS LIBRARY, FRoM THE OuTCOMES OF REQUESTS FOR BooKs DuRING THE SURVEYS IN 1972 .AND 1974 1972 1974 Standard "Semester Loan" "Four Week Deviation, <1 Policy Acquisition: Pa = V/W 88% Circulation: pc = U/V 77% Library: PI = T/U 89% Users: Pu = SIT 80% Satisfaction Level Ps = S/W 48% DISCUSSION Satisfaction level The 1972 observation found a satis- faction level Ps of 48 percent, and the 1974 observation found one of' 56 per- cent. Comparison of performance mea- sures for individual factors, as discussed below, isolates the cause of the increase of 8 percent in satisfaction level be- tween the two observations. The satis- faction levels found in this study are comparable to those found in the other studies reviewed. These obversations suggest that satisfaction levels of 40 to 60 percent are to be found commonly in academic libraries. Similarity of or- ganization and procedures may explain the apparent universality of these satis- w - 423 s - 203 ·Du • 49+3 Fig. 2 Branching Diagram of the Outcomes of Requests for Books from Sears Library from the 1972 Survey During the "Semester Loan" Policy Loan'' Policy 1972 1974 91% 1.6% 1.4% 87% 2.2% 1.7% 86% 1.8% 1.9% 82% 2.5% 2.2% 56% 2.4% 2.4% faction levels. This range is somewhat higher than Gore's estimate but still does not seem to us to be a high level of satisfaction for a service , Acquisition The performance measure reflecting acquisitions, Pa for 1972 was 88 percent and for 197 4 91 percent, a slight in- crease, but since it is about two ·standard deviations it is possibly not significant. The library has acquired some 90 per- cent of books requested by users. Similar levels of acquisition were found when we applied our analysis to the other studies reviewed. 14 Even those libraries that have considerably larger collections, such as Yale with some 3,000,000 volumes in comparison to w - 437 s ~ 245 D u • 50 + 4 Fig. 3 Branching Diagram of the Outcomes of Requests for Books from Sears Library from the 1974 Survey During the "Four Week" Loan Policy Sears with 200,000 volumes, have ac- quired some 90 percent of books re- quested by their users. Therefore, ac- quisition policies and procedures work very well; evidently they have been de- veloped to a high art in academic li- braries of various sizes. As a result, the size of academic libraries was eliminat- ed as a significant factor that affects sat- isfaction, at least in the libraries stud- ied. Theoretically, interlibrary loans are designed to supplement an acquired li- brary collection. A corollary finding of acquisition performance is that the the- oretical upper limit of demands that in- terlibrary loan could satisfy (if used for supplemental purposes) is some 10 percent of total demands on those aca- demic libraries. In reality this is lower: Interlibrary loans satisfy less than 5 per- cent of demands in academic libraries. One might be concerned that users of the Sears Library are conditioned to seek fewer titles than users of the larger Yale Library. However, we simply note that acquisition has a generally high performance measure and should not command our attention at present. Circulation The performance measure of circula- tion policy, Pc, for 1972 when a "semes- ter loan" policy was in effect was 77 per- cent and for 197 4 when a "four week" loan policy was in effect was 87 percent. In other words, of those books that were requested by the users and had been ac- quired by the library 23 percent in 1972 and 13 percent in 197 4 were out on loan or in use in the library. Therefore, a change in loan policy improved the cir- culation policy performance some 10 percent, a factor which has to be con- sidered quite significant. This shows the power of alteration of loan policies. However, such alteration cannot be con- sidered only in terms of days a book is allowed to be out. If so, the "no loan" policy would be most effective. The time User Frustration I 15 requirements for effective use of a book by a user have to be taken into account as well. Therefore, the length of a loan policy should be determined as a com- bination of time factors involving vari- ous effectiveness aspects: effectiveness of the library to a number of users and ef- fectiveness of the book to individual users. This can be best discussed in terms of the mean usage time associated with a borrowing. As Pc approaches 100 percent, mean usage time goes to zero. The quantity of interest is the product of the two. 15 The problem of ioan policy is compli- cated by the demand distribution. A few books are requested quite often, while demand for others falls off rapidly. De- mands for books from libraries approx- imately follow .a Bradford law. The "bunching" effect (effect of the Brad- ford "nucleus") can be alleviated by use of demand dependent loan periods, book duplication, and recall requests. Models of Morse, Buckland, and Shaw incorporating these features are rele- vant to the analysis of Pc· The in-house use of books was consid- ered in this study together with books on-loan because both represent a form of circulation. The data in Table 1 pre- sent on-loan and in-house use separately, but a separate measure for each type of circulation although possible is not nec- essary. The interesting question to be asked is: What is the ratio between the in- house use of books in a library and the external on-loan use of books as mea- sured by exposure time? Obviously, loan policy affects both, the in-house .and on- loan circulation. In gathering data for the simulation of loan policies, it was found that 44 percent of books used at Sears Library in the fall of 197 4 were through loan and 56 percent were used in-house. For every book borrowed, an- other was used in the library. Only 7 percent of books circulated were re- called.16 Therefore, loan policies affect 16 I College & Research Libraries • January 1977 considerably more than just the on-loan circulation. The option of recall is rela- tively unused. Many users are not aware of it or do not understand its meaning. Let us consider the assumption that unsatisfied demands are only due to the fact that a book is in circulation, as used by Shaw.1 7 The level of satisfac- tion. Ps', in this simplified case may be calculated from the branching diagram as follows: Ps' For 1972: Ps' s S +De 203 203 + 85 245 Ps' == ---- 245 + 52 For 1974: .70 .82 These formulas allow for a direct comparison of loan policies in simula- tion. · Library Operations The performance measure of library operations for 1972 was 89 percent and for 197 4 a slightly smaller 86 percent. In other words, of those books that were requested by users, and acquired by the library, and not in circulation some 11 percent in 1972 and 14 percent in 1974 could not be found because of some malfunction of library operations. In 1972 this placed the library perfor- mance first and best among four factors studied; in 197 4 it fell down to third place, a change of borderline statistical significance (about 1~ standard devia- tions ) . There are a variety of causes of user frustration included in this ·cate- gory, e.g., inadequate procedures, pol- icies or directions, huma._n mistakes on the part of library operators, etc. More debatable is the inclusion of stolen books as a cause of frustration. Are they due to inadequate library se- curity or due to efforts of thieves who overwhelm all economically feasible and legally allowable security measures? The majority of the library malfunc- tions in both time periods studied fell in the category of "missing books." In the 1972 study approximately 8 percent of the books which were not in circula- tion were missing, and in 197 4 the frac- tion had risen to 12 percent. (The cor- responding numbers of events can be found in Table 1.) The majority of these were not known to be missing by the library staff; that is, their absence was discovered during the frustration survey. This suggests a need for im- provements in security, inventory checks, shelving procedures, and shelf reading. (Some missing books may have been misshelved by mistake and others purposely misshelved as a kind of "pri- vate reserve" system.) If the number of missing books con- tinues to climb, and is, in fact, related to pilferage, then security will eventual- ly become a major practical issue, with important economic and psychological consequences. At present, missing books are on the way to becoming a major fac- tor contributing to user frustration, and they have already become the over- whelming factor in library .malfunc- tions. It is customary to indicate the number of missing books in terms of a percent of the total collection or of .acquisitions for a given time period. As a measure of the effectiveness of a li- brary this is grossly misleading, because it is the books in high demand that are missing most often. Thus a small num- ber of missing books which are in high demand can · decrease library perfor- mance quite significantly. User Performance The measure of user performance, Pu, for 1972 was 80 percent and for 197 4 82 percent. In other words, of those books that were requested by users, and available (i.e., acquired and not in circulation and in their place) 20 percent in 1972 ( 18 percent in 1974) were not found b~cause of some mis- take by users. Of the four performance measures, user performance was third in 1972 and worst in 1974. Therefore, library user performance is a most ap- propriate choice for attempts at im- provement. Some causes of user ineffectiveness are due to normal human errors: copying incorrect call number, looking at the wrong place on the shelf, wrong spell- ing, etc. Even these causes should be of library concern: Library catalogs, signs, shelving, etc., can be designed in a way to minimize common human errors. Highly visible signs that alert users to commonly committed errors and to users' own fallibility may minimize their errors. There were also "user errors" that could not be clearly .attributed to user errors and for which the library has to assume some responsibility. For instance, in the cases where the catalog card indi- cates that a particular book is located in a special area of the library, it is not to be assumed that a user can follow that code and retrieve the book. For exam- ple, "q" above a call number means that the book is located on oversize book shelves. How . many users will, in fact, detect the difference and look for over- size shelves? "Geology" . above a call number means that the book is in the collection of the Geology Department and not that the book is on the subject of geology. If a book is placed on re- serve and a user doesn't know that in ad- vance, it is highly unlikely that he or she will ever find it. Nobody looks for books "on hold,'' "in process," etc. An appropriately designed orientation program for users might improve the overall library effectiveness, and there- fore should receive a great deal of at- tention. CoNCLUSIONS We have presented and applied a User Frustration I 11 ''branching" analysis of frustration sur- vey data. The resulting performance measures are independently significant, and their product determines the overall satisfaction rate. The three measures not directly relat- ed to circulation remained substantially constant during the time between the two parts of our study. This enhances our confidence that they .are not only easily measurable but also meaningful parameters. Of course, they provide a tremendously convenient way of sum- marizing a great deal of data and of comparing the performance of differ- ent libraries. At any time the overall satisfaction is most affected by whichever perfor- mance factor is the lowest. At the time of the first study the circulation factor was lowest. (The analysis reported here was, in fact, conducted after the change in policy, so that we did not at the time know that that factor was in fact the most crucial one.) The substantial in- crease in this factor is exactly in accord with the predictions of the simulation study which led to its adoption.18 At present the lowest factor is user performance. It has remained fairly constant (within statistical errors) dur- ing the two year period of the study. We have also performed "retrospective" analyses of data from earlier studies men- tioned and find that user performance at the 80 percent level seems quite typi- cal. This limiting factor is also found in our analysis of Meier's data showing that library personnel in a closed stack library are 81 percent effective in re- trieving books. Both Morse and Buckland concentrat- ed on loan policies, which are but one factor in frustration. In general, there were no attempts to combine into one model all or even most factors studied. The present paper represents a step in that direction. 18 I College & Research Libraries • january 1977 This research was supported by grants from the National Science Foundation (GN-36085) and from the Cleveland and Gund Foundations for the Complex Sys- tems Institute of CWRU. The authors acknowledge the assistance of students in the <