College and Research Libraries By JOHN F. HARVEY Measuring Library Audio-Visual Activities T HIS PAPER will discuss the measure-ment of library activities with spe- cial reference to audio-visual materials. 1 It will do this, first, by placing the sub- ject in its proper setting with a discussion of the contribution of measurement to library administration, and second, by discussing certain problems of measure- ment in libraries especially as they relate to audio-visual activities. Much of the latter discussion is based upon the expe- riences of the ACRL Committee on Au- dio-Visual Work in 19522 and again in 1955 3 in attempting to collect data on audio-visual activities in American col- leges and universities. MEASURING LIBRARY AcTIVITIES The types of problems encountered when attempting to measure audio-visual activities are similar to those met in try- ing to measure other library activities. Problems which arise in measuring an- nual expenditures for films are much the same as those encountered when measur- ing e~penditures for binding or for mag- azines. Certain statements can be made concerning the_se general problems of li- brary measurement which apply also to the measurement of audio-visual services. 1 The author would like to acknowledge the helpful ad- vice of Fleming Bennett, Dale Bentz, and Budd Gam· bee in preparing this article. 2 Fleming Bennett, "Audio-Visual Services in Colleges and Universities in the United States," CRL, XVI (1955), pp. 11-19. For this survey a special qu estion- naire was mailed. in 195 2 to all colleges on the ALA Publishing Department mailing list, 1726 in all. 3 John F. Harvey, "ACRL A-V Statistics, 1954-55," (Unpublished manuscript, 1956), 3p. For this survey a fourth page of questions was a ttache d to the annual ACRL statistics questionnaire an d mailed in the su mmer of 1955. Dr. Harvey is librarian} State Teachers College} Pittsburg} Kansas. MAY} 1957 This discussion should begin with the statement that all measuring of library activities-collecting statistics on use, expenditures, size of collection, etc.-is not essential to library service. Excellent service may be given without measuring anything or collecting any statistics. No doubt many libraries have done this, so for a library association as well as for a library such an activity may be regarded as a luxury. Purpose of Measurement Even though it is not an essential op- eration, measurement is defensible on a logical basis if it will contribute toward improved Iibrary service. In fact, any ac- tivity can be defended logically if it will improve service, though its cost should be commensurate with results. No other defense of measurement is ac- ceptable. Collection of statistices to sat- isfy the librarian's curiosity or because the routine has been carried on for many years is not defensible. Each series of sta- tistics now being collected should be eval- uated carefully in the same manner as a series newly contemplated. Both chief li- brarians and association committee mem- bers should ask such questions as: Does the use of the figures in connection with improved service justify their collection, both kind of use and amount? Are the service needs being served legitimate, or important, or is the project costing more than it is worth? Another important consideration is that the library must have some need for improving service in an area before it is justified in collecting statistics related to it. This may seem to be an obvious gen- eralization, but many librarians collect statistics without fully realizing the im- 193 plications of the connection between the activity and service. Careful Description Necessary After an area of weakness has been dis- covered and the librarian has decided thnt collecting statistics will aid in im- proving the situation, then this area of weakness must first be described clearly . . This is often difficult. It is difficult to de- termine which type of audio-visual ma- terial is not being circulated, to isolate the subject field and that part of the fac- ulty and student body not participating, and to describe the situation so that proper understanding of it can be reached. In order to understand the perform- ance of a library department and to de- scribe it in some detail, it is necessary to measure it carefully in meaningful terms. Data collection thus becomes important as a tool with which to learn more about the situation and can lead to the thor- ough understanding on which logical de- cisions can be based. Comparison with a Standard Once information on a situation is available, in order to evaluate this situa- tion the pertinent information should be compared with a suitable standard. Set- ting up such a standard will probably prove the most difficult part of the pro- cedure. Standards are uot available in all areas of library service and of the ones we have many are merely averages or else goals to shoot at. In lieu of generally recognized stand- ards, there are several alternatives . Com- parison of the various departments with- in a library can prove helpful if sufficient care is taken to recognize their differ- ences. Comparison of one department's performance over several years is usually even more helpful. Comparison with the same department in another, similar li- brary can be useful to the extent that the two departments have comparable goals. Many different standards can be used and they can be expressed in several dif- ferent terms. Film use per student, film expenditures per student, record circula- tion per student credit hour of enroll- ment, and departmental expenditures per circulation charge for audio-visual materials are just a few of the approaches to measurement in terms allowing com- parison with other libraries and with pos- sible standards. When to Collect Data After measurements have been made and a sufficient amount of data collected to allow a full description of the situation the librarian may be able to stop measur- ing until he has decided what method of improvement to use. There is no need to continue collecting data after enough in- formation is available to establish the picture-no need to prove again what is known already. In other cases, however, continuous data collection will be desir- able. Then, after the method of improve- ment has been applied, in many cases data can again be collected to see if any improvement in results has occurred. PROBLEMS OF Aumo-VISUAL MEASUREMENT This section will discuss several pro b- lems of data collection on audio-visual activities. Certain of these problems re- late to the ACRL mail questionnaire, others to data collection within the indi- vidual library, and still others equally to both, but all were met with or implied in the recent ACRL collection of audio- visual statistics and should be anticipated by others attempting to collect data in this field. Purposes of Audio-Visual Statistics A list of possible reasons for collecting audio-visual data is in order here: I. Information may be needed for reports required by the local college president, college budget officer, public relations 194 . COLLEGE AND RESEARCH LIBRARIES officer, by the regional accrediting asso- ciation, or by ACRL. Of course, such in- formation should be collected only if it can be used to improve library service, but in these situations the local librar- ian has little control over the decision to request the data. 2. Information may be needed by the li- brarian for comparison with perform- ance standards for operations suspected to need improvement. 3. Information may be collected for a reading, viewing, and listening record for each student. Such a record would be valuable in counseling and in place- ment. Probably any one of these reasons would be sufficient to make data collec- tion desirable. A C omp.lete Return Obtaining a complete return from all libraries with a mail questionnaire is one of the most difficult problems for an asso- ciation committee to solve. Getting com- plete figures in the individual library is often surprisingly difficult, too. In the 1952 survey of the ACRL Committee on Audio-Visual Work, only 34 per cent re- tur,ned completed questionnaires, a per- centage so small that few conclusions for the. entire group could be derived from the data. About the same ratio returned correctly completed forms in the survey undertaken in 1955. Including a stamped and self-addressed envelope with the information form and sending a follow-up letter some time after the first wave of returns is received are helpful procedures in securing a com- plete return. It is important also that the questionnaire contain a full explanation of the need for the information- audio- visual or otherwise-who is collecting it: and how its collection will benefit li- braries. No attempt should be made to . collect data already available elsewhere. Not only is a problem presented by in- complete returns, but also there is need to have forms completed in their entirety without questions being omitted. The problem is often seen in the individual library where incomplete record keeping hinders analysis of library problems. Re- turns from the 1955 audio-visual ques- tionnaire were quite incomplete in terms . of forms completed properly. Some li- brarians apparently failed to see this page of the questionnaire, since they left it blank; others thought it was not im- portant and threw it away; still others completed half of it and stopped; and many others reported contradictory fig- ures. Several procedures may be recom- mended as helpful in achieving complete reporting. To reduce chances of misun- derstanding, the information form should contain clear definitions of the technical terms used. For instance, a film and a record should be defined on audio- visual surveys. No valid comparisons can be made with the data unless all respond- ents understand all questions in the same way. Also, units used in reporting must be the same as those called for on the in- formation form, or they may not be com- parable to the other answers obtained. Printing these units on the form may en- courage respondents to use them. Clarity of wording and logical arrangement of questions are also important, as is provid- ing for simple answers such as a number, or "yes" or "no." And, at the end, a note should suggest that the respondent check over the entire questionnaire to make sure all questions have been answered correctly. Another way of clarifying the question- naire's purpose to the respondent is to list all items included in the term, "au- dio-visual materials," as they are covered in the particular questionnaire. Such a list might include 16mm. and 35mm. mo- tion picture films, 35mm. filmstrips, slides, records, maps, globes, pictures, tape and wire recordings, unexposed film, television sets, bulletin boards, mi- crofilm, microcards, microprint, dictating 195 machines, radios, and museum objects. In addition, a survey might include equipment, such as cameras, projectors, screens, and phonographs. If such a list is given, it should reduce the number of in- formation forms returned in which .the librarian reports that he has no audio- . visual materials and fails to realize that such common library materials as pic- tures, microfilm, maps, and bulletin boards are included. This was one of the difficulties with the 1955 survey which re- ceived many reports showing no audio- visual materials in large libraries, librar- ies which no doubt had at least a few of these materials. Frequently, librarians, working with a new questionnaire, do not have easily available the figures necessary for correct answers. This was one of the chief pro b- lems in the two ACRL audio-visual ques- tionnaires. Reporting a slide collection as containing "hundreds of slides," as did one librarian, is not very helpful. Many librarians had not maintained budget and expenditure control over these activi- ties specifically, so they could not give cost figures for audio-visual materials, maintenance, and equipment. It is useless to ask for data not readily available, and any group wishing to collect audio-visual data should either give respondents two years of advance warning and list the breakdowns desired or else collect only what is readily available. Sampling Techniques The use of sampling techniques can be suggested as a means of collecting data with relatively little expenditure of time. They may be used to advantage by both the association committee and the indi- vidual librarian. For example, the micro- film machine use totals obtained through taking a sample for a dozen selected weeks will often provide a good approxi- mation of the totals which might be ob- tained through counting painstakingly every single usage during the year. Several library departments should be able to obtain satisfactory estimates of total activities by sampling. For instance, it should be helpful in counting monthly book, periodical, and au d i ·o- vis u a I charges, the number of persons using a reading room or stack area, the number of books patrons remove from shelves, the number of reference questions asked, the number of books on library shelves, the number of people viewing library film showings, attendance at children's story hours-in fact, in any situation where an exact total is not needed and where there are no large and irregular fluctuations in the accumulation of the data. The Library with No Audio-Visual Materials On any mail information form pro- vision should be made at the beginning for the librarian with no audio-visual materials to check, so that he need not fill in the remaining blanks. Provision for such checking and for listing all mate- rials should enable respondents to see where they stand immediately and to complete the form properly. Provision should also be made at the beginning for indicating whether or not the questionnaire seeks information on library audio-visual materials even though the main campus collection of these materials may be housed elsewhere. Sev- eral librarians indicated on the 1955 au- dio-visual survey that since the main campus collection was elsewhere, they assumed the survey was not interested in their own audio-visual holdings-an as- sumption which was incorrect. The problems of measuring holdings are concerned primarily with defining what is to be measured. Which gives a truer picture of the library's holdings, the number of volumes held or the num- ber of titles? The number of reels of film, film titles, or prints? The number of al- bums, records, ti ties, or slides? 196 COLLEGE AND RESEARCH LIBRARIES .., In general it is reasonable to count audio-visual materials in the same man- ner as printed materials are counted. If the library counts books by title, then it should count films and records by title; if by volu{nes, as do most libraries, then by reel and record. At any rate, in collecting data on hold- ings, the unit should be specified and results asked for in the same units from all libraries. Librarians occasionally try to measure the caliber of holdings by using a check- list, but more often they are content with determining the size of holdings. The exact significance of size is not clear, but no doubt it is thought to be related to caliber. Where a sufficiently comprehen- sive checklist can be checked, one which is well adapted to the needs of the par- ticular library, this should be the best means of measuring a book, periodical, film, or record collection; I?u t few such lists have ever been compiled. Measuring Expenditures In order to show expenditures in mean- ingful categories it is necessary to organ- ize the library's financial records to re- veal this information. This will often re- quire keeping financial records in con- siderable detail. For instance, in order to ascertain the extent to which film and filmstrip collections are being built up to serve the school of medicine, it may be desirable to know the film and film- strip expenditures in that area both be- fore and after the build-up was started. These expenditures are of little value un- less they can be compared over a period of several years and also with expendi- tures of other libraries collecting film and filmstrip material for the same pur- pose. The units in which expenditures should be obtained provide no room for difference of opinion, all being expressed in dollars. Other helpful breakdowns, however, might call for dollars spent on audio-visual materials per student credit MAY_, 1957 hour of enrollment in medicine or audio- visual circulation charges in medicine per dollar spent for audio-visual mate- rials. Measu1·ing Use The difficulties of measuring library use are great and will be dealt with only briefly. The most important problem is not how many pictures or books go out the door, but their impact on the library community. It is very difficult to measure impact on the community, i.e., the ~xtent to which education and recreation are improved over what they might be with- out the library, but the point is that measuring circulation charges is o'nly a crude and indirect method of gauging this impact. It measures the fact that the pa- tron charged out a picture, not that he looked at it or that it had an effect on him. We now measure only what should lead to positive results, not the results themselves. In measuring use of audio-visual mate- rials there are several problems peculiar to the kind of material. For instancer should use be measured according to the· number of films and records charged out, by the number of persons to whom these films and records were shown or played, or by the number of times they were shown or played? Since there is now no good way of measuring directly the im- pact of films and records on patrons, li- brarians should try to measure it indi- rectly as accurately and completely as possible. This suggests measuring the en- tire number of people who saw the film or heard the record, though this is more difficult than merely counting the num- ber of times it was charged out. Another problem is caused by the dif- ferences from campus to campus in meth- ods of promoting and circulating such material. In one library home circulation of records may be promoted while in an- other it may not be allowed. The best that might be done with figures of use is to group the departments with like 197 treatment of materials and avoid com- parisons between unlike departments wherever possible. Analyzing the Data and Interpreting Results Interpreting the results of any data col- lection should be relatively easy if it has been set up properly. However, the ques- tioner, reporter, and analyzer must all three have understood the questionnaire, its purpose and procedure in the same way; otherwise the data cannot be an- alyzed correctly and the conclusions will not be valid. Results can often be put into tables and frequency charts which will bring out the relationships of the data. It should usually be a question of whether or not the data show expected results, since data will usually be collect- ed to prove or disprove one or more hypotheses or to be compared with a per- formance standard. But care must always be t~ken to ~arry out careful, scientific analysis of data, or conclusions will be biased. What analyses .of audio-visual data are likely to be helpful and meaningful and to be worth the effort made to collect them? Of course, the answer to this ques- tion as to so many others depends on the l~cc;tl situation. But several breakdowns may be suggested as having potential value. These breakdowns may be _divided into those showing a potential for-service and those showing service given. In order to evaluate the potential for service it may be desirable to obtain fig- ures representing the total number of materials, broken down by type as seems necessary, the total pieces of equipment. as well as the expenditures for new mate- rials, new equipment, and equipment re- pair, and the items 'Yhich were newly acq~ired in these categories. With stu- dent enrollment figures, the librarian can then work out ratios showing the number of materials available per student, ex- penditures per student, and other po- tentials · for . service, which may be com- pared with figures from other campuses or other parts of the library. They will be more meaningful 'in showing how well prepared the library is to give good serv- ice if broken down by subject, by de- partment, or by school of the institution. To show service, the usual use figures are necessary, broken down by kind of material or service and again by depart- ment or school of the institution if they are to be more meaningful. With such figures the librarian can study potential, the extent to which po- tential is being realized, and, further, can identify the weak and strong areas in the operation. For instance, the record col- lection may be found to rank first in the region, or perhaps filmstrips are being used at a high rate per student enrolled per title available; but the fact that pic- tures are not being used, suggests the need of either budgetary changes in au- dio-visual acquisitions or else more in- tensive promotional activities. Again, physical education may be using film- strips heavily per student credit hour of enrollment, but biology may . not. Furthermore, cost analyses can be made which, when compared with analyses in other libraries, will suggest whether or not the library is paying too much for the level of service provided. Such an- alyses are often made in terms of unit costs. For example, is the acquisitions cost per film significantly higher in Li- brary A than in a representative sample of similar libraries? Or, in such a situa- tion, is the annual repair cost per piece of audio-visual equipment higher than the average? Or is the cost per recording circulated higher than it is in compar- able libraries? Such unit cost analyses can be used to identify areas of operation needing careful procedure analysis lead- ing to work simplification. CONCLUSION This has been an attempt to describe the various problems which confront a (Con tinued on page 204 ) 1.98 COLLEGE AND RESEARCH LIBRARIES --- - ------------------------------------------------------------