ACRL News Issue (B) of College & Research Libraries February 1988 / 85 lished in Improving Reference Management, pa­ pers based upon a workshop fo r public services and reference managers, May 10-12, 1984, Atlanta, Ga. (Chicago: R A SD /A LA , 1986), 55-70. A new approach to reference statistics By John M. M axstadt Assistant Reference Librarian Louisiana State University Libraries Many reference librarians consider collecting reference desk statistics a waste of time. All too of­ ten they are right, in two ways. Time-consuming statistical procedures perform ed every hour of every day by reference professionals “waste” time by taking it away from patron service; and statis­ tics systems designed without a sound scientific ba­ sis are a waste of anyone’s time. Before July 1986, the method of collecting refer­ ence desk statistics at Louisiana State University Libraries suffered from both of these common fail­ ings. Librarians and paraprofessionals assigned to the reference desk were expected to record every patron question with a tally mark on a statistics sheet which classified questions by hour of day and by type (information, reference, research, or card catalog). This system was a distraction at best, and positively hindered reference service during hours of peak usage. At such times, the desk staff found it practically impossible to record every single ques­ tion; some left many questions unrecorded, while others set down large numbers of marks at random, simply to reflect how “busy” the shift had been. To make matters worse, there was considerable variety in staff interpretations of the basic question categories (information, reference, research, card catalog). Also, the library administration wanted statistics kept on the types of patrons (faculty, stu­ dents, etc.) served at the reference desk, which would have made statistics taking hopelessly cum­ bersome under the system in use at that time. Tow ard the end of fiscal year 1985/86, senti­ ments were strongly in favor of a new approach to reference desk statistics. The ideal system would collect statistics by both question type and patron type in a more scientific and statistically sound m anner while freeing the desk staff to concentrate on the information needs of patrons. T he n e w approach Reference Services Division head Jane P. Kleiner became interested in sampling reference desk sta­ tistics when she served as ACRL liaison to the Pub­ lic L ib ra ry D ev elo p m en t Task F orce. A fter Douglas L. Zweizig (author of O utput Measures fo r Public Libraries) spoke to the LSU Libraries staff about output measures and the successes other college and university libraries have had w ith sta­ tistical sampling, it was decided that the LSU Li­ braries Reference Services Division should try a similar approach. Rather than attem pt to record every question asked during approximately 4,000 hours of service throughout the year, a small num ­ ber of selected hours would be designated as statis­ tics sessions, and statistics would be recorded only during those hours. The exact number and distri- Recruitment open for editor of Rare Books & Manuscripts Librarianship ACRL’s newest journal, Rare Books & Manuscripts Librarianship, will require a new editor to serve on a volunteer basis when Ann Gwyn completes her term of ser­ vice. The incoming editor will assume full editorship in July 1988. Besides ACRL membership, candidates should have a background of service in aca­ demic or research librarianship, as well as experience and expertise in special collec­ tions librarianship; experience in research, editing, and bibliographical activities; a concern with publication as a means of pro­ fessional communication; and an ability to analyze manuscripts for content, research methods, form, structure, or style. Together with the editorial board, the editor is charged with encouraging research and w riting th a t may be appropriate for the jo u rn al, soliciting topics and suggesting them to appropriate authors, and editing and refereeing manuscripts. Persons wishing to be considered for the editorship should communicate their inter­ est, accompanied by a statement of qualifi­ cations and names of references, By April 1, 1988, to Ruth J. Person, Dean, College of Library Science, Clarion University, C lar­ ion, PA 16214; (814) 226-2271. 86 / C&RL News bution of the hour-long statistics sessions would be determ ined as a statistical sample from a total “population” of 4,000 + hours. The Reference D epartm ent turned to W illiam G. W arren and Kung-Ping P. Shao of the LSU D e­ p a rtm e n t of Experim ental Statistics for advice. They studied the reference desk statistics for a typi­ cal m onth (April) of the previous year, and con­ cluded th a t the figures approxim ated a Poisson distribution—a statistical model used to predict the arrival of travelers at bus and train stations, am ong other things. The LSU Libraries adm inistration had decided th a t a 90 % confidence level and an error range of ± 10 % for the total num ber of questions asked d u r­ ing the year would be sufficiently precise. W arren and Shao found in the April 1986 statistics an aver­ age (mean) rate of 32 questions per hour and a stan­ dard variance (using the Poisson model) of ± 14 questions. Applying these figures to the equation below (a standard equation for determ ining the size of a simple random sample) yielded a sample size of 52 hours out of a total of 4,103 hours of ser­ vice. Concerned th a t greater variations might oc­ cur in other months of the year, W arren and Shao suggested increasing the sample to 60 hours. They also em phasized the im portance of distributing these 60 hours individually and random ly through­ out the year in order to m aintain the validity of the sample as a simple random sample. (where σ equals the standard variance, equals the mean rate of questions, D equals the acceptable range of error, and Z is a standard norm al deviate determined by the confidence level/2) The statistics sessions were distributed using a ta ­ ble of random numbers. Each hour of service in the fiscal year was num bered from 1 to 4,103, and 60 random numbers in th a t range were recorded. A list of the days, dates, and hours corresponding to those random num bers was compiled and rear­ ranged into a calendar of statistics sessions for fiscal year 1986/87. Between two and nine sessions fell in each m onth, w ith an average of five per m onth. N ew s ta tis tic s sheets w e re d e sig n e d , w ith columns for patron type (faculty, graduate stu­ dent, u n d erg rad u ate student, and “o th e r”) and rows for question type (information, reference, re­ search, and online catalog training—the LSU Li­ braries had recently converted to a NOTIS auto­ m ated system w ith online catalog). Each sheet was to be used for one statistics session only. A single m ark would indicate both patron type and ques­ tion type, so the num ber of undergraduate students asking reference questions (for example) w ould be recorded for each session. Because it w ould be necessary to ask each patron w hether he or she was a faculty m em ber, graduate student, or undergraduate student, there was con­ cern th a t the process would consume a great deal of desk staff tim e during statistics sessions. To avoid delaying service to patrons and possibly biasing the sample, it was decided to assign two of the d ep art­ m ent’s eight graduate assistants to take statistics during each session. The assistants would be able to concentrate fully on the statistics, while the desk staff could provide uninterrupted reference ser­ vice. Also, the eight assistants could be trained to record statistics more uniformly th an the varied group of librarians, paraprofessionals, and assis­ tants assigned to the desk at various times of the day and week. To further insure uniform ity in statistics recording, a m anual was w ritten containing de­ tailed definitions and examples of the four types of questions. Results At the end of the fiscal year, the figures from all the statistics sessions were totaled and m ultiplied by 68.383 (4.103 hours of service divided by 60 hours of statistics equals 68.383) to produce the to ­ tals used in the annual report. Questions were di­ vided by source (in person or telephone), type (in- fo rm atio n , reference, research, online catalog training), and patron type (faculty, graduate stu­ dent, undergraduate student, “other”), so there were 32 individual question categories ( 2 x 4 x 4 = 32). For example, in-person faculty inform ation questions w ould be one individual question cate­ gory. Individual categories were combined into columns (all in-person faculty questions, for exam­ ple), rows (all in-person inform ation questions, for example), and larger groupings, including a grand total of all questions asked. In all, the statistics yielded 53 separate d ata elements (totals of differ­ ent combinations of patron and question type) for cross-comparison and comparison w ith the pre­ vious year’s figures in the annual report. Shao analyzed these d ata elements, using the original statistics sheets, to determ ine an error range for each element at a confidence level of 90% . For the grand total of all questions asked in the year, the error range was ± 1 1 .2 3 % , very close to the desired range of ± 10 % . The more specific data elements had wider error ranges, because they comprised smaller parts of the sample population and in m any cases had greater variance from one statistics session to the next. Some of the individual question categories had error ranges of ± 50 % and more; those figures were considered only indicative of a general range into which the actual num ber of questions might have fallen. Even w ith these levels of error, the new a p ­ proach is considered far more reliable and statisti­ cally valid th an the old approach, in w hich the in­ a c c u ra c ie s w e re su b je c tiv e (re s u ltin g fro m unpredictable hum an error) rather th an objective 88 / C &R L News and ran d o m , and w ere therefore not subject to m easurem ent or control. The error levels in the present system could be reduced by increasing the sample size; however, the library adm inistration has decided th a t the present level of accuracy is suf­ ficient for the tim e being. T he reference staff have greatly appreciated their release from the tyranny of recording statis­ tics every hour of every day, and have reported im ­ proved interactions w ith patrons as a result of greater freedom to concentrate on service. G rad u ­ ate assistants w ere scheduled for only one or two statistics sessions in the average m onth, so the extra duty did not impose any great hardship. In fact, statistics duty frequently helped them m ake up hours lost during breaks and university holidays. T he assistants becam e quite ad ep t at statistics­ taking during the year, and encountered surpris­ ingly little difficulty asking each patron his or her status. Small modifications have been m ade in the sta­ tistics system for fiscal year 1987/88. The most sig­ nificant change is th a t the “other” category of p a ­ trons has been divided into four sub-categories (faculty and staff of other universities, other infor­ m ation professionals, elem entary and secondary school students, and “other”). In general, how ­ ever, the new sam pling approach to reference desk statistics has been highly successful at LSU L i­ braries, and is considered w orthy of em ulation by other reference departm ents who w ould like to in­ crease the accuracy of their desk statistics while de­ creasing the effort and resources devoted to collect­ ing them . ■ ■ How to p a rticip a te in ACRL Information for new members and potential members. T o join W rite or call the Association of College and Re­ search Libraries for application forms and infor­ mation: ACRL Office, 50 E. H uron St., Chicago, IL 60611-2795; (312) 944-6780. To learn about ACRL’s cu rren t program s, officers, and units Check the ALA H andbook o f Organization and read College & Research Libraries N ew s regularly (as a m em ber of ACRL, you will receive both p u b ­ lications) . To a tte n d conference p ro g ram s, ACRL B oard meetings, or committee meetings For inform ation on tim e, places, and general program content, consult C& RL News, Am erican Libraries, and th e official conference program s and schedules. For additional inform ation, contact the ACRL office. To atten d preconferences For inform ation about tim e, place, and precon­ ference them e, check C & R L N ew s and Am erican Libraries. For additional inform ation and registra­ tion m aterials, w rite to the ACRL office. To take a continuing education course To receive a course brochure, w rite to the ACRL office. To register, use inform ation published in C C R L N ew s or in the course brochure. To establish a local chapter of ACRL Check the procedures listed in “Guidelines for ACRL C hapters” (available from the ACRL of­ fice) . Send a petition w ith the signatures of at least 25 ACRL m em bers to th e ACRL office for a p ­ proval by the Board of Directors. To request a speaker in the C hapter Speakers p ro ­ gram W rite Alia Al-Taqi at the ACRL office and spec­ ify your date and topic. E ight chapters qualify each year for a free visit from ACRL officers or th e ACRL Executive D irector.