College and Research Libraries An Evaluation of Reference Desk Service John 0. Christensen, Larry D. Benson, H. Julene Butler, Blaine H. Hall, and Don H. Howard Recent studies of academic and public libraries across the country are revealing serious deficiencies in the quality of reference service-only 55 percent of the factual and bibliographic questions asked are answered correctly. A five-part management study was conducted of the Brigham Young University Library reference service to determine the quality of service provided by the student and department assistants, the adequacy of the training program, and the effectiveness of the system of referring patrons to subject specialist librarians. The major problems revealed include the following: (1) the student and department assistants feel isolated from the subject specialists; (2) the referral system works poorly; (3) the training program is inconsistent and often ineffective; and most seriously (4) the student assistants answered only 36 percent of the unobtrusive test questions correctly. he objective of a library is not merely to warehouse informa- tion, but to help patrons find the information they desire. Though all aspects of librarianship are concerned with this goal, it is at the refer- ence desk that the process of determining the patron's need, formulating a research strategy to fill that need, and providing ac- curate and complete information to the patron is finally tested. Providing professional-quality reference service is a complex process, requiring extensive sub- ject expertise, knowledge of library collec- tions and systems, and years of practical experience. Studies of academic and public libraries across the country reveal serious deficien- cies in the quality of their reference ser- vice. Recent studies show that only 55 per- cent of the factual and bibliographic questions asked are answered correctly. 1 In addition, librarians often provide only partially correct information, demonstrate a lack of familiarity with basic reference sources, allow an "internal clock" to limit the amount of time they devote to patron queries, negotiate patron needs poorly, and neglect to refer patrons to someone more informed. 2 In the Lee Library (main library) at Brigham Young University, the five subject-reference departments were reor- ganized in 1986, creating a patron service system staffed by one full-time parapro- fessional department assistant and several student reference assistants for each de- partment. The purpose of the reorganiza- tion was to make "better use" of profes- sional librarian subject specialists by removing them from staffing reference desks so they could place greater empha- sis on collection development and faculty liaison responsibilities. The reorganiza- tion was based on a widely held premise that nonprofessionals could answer most of the questions brought to the reference desks and could be properly trained to re- fer the questions too difficult for them. 3 The subject specialists became involved in John 0. Christensen, Larry D. Benson, H. ]ulene Butler, Blaine H. Hall, and Don H. Howard are reference librarians in the Harold B. Lee Library at Brigham Young University Library, Provo, Utah 84602. 468 Evaluation of Reference Desk Service 469 offering patron service at the reference desks only when the student reference as- sistants determined that they needed ad- ditional subject expertise. The questions of how effective nonpro- fessionals are in providing reference ser- vice and what training is necessary have become increasingly important. Martin Courtois and Lori Goetsch observe, "Properly trained nonprofessionals with an understanding of reference service and a clearcut referral relationship may en- hance the job roles of both nonprofession- als and professionals.'' 4 Richard Johnson indicates that the primary ingredient nec- essary for learning is motivation. 5 Beth Woodward and Sharon VanDerLaan contend that although motivation is not a problem for graduate students in library science, it can be for other nonprofession- als working on a reference desk. 6 Both kinds of student assistants work in the Lee Library. Recognizing the potential for a decrease in reference service effectiveness and perceiv- ing the problem nationally, the library ad- ministration appointed a committee to eval- uate the quality of service received by library patrons at the reference desks. For compari- son it would have been desirable to evaluate the reference service before the change, but this was not possible. Most evaluative stud- ies in the library literature have focused on how professional reference librarians per- formed. With reference desk duty being shifted totally to nonprofessionals in the Lee Library, this situation provided an easy op- portunity to evaluate nonprofessional ser- vice at the reference desk. A five-part management study was de- signed to assess the quality of the refer- ence service provided by the student ref- erence assistants in the Lee Library after a full semester under the new system. 7 The study focused on determining the accu- racy of information patrons received, the level of skill student reference assistants used in negotiating patron needs, the abil- ity of student reference assistants to refer patrons to professional librarians, and the degree of satisfaction patrons felt about the service they had received. The five parts, conducted during the winter se- mester of 1987, included the following: • Patron Suroey: After completing a refer- ence interview, approximately 100 pa- trons were approached out of view of the student reference assistants and asked to complete a short question- naire. Questionnaires were adminis- tered at all reference desks during all hours of reference operation over a two- week period. • Reference Assistant Suroey: All student reference assistants employed at the five reference desks in the library were asked to complete a questionnaire re- garding their background, work envi- ronment, reference training, adequacy of support from department assistant and subject specialists, and job respon- sibilities. • Department Assistant Suroey: The five paraprofessional department assis- tants, who work at the reference desks and supervise the student assistants, were asked questions about their educa- tion and experience, job responsibili- ties, work environment, effectiveness of reference service and training, and the role of subject specialists in provid- ing subject expertise. • Subject Specialist Suroey: The eighteen professional reference librarians/ sub- ject specialists responded to questions about their perception of the quality of reference desk service given by student reference assistants and department as- sistants, the reference training pro- vided, and their own involvement in reference service. • Unobtrusive Question Test: Fifteen library employees acted as proxy patrons and asked five preresearched questions, one at each of the five reference desks. The proxies then completed a form describ- ing and evaluating the service they re- ceived from each student assistant. The questions, typical of those asked at the various reference desks, required sub- stantive factual or bibliographic infor- mation as an answer. Fifteen of the seventy-five questions were designed to test question-negotiation skills. A representative selection of the ques- tions used for unobtrusive testing is found in appendix A. This report integrates the responses of 470 College & Research Libraries these five questionnaires, showing com- parisons whenever possible, under the following major categories: (1) quality of reference service, (2) training, (3) role of professional librarians in reference desk service, and (4) conclusion and recom- mendations. QUALITY OF REFERENCE SERVICE Because the quality of the reference ser- vice performed at the reference desks by the student reference assistants was the prime focus of the evaluation, the study attempted to obtain pertinent information about this service from patrons, patron proxies, paraprofessional department as- sistants, professional subject specialists, and student reference assistants them- selves. Reference Interview The study addressed the four compo- nents of the reference interview: 1. Question negotiation between stu- dent reference assistants and patrons 2. Search strategies used by student ref- erence assistants in providing or at- tempting to fulfill the patron re- quests 3. Correctness of the answer provided 4. Referral, when required, to another, more appropriate, source to obtain a satisfactory answer (i.e., other stu- dent reference assistants, the depart- ment assistants, the subject special- ists, another library department, or a nonlibrary resource) In addition, patrons, proxies, department assistants, and subject specialists rated their perception of the quality of the refer- ence service provided by the student ref- erence assistants. Question Negotiation. The most basic ele- ment of effective reference service is for the student reference assistant to know as precisely as possible what the patron needs. Of the seventy-five questions in the Unobtrusive Question Test, fifty-two needed or could have benefited from ne- gotiation with the patron to determine the exact need. The student reference assis- tants negotiated only thirty-one questions (60 percent). July 1989 11 Although the student reference as- sistants recognized and responded well to explicit requests for negotia- tion, they did not perform as well on the implicit requests." Fifteen of the fifty-two questions, called "escalator questions," required the stu- dent reference assistants to negotiate the patrons' specific needs, because the proxy patrons asked the questions in very broad terms. The student reference assistants negotiated all five of the explicit questions (e.g., "I need information on Eskimos"), but only five of ten (50 percent) of the im- plicit ones (e.g., "Where is your po- etry?"). The questions with the implicit need for negotiation attempted to account for the numerous patrons who appear to know exactly what they want, but must have their real need negotiated before they can be given effective service. Al- though the student reference assistants recognized and responded well to explicit requests for negotiation, they did not per- form as well on the implicit requests. This suggests that the problem may not be an inability to negotiate with patrons, but ei- ther a failure to recognize more subtle re- quests or an unwillingness to question pa- trons who seem knowledgeable and informed and state their requests in such specific, positive terms. The department assistants and subject specialists rated the student reference as- sistants' ability to negotiate questions 3.6 (1 = very weak; 5 = very strong) and 2.8 respectively (see figure 1). Perhaps the de- partment assistants themselves, even though they work more closely with the student reference assistants, are not as aware of what good negotiation is as the subject specialists. Search Strategies. Once a patron's needs have been clarified, the student reference assistants must then develop a strategy for providing the appropriate information. Essentially this process involves selecting appropriate reference books or other in- formation sources and using them effec- Evaluation of Reference Desk Service 471 Subject Department Skill or Trait Specialists Assistants a. Subject knowledge 3.0* 4.0 b. Knowledge of reference books 3.2 3.4 c. Question negotiation ability 2.8 3.6 d. Developing search strategies 2.4 3.6 Composite Rating 2.9 3.7 *l=Very weak; 5=Very strong FIGURE 1 Effectiveness of Student Reference Assistants in Reference Skills as Perceived by Other Department Staff Answer Total Ques- Correct/ Correct/ Search Process Used tions Complete Referred Incomp. Wrong None Answered w/o consult- ing a source 14 0 · 2 3 4 5 Used one source only 18 12 5 0 1 Used several sources 17 5 1 3 2 6 Gave patron source w/explanation 10 5 4 0 1 Gave patron source w/o explanation !k __£ _Q_ ~ ..1! ~ Totals 75 24 3 19 12 17 FIGURE2 Effectiveness of Search Process Used in Answering Questions tively to give as full an answer as possible to the patron query. The data in figure 2 show some prob- lems in the search strategies used by the student reference assistants to find re- quested information. Superficially, it ap- pears that the greatest number of ques- tions were answered either correctly or correctly but incompletely by using one source, and the fewest number of ques- tions were answered by not consulting any source. However, this result reflects the type of questions used in the test more than it does the value of using a certain number of sources in finding an answer. More likely, the data show that when stu- dent reference assistants find an answer, correct or incorrect, from the first source 472 College & Research Libraries they consult, they search no further. The fact that nineteen questions (25 percent) were only partially answered suggests that student reference assistants may be relying too heavily on too few sources, may not understand the question fully enough to provide all the information re- quired for a complete answer, or do not have or take enough time. The above data also show that when pa- trons are referred to specific sources to find the information on their own, a com- mon practice at most reference desks, they find more correct answers when the stu- dent reference assistant shows them how to use the reference source rather than just pointing it out. Correctness of Answers. The ultimate test of the student reference assistants' effec- tiveness is the number of correct answers they provide to patron queries. In the un- obtrusive question test (see figure 3), stu- dent reference assistants demonstrated serious deficiencies in their ability to pro- vide complete and correct information. Only 36 percent of the seventy-five ques- tions asked by proxy patrons resulted in completely correct answers. Another 25 percent received partially correct or in- complete results from their query. And even with the inclusion of the correct-but- incomplete answers, the result is still only 61 percent with any degree of correctness. As noted earlier, the profession at large se- riously questions the acceptability of cor- rect responses to 55 percent or less of the questions asked at the reference desk. The perception of other reference de- partment staff of the student reference as- sistants' effectiveness in those factors es- sential for good reference service also supports the view that student reference assistants do not perform at a desirable or acceptable level (see figure 1). Referrals. Another important compo- nent the study attempted to measure was the performance of the student reference assistants in referring patron requests to someone more qualified than themselves when necessary. Crucial to this process is the ability of the student reference assis- tants to recognize when they need to refer and their willingness to do so. The study addressed three aspects of referral: The July 1989 unobtrusive question study provided in- formation about what student reference assistants did when unable to answer questions; the student reference assistant questionnaire asked about the availability of the professional librarians for referrals when needed; and the subject specialist questionnaire asked how many referrals the professional librarians had received. In the unobtrusive question test, the student reference assistants could not pro- vide a suitable answer to seventeen of the seventy-five questions asked (see figure 3). Of these, they referred eleven ques- tions (65 percent), but did not refer seven unanswered questions (35 percent). Stu- dent reference assistants may recognize their own limitation, but over a third of the time they do not seek more informed help for patrons. It appears that when student reference assistants find some information, even if it is incorrect or incomplete, they terminate the interview. Their lack of subject exper- tise in many of the disciplines they are re- quired to service may leave them unable to assess accurately the appropriateness of the information found. A satisfactory re- ferral program needs to consider more than just the student reference assistants' ability to find information. It is not just some information that is important, but correct and complete information. Compounding the problem of assuring an acceptable referral situation are the stu- dent reference assistants' perceptions about the availability of the professional li- brarians when they were needed for refer- rals. Although the working relationship between the professional librarians and the student reference assistants and de- partment assistants is perceived to be quite positive (see figure 6), over half the student reference assistants commented on the lack of subject specialist availability when they needed to make referrals. The subject specialists also estimated receiving less than two referrals per hour for all the hours they worked and less than one per hour during assigned backup hours dur- ing a typical midsemester week. They also reported receiving fewer than one ap- pointment negotiated at the reference desk per week. Evaluation of Reference Desk Service 473 Result No. · Percent Positive Results Correct and complete 24 32 Referred appropriately ~ __i Total 27 36 Negative Results Correct but incomplete 19 25 Incorrect 12 16 Unable to answer 12 23 Total 48 64 FIGURE3 Correctness of Answers Given Proxies It is impossible to determine from the data available whether the low number of referrals was caused by the lack of avail- ability of the subject specialists or the fail- ure of the student reference assistants to refer as often as needed. But the data do indicate a problem of too few referrals. Be- cause a satisfactory referral system is es- sential to the reference system used in the library, steps must be taken to create a more workable referral environment to overcome the deficiencies discovered through this study. ''It is important for the library to es- tablish objective standards or expec- tations for student reference assis- tants that can be applied uniformly in evaluating their performance and in developing corrective measures to improve that performance.'' Overall Effectiveness The subject specialists and department assistants gave very similar ratings (see figure 4) to the student reference assis- tants on approximately twelve factors im- portant to their effectiveness as reference assistants. The greatest discrepancy be- tween the subject specialists' and the de- partment assistants' ratings is in the ~bil­ ity of the student reference assistants to give effective service in the department, which was the second lowest rating by the subject specialists (2. 7) and the highest rating by the department assistants (4.2). Clearly, there is not universal agree- ment on the standards upon which these perceptions are based. It is important for the library to establish objective standards or expectations for student reference as- sistants that can be applied uniformly in evaluating their performance and in de- veloping corrective measures to improve that performance. Work Environment Another important factor affecting the quality of reference service is the work en- vironment. The student reference assis- tant survey specifically addressed this is- sue. Responses from the patron and proxy surveys also give additional insight. WorkLoad The student reference assistants were asked to evaluate their work loads at the reference desks in relation to the flow of patrons requiring service. The patrons 474 College & Research Libraries July 1989 Subject Department Skill or Trait Specialists Assistants a. Subject knowledge *3.0 4.0 b. Knowledge of reference books 3.2 3.4 c. Question negotiation ability 2.8 3.6 d. Developing search strategies 2.4 3.6 e. Question referral ability 3.0 4.0 f. Awareness of library policies 3.6 4.0 g. Relying on one another for help 3.5 4.0 h. Enforcing library policies 3.1 3.8 i. Ability to help patrons under- stand library systems 3.4 4.0 j. Overall ability to give effect- ive service in department 2.7 4.2 k. Overall ability to give effect- ive service in your subject areas 2.8 n/a 1. Backing up other student reference assistants n/a 4.0 Composite Rating 2.8 3.8 *l=Very weak; 5=Very strong FIGURE4 Effectiveness of Student Reference Assistants as Perceived by Other Department Staff and proxy patrons reported on the avail- ability of the student reference assistants to offer assistance at the desk. Figure 5 compares reported availability with the student reference assistants' evaluation of their work load. Eight-six percent of the time the proxies received help either im- mediately or within three minutes. Eighty-five percent of the time the patrons received help within three minutes. These results correspond with the student refer- ence assistants' perception that the de- mand for patron services, at 3.3, is neither too light nor too heavy. In general, the stu- dent reference assistants felt c9mfortable with their work load and their ability to perform both their reference and other de- partmental responsibilities in the time they had available. However, some student reference assis- tants commented that at times patron ser- vice suffers because student reference as- sistants do not take sufficient time to help one patron when another one is waiting. Often they are not able to return to the first patron for follow-up. Consequently, the appearance of service is good, but in real- ity the student reference assistants know that more complete service could have been provided had there been sufficient Evaluation of Reference Desk Service 475 Availability Work load SRA was Reported by Duty Rating Proxy Patron Patron service 3.3* Other duties 2.8 ~ !! No. !! Free to help 31 41 53 55 Helping another patron** 26 34 29 30 Working on project** 8 11 n/a n/a Away from desk 8 11 9 9 Other _g_ 2 ~ ~ Total 75 100 97 100 •1=Too light; 5=Too heavy ** less than 3 minutes wait FIGURES Evaluation of Availability and Work Load As rated by Working relationship with SRAs DAs Department chair 5.0 Subject specialists 4.2 4.6 Department assistants 4.4 n/a** Student reference assistants 4.4# 4.6 Being treated fairly 4.5 4.8 #with other SRAs in department **only one DA in each department *l=Very negative; 5=Very positive FIGURE6 Working Relationships time or additional desk personnel. Emotional Climate Another factor affecting the quality of the reference service is the emotional cli- mate under which the staff work (i.e., their relationships with others in the de- partment, feeling appreciated, and feeling important to the department). Both the student reference assistants and the de- partment assistants ranked the emotional climate as quite important (see figure 6) considering their relationship to the vari- ous groups of people they work with and 476 College & Research Libraries their feeling of being treated fairly in work assignments as "positive" or "very posi- tive." Significantly, however, the lowest ranked relationship is that between the student reference assistants and the sub- ject specialists, suggesting that steps need to be taken to provide more training and periodic social interaction for the student reference assistants with the professional librarians. An improved relationship may improve the referral of patron queries to the subject specialists for professional help. STUDENT REFERENCE ASSISTANT TRAINING The major reason for studying the effec- tiveness of the reference service provided by nonprofessional student reference as- sistants and department assistants was to determine the quality of reference service and to suggest changes; therefore, the study addressed the vital issue of training. This becomes a critical issue when stu- dents are hired to perform what the library profession considers a professional re- sponsibility, requiring substantial educa- tion, subject expertise, and a broad back- ground. The study focused on preemployment preparation and background for the job, training topics found most helpful for ini- tial and in-service training, and training methods and practices. Preemployment Background Most undergraduate student reference assistants were studying toward degrees in the subject areas of the library where they worked. However, some graduate student reference assistants were in the master's of library science program and did not always have an undergraduate de- gree related to their work assignment. The supervisors generally felt that these stu- dents' library training and motivation compensated for possible subject knowl- edge deficiencies. In addition to their aca- demic major, the student reference assis- tants also felt that almost any academic study, work, or personal relations experi- ence was valuable in preparing them to deal effectively with library patrons. The success of the student reference assistants July 1989 is strongly influenced by the relevance of their preemployment preparation to the job. Initial Training The surveys of all three types of refer- ence personnel asked about topics and priorities for initial training, in-service training, and methods of instruction. Al- though not all survey questionnaires asked the same questions in the same way, many comparisons were possible. Initial Training Topics. Because it is im- portant for student reference assistants to be properly grounded early in their refer- ence experience, the study focused on topics for initial training-those presented during the first month of employment. The responses (see figure 7) probably were prompted by the anxiety every new employee feels when assigned to the ref- erence desk for the first time. Before they assist their first patron they want to know initial job expectations and routines, how to treat the patron, how much service to provide, and what sources will answer the questions. While the student reference assistants were mainly concerned with training that would improve their competency at the reference desk, the department assistants, who oversee the department routines and the students, placed more emphasis on the supervisory and management func- tions. The subject specialists emphasized the importance of knowing the desk routines, policies and procedures, and effective question negotiation. Their comments also showed a strong concern with giving good service and the need to be trained adequately to provide that service for the long term. In response to a question about ''com- plex'' search strategies (not on the student reference assistant survey), the subject specialists and the department assistants disagreed about having the student refer- ence assistants trained to handle the more difficult research problems. The subject specialists preferred to have the hard questions referred to them rather than train the assistants. Because the specialists are often not available, the department as- Evaluation of Reference Desk Service 477 Topic DAs SRAs Ave. SSs Reference Sources at Desk 4.9 4.95 4.4 Patron Service Priorities 5.0 4.7 4.85 n/a Desk Routines 4.6 4.9 4.75 Reference Sources on Index Tables 4.6 4.9 4.75 4.4 Question Negotiation Skills 4.8 4.6 4.70 Search Strategies for answering quest.4.4 4.9 4.65 3.8 Computer Systems 4.6 4.7 4.65 4.4 Job Expectations 4.8 4.4 4.60 n/a Tour of the floor 4.6 4.5 4.55 n/a Library Policies and Procedures 4.6 4.3 4.45 Reference Sources in Reference Stacks 4.0 4.5 4.25 4.2 Card Catalog and Card Indexes 4.2 4.2 4.20 4.4 Department Organization 4.8 3.4 4.10 4.4 Subject Collections in Stacks 3.4 3.9 3.65 n/a Tour of Other Areas 3.8 3.4 3.60 Reference Sources on Other Floors 3.4 3.7 3.55 n/a Composite Rating 4.4 4.4 *l=Not important; 5=Very important Underlined numbers under SSs correlate with other columns. FIGURE 7 Compared Rankings of Training Topic Importance for Initial (First-Month) Training of Student Reference Assistants sistants felt the students should be trained in the complex strategies themselves, or at least have an opportunity to review the strategies used by the professional to solve a problem so that when a similar question is asked in the future and the professional is not available, the student reference assistants can give proper ser- vice. Initial Training Effectiveness. All reference departments train the student reference assistants when they are hired, but the li- brary has no organized program for such training. Each department develops its own training program to meet its per- ceived needs. However, the student refer- ence assistants and department assistants differed in their perception of the effec- tiveness of this training. Significant im- provements can and should be made in the training program. In spite of the lower ratings on several factors (see figure 8), the student reference assistants felt (and the department assis- tants agreed) that they are adequately pre- pared to function effectively at the refer- 478 College & Research Libraries July 1989 Differ- Training Factors OAs SRAs ence Prepared adequately to function 4.2* 4.1 • 1 Covered areas they needed to know 4.4 4.2 .2 Provided at appropriate time 4.2 3.9 .3 Side-by-side, on-the-job instruction 't.4 3.9 .5 Structured and clearly organized 4.4 3.8 .6 Supported with useful exercises 4.2 3.5 .7 Effective general library orientation 4.0 3.2 .8 Presented in understandable way ~ u ~ Composite Rating: 4.3 3.8 .5 *1=Poor; 5=Excellent FIGURE 8 Effectiveness of Initial Month's Training Programs ence desk. Their having, so far as they know, functioned satisfactorily thus far with whatever training they received probably justifies their feeling that they have been prepared adequately. Another possible explanation is that neither of these groups may be fully aware of the competency expected by the subject spe- cialist for their disciplines (see figure 4). This perception is supported by the 36 percent correctness of the answers given by the student reference assistants in the unobtrusive question test (see figure 2). A number of student reference assis- tants commented on the need for the pro- fessional librarians to become more in- volved in the training programs so the level of effectiveness can be raised. Such involvement would require a greater time commitment by the subject specialists in preparing training materials and interact- ing with the student reference assistants than is currently being done, but it is es- sential if student reference assistants are to provide effective reference service. . In-service Training Methods. Weekly training meetings were the most often used in-service training method (men- tioned by all respondents) and received a relatively high effectiveness ranking (see figure 9). All the other training methods (except for memos, notices, and one-on- one instruction) are probably used some- time during the weekly meetings. If the group sessions involve the subject special- ists who show-and-tell, review search strategies, and provide follow-up with ex- ercises for practical application, then the staff meeting could be the vehicle for im- plementing the methods deemed most important. The department assistants and the stu- dent reference assistants agreed that the most useful method of in-service training is the one-on-one, side-by-side personal instruction. They also agreed on the im- portance of the subject specialists being involved in that method of training and giving them instruction in research strate- gies. One representative comment from a student reference assistant showed how strongly they felt about such a training regimen: Subject librarians should try to have more dia- logue with student workers-this would create a better working relationship, and both would learn more about the other's work. Student reference assistants also recom- mended the use of exercises or problems that give them hands-on experience: Evaluation of Reference Desk Service 479 Times Dept. Student Aver. Training Methods Used Asst. Assist. One-on-one personal instruction 27•• 5.0• 4.5 4.75 Weekly reference staff meeting 33 3.6 4.1 3.90 Instruction in research strategies 27 3.8 4.0 3.90 Instruction from subject specialists 31 3.8 3.8 3.80 Worksheet exercises or problems 25 4.0 3.6 3.80 Show-and-tell of reference sources 28 3.8 3.7 3.75 Subject bibliography handouts 26 3.8 3.5 3.65 Posted memos or notices 27 3.5 3.3 3.40 Orientation visits to other levels 21 3.3 3.5 3.40 Guest speakers from other areas 18 ~ u 3.35 Composite Ratings: 3.8 3.7 3. 74 ••33 total respondents •1=Not useful; 5=Very useful FIGURE 9 In-service Training Methods Used and Their Usefulness Provide a written training manual with library policies, as well as specific sources and strate- gies for our subject areas. The manual should include exercises that give hands-on experi- ence. The show-and-tell approach was con- sidered fairly effective, but it would be more so if used in conjunction with practi- cal exercises, more graphics, and better written explanations of the search strate- gies that relate to the title and concepts presented. Methods involving less personal interaction-posted memos, notices, and handouts-were not considered as effec- tive. The two groups also agreed that in- formation about other levels of the library is not a high priority. However, the lack of adequate referrals noted in the unobtru- sive question test suggests that student reference assistants may not be fully aware of the importance of being in- formed about the resources and services available on other floors. The data and comments on training pre- sented above indicate that the most effec- tive methods seem to be those that make it possible for the new employee to interact one-on-one with experienced personnel and have follow-up problems or exercises to get them involved with the sources and strategies. However, there is no substitute for on-the-desk experience. As one stu- dent reference assistant remarked: ''Expe- rience and time are the two best ways of learning the reference desk, provided that one is trying to improve all the time." Many of the statements made by the stu- dent reference assistants indicated a strong desire to have their desk experi- ence include more involvement with the subject specialists so their training could be enhanced. They suggested an appren- ticeship environment where they could benefit from the experience and the exper- tise of the professionals. "The decision to remove profes- sionallibrarians from assigned hours at the reference desks and to replace them with student reference assis- tants was very controversial." Most departments in the library have a large number of disciplines for a short- term student employee to master. Unfor- tunately, student reference assistants may graduate by the time they achieve the level 480 College & Research Libraries July 1989 Perceived Value Yes No % Yes Y. No Monitor patron demands 15 3 83% 17% Keep abreast of new reference tools 13 5 72% 28% Train/monitor reference assistants 16 2 89% 11 Yo Strengthen esprit de corps 12 6 67% 33% Strengthen ties to academic department 8 8 50% 50% Receive more referrals from desk 12 6 67% 33% Improve overall reference service 13 5 72% 28% FIGURE 10 Rating of Values of Professional Librarians Working Assigned Reference Desk Hours of competency that would permit them to provide excellent service. They sometimes become frustrated by the lack of adequate training and professional support they feel is necessary for them to be satisfied with the service they provide. PROFESSIONAL LIBRARIAN REFERENCE DESK SERVICE ROLE The decision to remove professional li- brarians from assigned hours at the refer- ence desks and to replace them with stu- dent reference assistants was very contro- . versial. The study surveyed all nineteen li- brarians involved in reference work to obtain the following information: 1. Their attitudes toward assigned desk hours 2. Their attitudes toward reference ser- vice as a professional responsibility 3. The role of assigned reference desk hours in maintaining and improving their professional competence 4. The role of assigned reference desk hours in maintaining and improving reference service in the library The subject specialists overwhelmingly (see figure 10) felt that having professional librarians work assigned hours at the desk would have direct and substantial impact on improved overall reference service. The data also show that monitoring pa- tron demands, keeping abreast of new ref- erence tools, training and monitoring ref- erence assistants, strengthening esprit de corps, and receiving more referrals from the desk would also be improved with professionals working assigned desk hours. The subject specialists themselves can obviously provide a higher quality refer- ence service at the desk than nonprofes- sionals because of their expertise and ex- perience. Ye~ working at the desk to train and monitor student reference assistants was the highest rated benefit the subject specialists perceived from assigned desk hours. As one observed, "Not working side by side with student assistants denies them the benefit of my experience.'' Cer- tainly, increasing their contact with the department assistants and student refer- ence assistants would enable them to en- gage in ongoing training and to act as role models for the other staff, preparing them to function more effectively during the hours they are alone on the desk and help- ing to relieve the frustration that results from the present lack of interaction. But the subject specialists also felt that their own reference expertise and capabil- ity suffered from lack of use. Seventy-two percent felt working at the reference desk would help them keep abreast of new ref- erence tools and maintain facility with other tools, and 83 percent felt it would help them monitor patron demands. One said, ''I lose track of lesser-used sources and have less feeling for what's really go- Evaluation of Reference Desk Service 481 No. Years Average of of Reference Suggested Desk Librarians Experience Hours/Week 7 1-9 2.2 7 10-19 7.5 4 20 or more 8.9 FIGURE 11 Subject Specialist Perception of the Benefit of Assigned Desk Hours and Recommended Hours · Based on Years of Reference Experience No. Years Average of of Professional Suggested Desk Librarians Experience Hours/Week 2 l-9 2.5 12 10-19 5.4 4 20 or more 10. 1 FIGURE 12 Subject Specialist Perception of the Benefit of Assigned Desk Hours and Recommended Hours Based on Years of Professional Experience ing on-patron needs, etc.'' In addition to seeing their skills slip away, several sub- ject specialists observed their job satisfac- tion deteriorating because they felt the central mission of the library-assisting patrons in filling their information needs-was declining while the subject specialists perform other assigned respon- sibilities. Some of the subject specialists said they go out to work at the desk from time to time for stimulation and a ''breath of fresh air.'' They feel the decision to remove the professional librarians from the reference desks was made without regard to the unique abilities of many of these librari- ans, their feelings of job satisfaction in a reference librarian role, their lifetime com- mitment to patron service, and their many years of experience. However, not all subject specialists agreed with having professionals as- signed desk duty. The variables that de- fined the two groups, according to the data, were length of time as a reference li- brarian and length of time as a profes- sional librarian. In general, those with more experience as reference librarians and those with more years in the profes- sion were the strongest in favor of as- signed desk hours. Figure 11 shows the average number of assigned hours recommended based on years of reference experience. Figure 12 shows the same information based on years as a professional librarian. Those recommending fewer desk hours have significantly less reference experience and are newer in the library profession. The subject specialists reported that working desk hours helps them keep up in all the subjects in their reference area, maintain their awareness of patron de- mands on library collections they are re- sponsible for building, assess the effec- tiveness of and need for library instruction of patrons, and train student reference as- sistants and department assistants. These are all important factors because all sub- ject specialists are expected to act as a backup resource at the desk during as- signed hours for all subjects, not just for 482 College & Research Libraries their own areas of expertise. Subject spe- cialists also view reference service as an important component in fulfilling their li- brary use instruction and collection devel- opment responsibilities. In general, then, the subject specialists recommend a re- turn to assigned desk hours to help keep themselves fully professional. CONCLUSION AND RECOMMENDATIONS This study has shown that removing professional librarians from the reference desks and relying solely on student and paraprofessional assistants has created se- rious problems with the reference service in the Lee Library of Brigham Young Uni- versity; the student reference assistants and department assistants felt isolated from the professionals who have the req- uisite knowledge and skills. The referral of patrons to the subject specialists worked poorly. The training given reference staff was inconsistent and sometimes ineffec- tive. And most of all, the student refer- ence assistants answered correctly and completely only 36 percent of the ques- tions asked of them in an unobtrusive test. Recommendations The problems with reference service dis- covered in this study are related to three major areas where action needs to be taken: 1. Setting standards of performance for reference service provided at the reference desks. Standards or job expectations need July 1989 to be established for question negotiation, search strategies, referrals, the amount of time to spend with patrons, and the per- centage of correct answers that will be ac- ceptable. Standards are also needed for the quantity and quality of professional time devoted to desk and desk backup re- sponsibilities. 2. Improving the training program for all personnel involved in reference ser- vice. A uniform, consistent, and soundly structured training program emphasizing an apprenticeship relationship between student reference assistants and profes- sional subject specialists should be devel- oped and implemented. 3. Involving the subject specialists more heavily in reference responsibilities. The professional librarians, with their ex- pertise and experience, should provide the foundation for improving reference service both directly and indirectly. They should be required to make a greater com- mitment to reference service in their job descriptions; they need to be more readily accessible to patrons and other reference personnel; and they need to be more in- volved in training other reference person- nel in their areas of expertise. With a constantly changing short-term reference staff, it is very difficult to create a solid foundation on which to build an im- proved reference service. Only an ongo- ing evaluation system can assure that standards and job expectations are main- tained. REFERENCES AND NOTES 1. Peter Hernon and Charles R. McOure, "Unobtrusive Reference Testing: The 55 Percent Rule," Library ]ournal111:37-41 (April15, 1986). 2. Peter Hernon and Charles R. McClure, "Library Reference Service: An Unrecognized Crisis-A Symposium," The Journal of Academic Librarianship 13:69-70 (1987). 3. See, for instance, Donald G. Frank, "Management of Student Assistants in a Public Services Set- ting of an Academic Library," RQ 24:51-57 (Fall1984); Charles A. Bunge, "Library Education and Reference Performance," Library ]ournal92:581 (April15, 1967); Laura M. Boyer and William C. Theimer, Jr. "The Use and Training of Nonprofessional Personnel at Reference Desks in Selected College and University Libraries," College & Research Libraries 36:193-200 (May 1975); and Martin P. Courtois and Lori A. Goetsch, "Use of Nonprofessionals at Reference Desks," College & Research Libraries 45:385 (September 1984). 4. Courtois and Goetsch, p.391. 5. Richard B. Johnson, "Determing Training Needs," in Training and Development Handbook, ed. Rob- ert L. Craig and Lester R. Bittel (New York: McGraw-Hill, 1967), p.36. Evaluation of Reference Desk Service 483 6. Beth S. Woodward and Sharon J. VanDerLaan, "Training Professionals for Reference Service," The Reference Librarian no.16:233-54 (Winter 1986). 7. As defined by Peter Hernon, a management study, as opposed to a research study, uses techniques that provide indicators rather than strictly valid statistical data that can be broadly generalized. Management studies, however, provide valuable information for making informed management decisions. 8. The apparent discrepancy in tables 11 and 12 for librarians with 20 or more years of experience: three librarians indicated they had 20 or more years of both reference and professional librarian experience, one librarian indicated more than 20 years of professional library experience and less than 20 years of reference experience, and one librarian indicated more than 20 years of reference experience but less than 20 year's experience as a professional librarian. APPENDIX A: REPRESENTATIVE SELECTION OF THE QUESTIONS USED FOR UNOBTRUSIVE TESTING 1. I need to find the address and director of the U.S. Information Agency. 2. Where can I find out which colleges offer associate degrees in engineering? 3. I'd like to find a short statement that ex- plains passive aggressive personality. 4. I have this citation: Proc Natl Acad Sci 72(9):3453+ on salmonella. Where is it? 5. How do you use Science Citation Index to find an article on phospholipid metabo- lism? 6. What is another name for folic acid and what is its chemical formula? 7. I need to know where the Ogoki River is located. 8. I want to read about Larry King, the radio talk-show host. I need information about his life and I need a picture of him. 9. Where are your books on animation? 10. I've been looking all day for the 1986 volume of the Middle East Journal. It's not in the book stacks or on the sorting shelves. Is there any place else I can check for it? 11. I need brief biographical information on important people in Italian history. 12. I need the address of the Lee County (Al- abama) Historical Society. 13. I need to know how many people in the world speak Spanish. 14. For my art history class I need to find out what the Ash Can School of painting is and who some of the artists who be- longed to it are. 15. When did they start rating films with "G," "PG," "R," and "X," and what is the official name of this system? Examples of Escalator Questions The correctness of answers given to these questions was determined by whether or not the desk assistant negotiated adequately to ar- rive at the "true" question (step 3). 1. Step 1: Where are your education ency- clopec:tias? Step 2: I need information on test scores. Step 3: I want to compare the ACT and SAT test scores. 2. Step 1: My teacher told me to look in the PDR to get some information on codeine. Where can I find it? Step 2: I need to know if it is habit form- in g. Step 3: I take Tylenol with codeine and want to know if I should be wor- ried about the codein:e in it. 3. Step 1: Where can I find Time and Newsweek magazines? Step 2: I need articles from the last few months. Step 3: I need information on the fifth amendment stand of Oliver North. Meet the innovators of the twentieth century ... Who,s Who in Technology Get in touch with over 387000 technology leaders in the all-new Who's Who in Technology. This 6th edition has increased coverage and comes to you in a new size. It's now published in two volumes (previously published in seven volumes) to save you money and shelf space-important con- siderations for librarians today. In addition, you now get over 387000 entries-an increase of 27000 over the previous edition. Entries provide everything you need to know about a particular biographee-complete career, educational7 and personal data. The indexing is extensive-more than any other publication of its kind. Locate potential consultants, job candidates, or colleagues by employer7 geographic location, principal expertise7 or technical discipline. 6th Edition. Edited by Amy L. Unterberger. About 2,700 pages in 2 vols. ISBN 0-8103-49 50-7. Order #20710-99884. $380.00. If your collection is missing this all-important reference source, order today by calling toll-free 1-800-223-GALEI Gale Research Inc. Book Tower • Dqll. m48 • lkrroit, MI 48zn.o748 WHO'S WHO IN a1!Wit9l-ft~