ACRL News Issue (B) of College & Research Libraries J a n u a ry 1 9 9 3 / 1 3 Evaluating the library BI program By Gail Z. Eckwright S tu d en ts’ com m ents shed some light on a program ’s effectiveness E valuating a bibliographic instruction (BI) program is a difficult task. How do you determine w hether or not the program is a suc­ cess? At the University of Idaho Library w e d e­ cided to subject our BI program to some scru­ tiny. We concluded that if w e expected students to listen to us, then perhaps they deserved to be heard as well. Consequently, w e devised an evaluation form w hich provided us with some interesting insights into our BI program and into the students’ perspectives on the pro­ gram as well. O u t w ith th e o ld University of Idaho librarians have long been giving formal “tours” and instruction to any and all comers. However, in the 1989 fall semester w e em barked upon a new program o f biblio­ graphic instruction aimed directly at the 975 students enrolled in the English 104 classes. We overhauled the library tour format that had been in use for more than ten years. We replaced the tours with two 30-minute bibliographic instruction sessions. The new BI is conducted in a remote area of the library, with chairs for all of the students. An LCD (liq­ uid crystal display) panel and an overhead pro­ jector are used to demonstrate the com puter­ ized portions of the instruction. During the first 30-minute session LaserCat is taught to the stu­ dents using the LCD panel and projector. The second 30-minute session is devoted to teach­ ing periodical indexes using overhead trans­ parencies made with PageMaker software. At the end of each instructional session the stu­ dents are given a written assignment that can be com pleted in 10-15 minutes. The assign­ m ent is then returned to the librarian w ho gave the instruction and ultimately returned by the librarian to the English 104 instructor. This method of BI does require the full co­ operation of the English Department and all of its instructional assistants (IAs) and faculty. The director of writing w ho oversees the IAs has given this program her w hole-hearted support, so participation has been at the 100% level! Not all of the instructors (and therefore, not all students) are as enthusiastic as the director of writing, however. During the 1989/90 aca­ demic year w e used a brief evaluation form to get some idea o f the students’ reactions to the BI program. E v a lu a tin g th e n e w Completion of the evaluation forms was o p ­ tional. Some instructors encouraged their stu­ dents to return the forms to the librarians; other instructors did not give the forms to the stu­ dents, and hence the evaluation process was halted dead in its tracks. We received 778 completed evaluation forms for the year. This figure represents about a 41% return rate. We had 12 librarians participating in the pro­ gram. The instructional format was consistent from librarian to librarian. Differences in teach­ ing methods, styles, and techniques certainly account for some of the comments made by the students. Also, every class had students w ho w ere disenchanted with the very idea o f library instruction, and their remarks and ratings usu­ ally w ere predictably low. Generally speaking, the instruction was considered a success if a majority o f those students returning the sur­ veys gave a 5 or better on the evaluation scale w here 1 was low, 7 was high, and 4 was in the middle. Gail Z. Eckwright is hum anities librarian a t the University o f Idaho, Moscow 14 / C&RL News The confidence q uo tien t Question 1 on the evaluation form had three parts. The first part, which could be referred to as the “confidence quotient” asked: After com­ pleting the library training session and related assignments, how confident do you feel about your ability to identify and locate desired peri­ odicals and books at the U of I Library? Of the students responding to this question, 137 (17.6%) gave a 4 ( neither “very confident” nor “not very confident” mark). In this same category, 80 students (10.2%) gave a 3 or be­ low mark, indicating that after the library ses­ sions they were not very confident about using We were told that V andal football is #1, that we needed haircuts, should provide doughnuts a n d coffee, a n d to f i n d chairs that d id n ’t “m ark us fo r life. ” the library’s catalog and indexes. The good news was that 561 students (72.1%) gave marks of 5,6, or 7, indicating that they felt confident and very confident about using the library’s catalog and indexes. Also, our success rate improved from the fall semester to the spring semester. In the. fall, 14% of the students gave a 3 or below in the confidence quotient; in the spring only 6% gave those marks. The confident and very confident students increased from 66% in the fall to 80% in the spring. The indication here is that perhaps our instruction improved as w e became more familiar with our “new and improved” teaching methods. Other parts to question 1 asked “How ef­ fective were the sessions? (a) LaserCat and (b) Periodicals.” The responses to these questions showed a great deal of variance from student to student. One librarian might receive a 2 from one student. Another student in the same class might give a 7. These wide-ranging responses made these survey questions more difficult to quantify. Perhaps at best the responses here can be attributed to learning theories and mo­ dalities; a teaching method that appeals to one student may not appeal to another. A librarian w ho examined these questions could determine in general how well a session w ent and per­ haps consider making some changes if a ma­ jority of the evaluations were at the 4 or lower range on the rating scale. Som e com m ents (thoughtful a n d o th e rw is e ) The last three questions on the evaluation form asked for narrative comments. These questions p ro v id ed for som e thoughtful, as w ell as thoughtless, answers. The questions were: What w ere the most valuable parts of the sessions? What can we do to improve the sessions? Any other comments? It is difficult, it seems, for any humans (es­ pecially college freshmen) to ignore an invita­ tion to make anonymous comments after be­ ing required to sit through instruction. These students were no exception. We were told that Vandal football is #1, that w e needed haircuts, should provide doughnuts and coffee and pop­ corn, and to find chairs that didn’t “mark us for life” (also known as waffle-bottom chairs). For­ tunately, many of the comments were more helpful and enabled us to improve our instruc­ tion. For example, some students indicated that a tour after the sit-down instruction would help them understand w here and how materials were located in the library. Some librarians then incorporated a modified tour into the instruc­ tion, and found that it did help the students. Some students picked up minor tidbits about the library, such as information about govern­ ment documents or interlibrary loans, which they found useful. The following excerpts from the students’ comments indicate that some of them appreci­ ated the sessions, but they saw some negative aspects about them, too: “Make them longer. Make students do more examples.” “I see no need for sessions. If you can read, the computer is easy.” “I hope if I can’t find what I’m looking for someone will be there to help.” “I did not learn anything from the sessions. All that was talked about I’d already learned on my own with just a little time spent in the library.” “Extend the sessions to a couple of days each.” “Try to make it exciting—it was tooo boring.” “Make it a little more challenging—it was very easy.” “Library work is just not the most exciting crap I’ve done.” “I felt assignments were good, because they gave you a little on hands experience.” “D on’t insult are [sic] intelligance [sic] so J a n u a ry 1 9 9 3 / 1 5 much. We are not stupid, shorten the LaserCat section. If a person can get onto the com puter they are easy to use. I figured them out last year in about five m inutes.” M o r e fe e d b a c k Other comments w ere somewhat more encour­ aging: “I understood everything until I w ent to do it.” “Most p eo p le cannot learn to look things [up] just by w atching a dem onstration. G ood effort bu t I think it is up to the students to go in and get experience w ith the system after th e to u r a n d s e s s io n s . E x p e r ie n c e = k n o w led g e.” [The most valuable part was] “Learning how to find stuff on the LaserCats. Before they w ere real confusing.” “It’s very helpful. Maybe m ake m ore inter­ esting so p eo p le d o n ’t blow it off. The ses­ sions are very helpful—b u t students will only get o u t of it w hat they w an t to— I d id n ’t re­ ally pay attention the second day, b u t th a t’s my problem .” “W hen w e w ent to actually use the equip­ ment we w eren’t stressed because w e d idn’t have a big paper due. We could ask questions w ithout feeling embarrassed or impatient.” “I think it’s great that you show students how the library is available for them!” “If I had no idea how to use the library/ LaserCat it w ould have been a good thing for me but I have b een using the LaserCat since it was put into the library because I found it much easier to use than the card catalog.” “I had trouble running the LaserCat before, but now it is a snap.” G e n e ra l com m ents Then there w ere the general comments regard­ ing the facilities, the teaching methods, etc.: “Can w e have a more comfortable room?” “Go slow er.” “D on’t hold the meeting in the dugeon [sic].” “Make them m ore to the pt. Just tell how to do things—w ho cares about the rest o f the stuff.” “Thank you, it did help.” “Go back to card catalogs.” “I think you needed to have more helpers w hen w e perform ed the tasks [assignments].” “Allow people w ho already know how to use the library the chance to test out.” “Offer it earlier [in the semester] and to all U of I students.” “Walk us through the library.” “D on’t treat students like grade schoolers.” 1 6 / C&RL News O n a n up note And, of course, the best part of the comments were those more positive and encouraging re­ marks, such as: “This will help in further research.” “LaserCat presentation on the overhead was effective.” “Now I know the magazines are o n all floors!” “I didn’t know about the indexes for the periodicals.” “I’m glad someone is kind enough to give us some idea of what the library is really for.” “I never realized how big the library really was until then & how much information they had.” “I am glad we took time as an English class to learn about the library otherwise I would really be lost.” “Helpful librarians." “I learned a lot, thank you! Before my con­ fidence level w ould have been a 1.” [Confi­ dence level=5] “It’s a nice library, the sessions did help.” Is it w o rth it? The many varied comments indicate that the students’ reactions cannot easily be summarized. The evaluations did show more favorable com­ ments than criticisms, which was gratifying to us as librarians and teachers. Given the varied response to the library instruction, is it worth our time to continue with the BI program? Ab­ solutely. If nothing else comes of it, students meet with one or two librarians. They learn our faces and perhaps our names, so that w hen they return to the library for help, something or som eone is familiar to them. The library becom es a little less impersonal and a little m ore approachable. And as the com ments indicate, the students do learn from the BI sessions. One final individual evaluation form further points to why we should continue. To the ques­ tion “What can we do to improve the sessions?” the student responded: “Omit them entirely. I didn’t go to the ses­ sions—I just am assuming that they are a waste of time as most sessions in libraries usually are. I, personally, always feel lost in a library.” I have no doubt that the English 104 ses­ sions are a necessary com ponent in our library BI program. At the UI Library we will continue to work at improving both the program and the presentation so that students no longer need “feel lost.” ■ DEFINITIVE BIOGRAPHY. THE BRITISH SCANDI­ ARCHIVO DEUTSCHES BIOGRAPHI­ NAVIAN BIOGRAFICO BIOGRA­ CAL BIOGRAPHI­ DE ESPAÑA, PHISCHES ARCHIVE II CAL PORTUGAL E ARCHIV, A continuation of ARCHIVE IBEROAMÉR- NEUE FOLGE the first British Bio­ The archive brings ICA II This Supplement to graphical Archive. together over This Spanish, Portu­ Deutsches Biogra­1991-1993 130,000 entries from guese, and Latin phisches Archiv 12 installments Scandinavia. An (DBA I) comple­American bio­775 (approx.) fiche accumulation of ments its predeces­graphical archive is Silver .........$11,600* 360 biographical sor by providing a continuation of Diazo ........$10,500* source works, unrivalled breadth the first Archivo. Prices are tentative. divided into two 1991-1993 of coverage to THE BRITISH sections: one for 12 installments personalities from Denmark, Norway, BIOGRAPHI­ 1,000 (approx.) fiche Germany's early and Iceland; the (24X) history to the mid­ CAL INDEX other for Sweden twentieth century. Silver ........ $14,800* and Denmark. Fol­ Covering some A quick reference Diazo ....... $13,400* 280,000 individuals source to the BBA, lowing a compli­ Prices are tentative. this set is also a cated pattern of from over 260 stand-alone refer­ change in leader­ INDICE biographical ence ideal for use ship and national BIOGRAFICO sources published between 1800 and as a biographical boundaries, the dictionary. archive traces a DE ESPAÑA, 1960, D BA II 1991 long history of fas­ PORTUGAL E represents a new standard in German 0-862-91390-X/4 cinating personali­ IBEROAMÉR- volumes/ approx. ties. biographical refer­ICA ence.1,600 pages/$995 1989-1991 12 installments 1990/3-598-32060-4 Price is tentative. 1989-1991 800(approx.) fiche 4 volumes; 2,400 12 installments * Prices require payment (24X); multi-volume pages/$800 1,300 (approx.) fiche in fu ll on receipt o f first index. (24X). shipment. S ilv e r......... $11,600* Silver .........$11,600* Diazo..........$11,000* Diazo ........$10,200* For more inform ation about other Biographical A rchives, contact Walter Jaffe. K.G. Saur • 121 Chanlon Rd. • New Providence, NJ 07974 A Division of R. R. Bowker