ACRL News Issue (B) of College & Research Libraries


S eptem ber 1983 /  271

Bibliographic Instruction

Evaluating Bibliographic Instruction: 
A  Handbook

E d ito r’s Note: C& RL News is pleased to have the 
opportunity to reprint (w ith  permission) the Con­
tents, Preface and Introduction to the new  hand­
book published b y the Bibliographic Instruction 
Section, E v aluating B ibliographic Instruction: A 
H andbook. The 129-page publication was coordi­
nated b y the BIS Research C o m m itte e ’s Subcom ­
m ittee on Evaluation, whose m em bers are: M ig­
non S. A dam s, D avid Carlson, Bonnie G. Gratch, 
L a r r y  H a r d e s ty , D a v id  N . K in g , J o h n  M a rk  
T ucker, R ich a rd  H u m e  W erkin g , and Virginia 
T ie fe l (c h a ir ). I t  is n o w  a v a ila b le  f r o m  
A C R L /A L A , 50 E. H uron St., Chicago, IL  60611, 
a t $13 f o r  A C R L  m e m b e rs , a n d  $17 fo r  n o n ­
m em bers.

C o n t e n t s

“E valuation and Its Uses,” D avid N. King.
“E v aluating in Term s of Established Goals and 

O bjectives,” V irginia Tiefel.
“ R ese a rc h  D esigns A p p r o p ria te  fo r B ib lio ­

graphic In stru ctio n ,” Bonnie G. G ratch.
“D a ta-G ath e rin g  In stru m en ts,” Mignon S. Ad­

ams.
“D a ta  M anagem ent an d  Statistical Analysis,” T. 

M ark Morey and Jacqueline Reihm an.
“Significant W orks,” R ichard H um e W erking.
“G lossary,” John M ark Tucker.
“B ibliography” (Part I, A nnotated List; P a rt II, 

Basic Textbooks).

P r e f a c e

Because the rap id  grow th of bibliographic in­
struction is a com paratively recent phenom enon, 
m any instruction librarians have simply not h ad  
th e  tim e to ev alu ate w h a t they are doing; and 
m any lack the specialist knowledge about, or expe­
rience in, program  evaluation. This handbook has 
been w ritte n  by librarians (w ith one exception) for 
librarians both to offer an introduction to basic 
precepts cf evaluation and to furnish some direc­
tion to move beyond th e m aterial covered in the 
handbook.

T he subtitle of A H andbook has been chosen be­
cause the authors wish to stress th a t this is not the 
definitive w ork in lib rary  instruction evaluation. It 
is an introduction to the topic, intended to give 
general direction, and especially encouragem ent, 
to librarians attem p tin g  evaluation in this im p o r­
ta n t area.

T he handbook is a collection of chapters on v ari­
ous aspects of evaluation and each reflects the a u ­

th o r’s approach to th e subject and style of w riting. 
T here is, th en , diversity in th e w riting; and m any 
w ho critiqued th e m anuscript noted some inconsis­
tency in style. But as there was no agreem ent on a 
uniform  style (e.g., betw een m ore form al and in ­
form al styles), th e decision was m ade not to a t­
te m p t a m ajo r ed itin g  of th e m a n u scrip t. I t is 
hoped th a t readers will not be distracted by this, 
an d  th a t, indeed, they m ight enjoy it. Also, the a u ­
thors invite com m ents and suggestions, all of w hich 
w ill be retained an d  referred to th e authors of a re­
vised edition.

In  closing, as chair I w a n t to th an k  th e m em bers 
of th e com m ittee responsible for th e creation of this 
h a n d b o o k . No fin e r g ro u p  of p ro fe ssio n a ls— 
w o rk in g  over a long tim e an d  d ista n ce—could 
have been found. T he contributions of several peo­
ple beyond th e com m ittee proved invaluable. They 
are K atherine Branch, W illiam  C row e, E van F ä r­
ber, C onstance Finlay, E lizabeth Frick, M artin 
Gibson, Jam es Kennedy, Tom  Kirk, M aureen Pas- 
tine, L inda Phillips, R onald Powell, D aniel Ream , 
Anne Roberts, and N ancy Taylor.

BI E v a lu a tio n  P ro g ra m

T h e  n ew ly  p u b lish e d  E v a lu a tin g  B ib lio ­
graphic Instruction: A  H andbook (see the In ­
troduction rep rin ted  in this issue) will be p re ­
sented for the first tim e to a m ajor gathering of 
librarians at th e M idwest F ederation of L ib rary  
Associations C onference in C in c in n ati. T he 
p r o g r a m , “ L ib r a r y  In s tr u c tio n : T h e  N ext 
Step—E v a lu a tio n ,” is scheduled for th e m o rn ­
ing of N ovem ber 4 an d  will feature a talk by 
M ignon A dam s, co o rd in a to r of in fo rm atio n  
services, SUNY-Oswego, a leading au th o rity  on 
lib ra ry  in stru ctio n  an d  a c o n trib u to r to th e 
handbook.

T he tw o-hour session will be led by V irginia 
T iefel, d irec to r of lib ra ry  user ed u catio n  a t 
Ohio State University, w ho served as chair of 
th e ACRL com m ittee th a t p rep ared  th e book. 
Several of th e  region’s leaders in lib rary  user ed­
ucation will assist in th e question-an d -answ er 
an d  group discussions th a t comprise th e b a l­
ance of the session.

This portion of the M FLA program  will be 
sponsored by th e Ohio L ib rary  Association’s 
Academic and Special L ibraries Division.



272 /  C &R L  N ew s

Steps in the Evaluation Process

Deserving of special thanks are M ignon Adams, 
w ho collected th e m anuscripts and supervised their 
p rep aratio n  for p rin tin g , and G lenda King, who 
p rep ared  the artw o rk  for several of th e charts and 
tables. — Virginia T ief el, Ohio State University L i­
braries.

I n t r o d u c t i o n

T he evaluation of library instruction program s 
has been a subject of much discussion during the 
past tw o decades. Although th e re seems to have 
been an increase in the num ber of published evalu­
ation studies in recent years, it is difficult to tell 
w h eth er or not this reflects any significant increase 
in th e  use of evaluation by th e m ajority of instruc­
tion librarians. E valuation is often assumed to be a 
complex, tim e-consum ing process; and for those 
u nfam iliar w ith the methods an d  tools, it may seem 
an intim idating prospect.

In  fact, evaluation is w h a t you make of it. The 
process m ay indeed take on th e aspect of a sophisti­
c ated  educational research endeavor, if th a t is 
w h a t you wish. But it is not always necessary to de­
velop large-scale, com plex projects in order to 
p r o f it fro m  e v a l u a tio n . T h e  process itse lf is 
straightforw ard, easily described in six basic steps.

Step 1. Describe the purpose of the evaluation. 
The first step in any evaluation effort is to m ake 
sure th a t you fully u n derstand the reasons for eval­
uating. W ho w ants to know and w hat you hope to 
learn from th e inform ation you obtain will d eter­
mine the kinds of inform ation you need to collect 
and how  you can best collect it. C hapter 1 ‚ “E v alu ­
ation and Its Uses,” discusses many of the factors 
w hich should be considered, and introduces some 
of the pro m in en t approaches to systematic evalua­
tion.

Step 2. Describe the program  in terms of its goals 
and objectives. Once you have a clear idea of why 
the evaluation is to be undertaken, it is im portant 
to develop a description of the program  as it cur­
rently exists. E ducational program s tend to evolve,

and statements of program  goals and objectives 
may not fully detail current practice. Be sure to 
look for any im p lied  goals an d  objectives th a t 
might not have been included in a form al state­
ment. C h ap ter 2, “E valuating in Terms of E stab­
lished O bjectives,” provides an overview of behav­
ioral goals and objectives, and explains a taxonomy 
of educational objectives.

Step 3. D eterm ine the criteria to be used for eval­
uation. W hen the program  has been described in 
sufficient detail, and the goals and objectives are 
clearly identified, evaluation criteria can be d eter­
m ined. If goals and objectives have been w ritten  
w ith care, this step is relatively easy. Just decide 
w h ic h  goals a n d  ob jectiv es sh o u ld  be stu d ie d  
w ith in  the context of the purpose of the evaluation, 
and w h a t standards w ould indicate success. But be 
sure to consider outcomes w hich m ay not have 
been anticipated by the goals and objectives, or 
w hich m ay indicate undesirable or counterproduc­
tive results of instruction. C hapters 1 and 2 include 
sections discussing the criteria th a t m ight serve as 
standards.

Step 4. Develop the evaluation procedures and 
overall design of the study. After the criteria have 
been determ ined, procedures for conducting the 
evaluation can be developed. If the evaluation is to 
be a m ajor sum m ative effort, or if you intend to 
m onitor your program  on a continuing basis by 
means of evaluation, you m ay find it useful to de­
velop an evaluation plan based upon one of the ap ­
proaches discussed in C hapter 1. An appropriate 
evaluation design should be selected at this point. 
C hapter 3 provides an overview of some com monly 
employed designs. The selection of the design will 
clarify most of the procedures to be used in the eval­
uation, w hich instrum ents m ight be adopted, and 
the kinds of statistical analysis th a t will be neces­
sary. You m ay find it helpful at this point to set 
tim e ta b le s, d e te rm in e  p ersonnel an d  tra in in g  
needs, and budget for any costs you anticipate. 
C h a p te r 3, “ Research Designs A p p ro p riate for



September 1983 /  273

Evaluating Bibliographic Instruction,” introduces 
concepts related to experimental designs and de­
scribes and illustrates their use.

Step 5. Develop instruments and collect data. 
The fifth step in the evaluation process is to develop 
the instrument or instruments to be used to imple­
ment the study. The keys to success in this step in­
volve pretesting the instrument to eliminate any 
unforeseen errors, and applying the instrum ent 
systematically. Deviations in the way data are col­
lected can irreparably compromise the entire eval­
uation process and render the data useless. Chapter
4, “D ata G athering Instrum ents,” describes a vari­
ety of instruments which may be used to collect in­
formation, and offers practical advice on choosing 
an instrument and/or developing your own tests 
and questionnaires.

Step 6. Analyze the data and report the results. 
Once the data are collected, they should be inter­
preted w ithin the context of the criteria established 
in Step 3. The analysis should point out which cri­
teria were successfully achieved, as well as identify 
components of the program th at need improve­
ment. Chapter 5, “D ata Management and Statisti­
cal Analysis,” presents the core concepts of statisti­
cal analysis and describes some of the procedures 
most useful for the analysis of instructional data.

Even if a report to outside clientele is not required, 
w riting up the result will help in interpreting and 
assessing the evaluation results.

The techniques and methodologies introduced 
in this handbook may be employed in a number of 
ways to help understand the effects of your pro­
gram and improve the quality of instruction. Care­
ful planning and attention to detail are necessary 
throughout the process.

You will not find the answers to all your evalua­
tion questions here. Each author provides refer­
ences for further reading, and an annotated bibli­
o g ra p h y  of suggested re a d in g s is in c lu d e d . 
Examples of evaluations and some of the more im­
portant sources for information on the evaluation 
of bibliographic instruction programs are included 
in Chapter 6, “Significant W orks.” A glossary is 
also provided. Even if you follow up on all of the 
sources mentioned, we encourage you to take ad­
vantage of the expertise of others on your campus.

You need not be an expert to succeed at evalua­
tion. Evaluation is not an exact science, and crea­
tivity, resourcefulness, and perseverence yield im­
pressive results. Your programs and your students 
will benefit from the effort.— David King, Hous­
ton Academy o f Medicine, Texas Medical Center 
Library. ■ ■