College and Research Libraries MARY JANE DONALDSON and JOHN F. HARVEY Library School Instructor Evaluation Since 1960 students have been asked .to evaluate all courses taken in the Drexel library school. Results of the evaluations are tabulated and implications are discussed; they are also compared with similar studies in other disciplines and at other institutions. Summer school courses were better liked than those taken during the year; women instructors scored higher than men; courses in specialized library work ranked higher than others. IN 1960 THE DREXEL INSTITUTE OF TECH- NOLOGY graduate school of library sci- ence intiated a program of evaluation of faculty members by students. Student ratings are obtained for each section taught by full-time instructors during their first three quarters at Drexel, and for the first four courses taught by part- time instructors. Normally the evalua- tions are obtained in the last class meet- ing before the final examination. The evaluations are intended to serve three purposes. Primarily, it is hoped that the ratings will aid the instructor in improving his teaching methods by pointing out specific areas needing im- provement and by revealing to the in- structor his students' reactions to him, both positive and negative, since "only by accident will the teaching of a man ignorant of the reaction of his class be effective.''1 Second, the ratings assist the administration in judging faculty mem- bers' effectiveness. A third purpose is to give students a voice in school adminis- tration. 1 W. R. Wilson, "Students Rating Teachers," Jour- nal of Higher Education, III (February 1932), 79. Miss Donaldson is Research Assistant and Dr. Harvey is Dean of the Graduate School of Library Science, Drexel Institute of Tech- nology. 470/ Since the 1920's, and especially in the last two decades, student evaluation of instructors has received increasing atten- tion from educators and researchers. In general, this research has shown student evaluation to be reliable and valid. In his survey of the subject in the Hand- book of Research on Teaching, H. H. Remmers cites various studies which have shown that student ratings were not appreciably influenced by the diffi- culty of the course, by the halo effect, by the grades given to raters, or by the instructor's popularity in extracurricular activities. Remmers cites further studies showing when twenty-five more ratings were averaged they were as reliable as the better mental and education tests, and when alumni graduated ten years earlier were asked to rate their college instructors, their ratings agreed substan- tially with those of the same instructors by students currently enrolled. 2 The use of student evaluation is ap- parently widespread: in 1960 Stecklein cited evidence that 320 colleges and universities in the United States had used student . ratings.3 Despite the amount of attention given student rat- ings on the undergraduate level, how- ever, this subject has received little at- 2 N. L. Gage, ed. , Handbook of Research on Teach- ing (Chicago: Rand McNally, 1963) , p. 367-68. 8 lbid., p. 368. Library School Instructor Evaluation I 471 tention at the graduate level and ap- parently none in library science. Yet it would seem that ratings might play a particularly important role in library education since, like instructors in other professional schools, most library school instructors have been trained primarily as professional practitioners rather than as teachers, and sometimes "the teachers drawn from the ranks of the profession are inadequately prepared for teach- ing."4 For the instructor with little ex- perience or training in teaching methods, student ratings can point out areas on which he must concentrate to increase his effectiveness. With the hope that a study of there- sults of student evaluations might pro- vide insights into the problems of teach- ing library science on the graduate level, a study was made of the five-year Drexel evaluation program. The data upon which the study was based were limited in several aspects: in the first place, since ratings were generally obtained only for part-time and full-time instruc- tors while they were still relatively new to the job, they do not necessarily pro- vide a representative sample of all sec- tions taught; second, the form of rating sheet has undergone slight modifications; 4 Thelma Eaton, "Who is a Good Library School Teacher?" Improving College and University Teaching, II (May 1954), 26. and third, in a few cases not all of the data was recorded on the master sheets. The evaluation forms filled out anony- mously by the students consisted of a list of attributes generally recognized to be associated with effective teaching. · For each attribute students were asked to rate the instructor on a four-point scale-excellent, good, fair, and poor. The rating unit was the individual class section, so if an instructor taught three class sections in a given quarter he re- ceived three separate ratings. In compil- ing the scores for each section a master sheet was made which recorded the number of times each point on the grad- ing scale was marked for each attribute. The total number of marks for each point on the grade scale was then fig- ured. The final rating score was the per- centage of all the marks which were at the excellent and good points of the scale. For instance, if thirty students in a section made a total of 200 evaluation decisions, 180 of which were at the ex- cellent and good points of the scale, the instructor's rating for that section was 90 per cent. Also recorded on the master sheet was a list of all voluntary student comments and the rank of that section compared with all other sections rated that quar- ter. At the end of the quarter-after all course grades had been turned in-a re- TABLE I. FREQUENCY DISTRIBUTION OF STUDENT RATINGS RATINGS FOR RATINGS FOR PART-TIME FULL-TIME RATINGS FOR RATINGS FOR ALL RATINGS INSTRUCTORS INSTRUCTORS WoMEN MEN ScoREs No. Per Cent No. Per Cent No. Per Cent No. Per Cent No. Per Cent --- 96-100 30 12.34 17 12.78 13 11.81 14 12.06 16 12.59 91-95 40 16.46 22 16.54 18 16.36 23 19.82 17 13.38 86-90 39 16.04 21 15.78 18 16.36 16 13.79 23 18.11 81-85 32 13.16 14 10.52 18 16.36 17 14.65 15 11.81 76-80 31 12.75 18 13.53 13 11.81 17 14.65 14 11.02 71-75 21 8.64 8 6.01 13 11.81 11 9.48 10 7.87 66-70 19 7.81 13 9.77 6 5.45 7 6.03 12 9.45 46-65 24 9.87 13 9.77 11 10.00 11 9.48 13 10.23 26-45 7 2.87 7 5.26 . . . ...... . . . . . . . ... 7 5.51 --- 243 100.00 133 100.00 110 100.00 116 100.00 127 100.00 4721 College & Research Libraries • November, 1966 port was given to the instructor on an evaluation form, and a copy was kept by the administration. The study considered separate course ratings representing 243 sections taught by eighty-four instructors. In analyzing these ratings the following questions were considered. 1. What was the range for individual instructors' scores? The scores ranged from 28 per cent to 100 per cent with the mean being 81 per cent shown in Table 1. Twenty-nine per cent of the ratings were above the 90 per cent lev- el and an equal number below 75 per cent. The widest range for the scores of an individual instructor was 47 points, from 42 per cent to 89 per cent; how- ever, the range of most instructors was considerably narrower, with the average range for instructors rated twice being seventeen points and the average for those rated three or more times being twenty-five points. Only one instructor was included in both the best twenty- five and the poorest twenty-five ratings. 2. Did a pattern of movement exist in the scores of individual instructors or the ratings as a whole? At Colorado State College of Education a survey of ratings taken over a period of years found that most instructors made signifi- cant improvements with successive eval- uations. 5 Riley cites three other studies which found a definite improvement in the performance of faculty members.6 At Drexel, however, · no such trend was ap- parent, and while some of the instruc- tors did receive successively higher rat- ings, as many received successively low- er ratings or had such varying scores as to show no pattern of movement. Therefore, while Drexel faculty mem- bers on a whole learned their teaching weaknesses their resulting teaching G W. D. Armentrout, "Relation of Class Size and Other Factors to Student Ratings of Teachers," ]ou.mal of Teacher Education, I (June 1950), 102. s John W. Riley, Pryce F. Ryan, and Marcia Lifshitz, The Student Looks at His Teacher (New Brunswick: Rutgers University Press, 1950), p. 31. seemed no more-or less-effective than before. Of course, this generalization hides the group which did improve and the group which was already very high. As a matter of fact, the mean score of 81 per cent excellent and good was it- self a quite satisfactory score. But mi- norities scored low or else had declining scores and were not invited to teach again. TABLE 2. MEAN ScoREs OF FACULTY RATINGS BY YEARS AND QUARTERS MEAN Year: N SCORE 1960-61 26 81.2 1961-62 46 80.5 1962-63 43 83.5 1963-64 60 80.5 1964-65 56 76.2 Quarter: Fall 52 79.0 Winter 54 78.8 Spring 43 78.3 Summer 77 82.5 Table 2 shows that the ratings as a whole have recently shown a downward trend. While the average ratings for the first three years ranged from 80.5 per cent to 83.5 per cent, with the academic years 1963-64 and 1964-65 the average score declined to 76.2 per cent. Whether this trend represented an increasingly critical and able student body or a some- what less effective faculty is not clear. 3. Was there any connection between favorability of ratings and time of year? In the Colorado State College of ·Educa- tion survey summer school courses were rated higher than those taught during the regular college year. 7 The same trend was evident at Drexel where summer quarter course ratings averaged 82.5 per cent while fall, winter, and spring quar- ter averages were 79 per cent, 78.8 per cent, and 78.3 per cent respectively. 7 "Relation of Class Size and Other Factors •.. ," op. cit., p. 102. Library School Instructor Evaluation I 473 TABLE 3. CoMPARISON OF PART-TIME AND FULL-TIME, AND oF MALE AND FEMALE INSTRUCTORS Mean Score Ratings: Full-Time Instructors 82.8 Part-Time Instructors 78.9 Ratings: Male Instructors 79.2 Female Instructors I 81.7 More striking than a comparison of aver- age scores, however, was the fact that while summer quarter ratings accounted for only 28 per cent of all the ratings they accounted for 68 per cent of the twenty-five highest ratings. 4. Which groups of instructors scored highest? A comparison was made of the scores received by women ( 48 per cent of all ratings ) and by men (52 per cent) . Table 3 shows the mean rating for women ( 81.7 per cent) to have been slightly but not significantly higher than that for men ( 79.2 per cent). Male in- structors made up half of the top twenty- PerCent Per Cent I PerCent of All of Highest of Lowest Ratings 25 Ratings 25 Ratings 45 40 32 55 60 68 - - - 100 100 100 52 52 64 48 48 36 - - - 100 100 100 five instructors but two-thirds of the bot- tom twenty-five instructors. A similar comparison was made be- tween part-time and full-time instruc- tors. Again the difference was slight, with the mean rating for part-time be- ing 78.9 per cent and for full-time being 82.2 per cent. Part-time faculty members tended somewhat more than full-time to scatter to either extreme. They rep- resented 55 per cent of all ratings but had 60 per cent of the top twenty-five ratings and 68 per cent of the bottom twenty-five. In general, sex and part- time or full-time status appeared to have TABLE 4. PERCENTAGE OF RATINGS FOR WmcH EAcH ATTRIBUTE ScoRED HIGHEST ATTRIBUTE Effectiveness in putting subject across Usual preparation for class . · Use of examples Stimulation of thought . . . . . Tolerance toward student difference of opinion Spirit of helpfulness in and out of class Has motivated me to do my best work . Balance of lectures, class discussion, student reports Stimulates exchange of ideas Assignments reasonable in length . Knowledge of and familiarity with subject Organization of material 3 16 4 3 25 25 1 0 1 2 50 5 3 9 4 4 33 24 2 0 1 3 59 3 Per Cent 4 23 4 3 16 27 0 0 0 1 39 7 2 14 3 3 30 32 0 0 0 3 48 3 5 18 5 4 19 18 2 0 1 1 52 7 474 I College & Research Libraries • November, 1966 TABLE 5. PERCENTAGE OF RATINGS FOR WHICH EAcH ATTRIBUTE ScoRED LowEsT ATTRIBUTE Effectiveness in putting subject across Usual preparation for class Use of examples Stimulation of thought . . . . . Tolerance toward student difference of opinion Spirit of helpfulness in and out of class . Has motivated me to do my best work . Balance of lectures, class discussion, student reports . . . . . . . Stimulates exchange of ideas . . Assignments reasonable in length . . Knowledge of and familiarity with subject Organization of material little bearing on teaching effectiveness, but the interesting clusters at extremes are hard . to explain. 5. In what characteris.tics were in- structors rated highest? Lowest? The strongest characteristic was "knowledge of and familiarity with subject," which was a strong point on half of the ratings. - "Tolerance toward student difference of 1 ' opinion" and "spirit of helpfulness in and out of class" were each strong points on 25 per cent of the courses. Table 4 shows no other characteristics to be rated high- est on more than 10 per cent of the courses. The weakest characteristics were "or- ganization of material" ( 31 per cent) , "stimulation of thought" ( 29 per cent), and "effectiveness in putting subject across" ( 18 per cent) as seen in Table 5. It is interesting to compare Drexel strong and weak points with the results of a large-scale rating program carried out among undergraduates at Brooklyn College and reported by Riley, Ryan, and Lifshitz in The Student Looks at His Teacher. In the Brooklyn study, as at Drexel, the highest scoring characteristic 18 9 0 29 11 3 5 6 4 7 1 31 25 11 0 29 5 4 6 8 2 8 1 33 PERCENT 10 6 · o 28 18 2 3 3 6 5 1 28 21 12 0 35 4 2 5 7 3 3 0 34 14 4 0 21 20 4 4 4 5 10 2 26 concerned knowledge of subject matter. Similarly, at Brooklyn the poorest scor- ing characteristic was "encouragement of thinking," and at Drexel "stimulation of thought" was one of the weakest points. Despite these correspondences, however, there was a significant differ- ence between Drexel and Brooklyn in- structors on "organization of subject material" -the third best characteristic of ten at Brooklyn, but the weakest at Drexel. 8 There were several differences in the weakest and strongest characteristics by sex and by full-time status of faculty members. For instance, while almost one-fourth of the full-time instructor rat- ings were strong in "usual preparation for class," this was a strong point for only one-eleventh of the part-time in- structors who usually had full-time jobs elsewhere in addition to their Drexel teaching. On the other hand, ratings for part-time instructors were 50 per cent higher than for full-time on ''knowledge ~f and familiarity with subject." Stu- dents also considered part-time instruc- s Riley, Ryan, and Lifshitz, op. cit., p. 82. Library School Instructor Evaluation I 475 TABLE 6. RATINGS OF TYPES O F CoURsEs Type of Course Mean Score Reference 81.0 Library Materials 78.9 Cataloging . 79.8 Administration 79.1 Special Types of Service 82.3 Backgrounds . 77.0 Children's Work 88.6 Information Science 79.8 Basic Required . . 79.2 Intermediate Required 79.5 Electives . . . . 82.0 Information Science . 79.8 tors more tolerant of student difference of opinion, on a two-to-one ratio. Still another significant difference between part-time and full-time instructors lay in the area of "effectiveness in putting sub- ject across" which was a strong point for one-fourth of the part-time instructors but for only one-tenth of the full-time in- structors. An interesting difference between the ratings of male and female instructors was in "tolerance toward student dif- ference of opinion." This attribute oc- curred as a weak point on ratings of fe- male instructors significantly more often than on ratings of males. 6. Were electives rated higher than re- quired courses? Although it may seem natural for students to have been more favorably inclined toward the courses they elected to take than toward re- quired courses, the studies done at Brooklyn College9 and at Colorado State College of Education10 found no impor- tant difference here. At Drexel, as shown in Table 6, elective courses had a slight but not significantly higher mean than basic and intermediate required courses . 8 Ibid. , p. 87. 10 " Relation of Class Size and Other Factors • . . ," op. cit ., p. 102. Per Cent P er Cent of P er Cent of of All Ra tings Top 25 Ratings Low 25 Ratings 19 20 24 15 4 12 12 8 12 11 12 8 14 28 16 12 4 20 10 20 0 7 4 8 100 100 100 29 12 28 25 12 28 38 72 36 7 4 8 100 100 100 Required courses, however, accounted for 54 per cent of all the ratings but only 24 per cent of the highest twenty-five ratings. 7. Which courses were mos.t highly rated? When considering the scores of groups of courses the most obvious pat- tern was the high ratings given to courses dealing with library service to children and young people in school and public libraries. The seven courses in this category were offered twenty-five times for an average score of 88.5 per cent, eight points higher than the over- all average. Also, special types of li- brary service, such as medical, law, spe- cial, college, etc., were unusually well represented among the top twenty-five courses. It is possible that the somewhat more favorable ratings given school and children's librarianship courses were re- lated to the similarly favorable ratings given in summer quarters, since during summer quarters the percentage of stu- dents and of courses in school and chil- dren's librarianship is relatively high. For single courses, the highest score for a required course rated more than ten times was for "Basic Reference Materials" with a mean score of 86 per cent. The lowest mean score for such a 476 I College & Research Libraries • November, 1966 TABLE 7. 25 HIGHEST RATINGS ~ "1:1 = ::I ·s = ~ a; C' ... u 0 s ~ ... ... ... ~ 0 ~ .s ~ ... .!:l ... I 0 0 > V) bO .S t: ~ ".d "' ~ t: :§ 0 "' ~ .9 ~ ~ ~ ~ CD ::I c.: ~ ~ Course ~ u i>< 0 ~ ---- A F F Advanced Selective Cataloging E 6 . . . •• 0 •••• 100 c p M Rare Book Librarianship E 9 63 Summer 100 D p F Workshop . . . . . . E 17 62 Summer 100 E F F Adult Education and the Library E 14 61 Spring 100 G p F Advanced Reference • • 0 • • E 4 60 Summer 100 s p M College and University Library Service E 9 . . . •• •• 0 •• 100 T F M Methods of Research in Librarianship E 4 60 Summer 100 B p F Selection of Library Materials . R 19 . . . . . ..... 99 K p F Folk Literature and Its Oral Presenta- tion . . . . . E 11 63 Summer 99 Q p F Library Methods Analysis • • • 0 E 7 62 Summer 99 DD p F Selection of Library Methods for Chil- dren E 20 61 Summer 99 H p M Government Publications E 22 64 Fall 98 I F M Special Library Service E 13 61 Summer 98 I F M Special Library Service E 10 63 Winter 98 cc p M Integration of Science Information Systems . . . . . E 5 64 Summer 98 M p M Introduction to Library Services R 22 61 Summer 98 w p M College and University Library Service E 23 62 Spring 98 BB p F Selection of Library Materials for Children . . . . . E 14 63 Summer 98 A F F Cataloging and Classification R 10 . . . • 0 •• 0 •• 97 G p F Advanced Reference E 8 64 Fall 97 J F F Selection of Library Materials for Young People . . . E 21 . .. • 0 ••••• 97 I F M Introduction to Library Services R 22 61 Summer 97 I F M Reference in Science and Technology R 24 ... • •••• 0 0 97 I F M Reference in Science and Technology R 26 63 Winter 97 N p F Adult Education and the Library . E 15 62 Summer 97 course was the 7 4 per cent received by "Library in Society," a required course on the history and sociology of libraries. 8. Is there any relationship between class size and ratings? Although class size was not a significant factor in the ratings at Brooklyn11 and Colorado, 12 small classes at Drexel were rated more favorably than large ones, as Tables 7 and 8 show. The average class size at Drexel was twenty, but the average class size for the highest twenty-five ratings was only 14.2. It should be noted, how- ever, that the average class size for the twenty-five lowest courses-18.76-was also somewhat lower than the over-all average. The mean score for classes hav- ing ten or fewer students was 86 per cent, compared to the over-all average of 81 per cent. While small classes com- prised only 10 per cent of all ratings, they made up 40 per cent of the twenty- five highest ratings. Furthermore, of all small classes, more than half received scores of 90 per cent or above. u Riley, Ryan, and Lifshitz, op. cit., p. 87. 12 "Relation of Class Size and Other Factors ... ," op. cit., p. 102. Library science instructors are faced with many teaching problems. In addi- tion to the fact-mentioned earlier-that many of them have had little or no train- ing in teaching methods, they must teach classes made up of students with Library School Instructor Evaluation I 477 TABLE 8 . 25 LowEST RATED CoURsEs Cl) ~ "'Cl ~ :3 Cl) '3 = IJ:.. ;;; 0' II) .... II) u 0 8 p::; .... .... II) II) .... II) II) IJ:.. 0 p., .s ~ .... II) N .... I 0 0 > v.i II) bl) E t: II) ·.o "' .... 1a ~ ;;; 0 ~ 0: ·z ] 0: II) Cl) ::l 0: p., ~ Course ~ G >< Ql >I:; -- v p M Introduction to Cataloging and Classi- fication R 8 61 Fall 28 F p M Audio-Visual Materials E 18 62 Spring 34 p p M College and University Library Service E 23 61 Summer 36 u p M College and University Library Service E 23 . . . • 0 •• • •• 37 p p M Selection of Library Materials R 28 61 Summer 37 0 p M Library Methods Analysis E 13 65 Winter 39 JJ p M Selection of Library Materials R 15 62 Summer 42 FF p F Reference in Science and Technology R 12 64 Spring 46 FF p F Reference in Science and Technology R 24 64 Winter 48 GG p M Reference in Social Sciences R 25 64 Fall 48 J p M Science Lit. Searchin~ and Abstracting E 21 . . . . . . .. .. 49 HH F F History of Books an Printing . . R 9 65 Winter 51 R p F Introduction of Cataloging and Classi- fication . . . . R 33 62 Summer 52 HH F F Reference in Social Sciences R 18 64 Fall 53 II p M College and University Library Service E 31 65 Spring 54 HH F F Library Administration R 35 65 Winter 56 AA p F Search Strategy .. . . . . . E 12 62 Fall 56 X p M Methods of Research in Librarianship E 19 63 Winter 56 z F M Reference in Humanities R 26 64 Spring 57 EE p M Law Librarianship . E 17 62 Spring 58 T F M Library in Society . R 7 . . . . . ..... 58 T F M Library in Society . . . R 13 64 Winter 59 L F F Introduction to Cataloging and Classi- fication . . . . . . . . R 17 65 Spring 59 HH F F Methods of Research in Librarianship E 11 65 Winter 60 y p M Advanced Reference E 11 64 Summer 62 widely varying backgrounds, library ex- periences, and goals in librarianship. Furthermore, there is the constant prob- lem of achieving a happy balance be- tween theory and practice. Add to this the lack of adequate textbooks and it seems clear that the task facing the li- brary science instructor is not an easy one. At Drexel it is felt that the student rating program is an important method of helping the instructor do an effective job. While ways of improving the rating sheets are constantly being sought the program itself has proven successful. • •