An Evaluation of Library Instruction Delivered to Engineering Students Using Streaming Video Previous   Contents   Next Issues in Science and Technology Librarianship Summer 2006 DOI:10.5062/F4TB14V7 URLs in this document have been updated. Links enclosed in {curly brackets} have been changed. If a replacement link was located, the new URL was added and the link is active; if a new site could not be identified, the broken link was removed. An Evaluation of Library Instruction Delivered to Engineering Students Using Streaming Video Jack M. Maness Engineering Reference and Instruction Librarian Gemmill Engineering Library University Libraries University of Colorado at Boulder Boulder, Colorado jack.maness@colorado.edu Abstract This paper evaluates the use of streaming video applications in the delivery of information literacy instruction to engineering students. It describes the University of Colorado at Boulder (UCB) Engineering Library's implementation of streaming video to support graduate distance programs in the Center for Advanced Engineering and Technology Education (CAETE), and reviews library literature regarding the use of streaming technology. As an initial evaluation of the project, the preliminary results of a survey (n=27) comparing satisfaction levels and learning outcomes between students who attended library instruction sessions in-class versus via streaming video are given. The results reveal no significant difference in satisfaction or learning outcomes between students who received library instruction delivered via streaming video or students attended live. Introduction CAETE is the distance education entity of the University of Colorado's College of Engineering and Applied Science. It offers masters degrees in aerospace engineering, electrical and computer engineering, engineering management, and interdisciplinary telecommunications. Using a combination of course management systems, such as Blackboard or WebCT, and streaming video or compact-disc captures of lectures, CAETE is able to offer the same classes to both on-campus and distance students. The lectures are recorded using Tegrity software, a program that captures the lecturer's desktop, physical image via a digital camera, and audio of the classroom (see www.tegrity.com). It coordinates all three data streams to provide a representation of the lecture and discussion to the distance learner via streaming technology on the Internet or via a CD mailed to students' residences. Three classrooms are outfitted with all the necessary video, audio, and software equipment, and CAETE has begun experimenting with equipment loaning to faculty for recording lectures. Using Tegrity, distance students (and on-campus students who wish to re-attend a lecture) can download presentation files and navigate through a lecture's sections by a drop-down menu. In an effort to provide library instructional services to CAETE distance students, and to all engineering students more generally, the Engineering Library began a relationship with CAETE in 2005 that enabled the Instruction Librarian to create short tutorials (teaching the basics of the catalog, specific databases, and search techniques) using Tegrity, and posting them on the Library's web site. Believing that this would enable students to ostensibly attend information literacy lectures at their leisure, and thereby contribute toward fulfilling the Library's objective in ensuring graduates are information literate, experimental tutorials were recorded in 2005 and promoted among CAETE and the College of Engineering at large. This article provides the results of an initial evaluation of this project. Literature Review The use of streaming video technology is not new in librarianship, and while a fairly extensive literature regarding its feasibility exists (cost, technical requirements, etc.), little has been written on its viability (how users perceive and use it) and efficacy (how well it imparts instructional content). The use of traditional VHS has been much more researched in librarianship, but this trend has not continued with the advent of new video technologies, as it has in other fields. It has been established in other disciplines that distance education is equally as effective as live education, and streaming video can even further bolster distance education's relative value: Although literature abounds with testimony that the performance of distance and local students is equivalent, there remains an intuitive feeling that in-class education is superior to distance education. By integrative streaming media in a Web-based framework to deliver critical components of a consistent teaching technology, we get a pedagogical solution to removing the difference between learning in class and learning at a distance (Sircar 2000, p. 55). But such literature does not abound in librarianship, and it is not wise to over-generalize research in other disciplines to the often unique issues associated with library instruction. Instruction for engineering students can be particularly problematic in this regard. Quigley & McKenzie (2003) found that the lecture format, which streaming video must use in most contexts, did not meet their instructional objectives in an engineering course. Arnold, et. al. (2003) found that "interactive" learning was more effective than "lecture based" sessions for engineering freshman, and based on the personality typology of engineering students, Williamson (2003) found that "active learning" was best suited for such students. But the asynchronous nature of streaming video tutorials can render the use of these practices problematic at best, impossible at worst. It is then quite necessary that science and technology librarians assess the use of streaming video in their instructional initiatives, even as they begin to employ the use of the technology in those initiatives. Within the library literature, much assessment of streaming technologies includes product comparisons (Cox 2004; Lawrence & Weigand 2005; Lee & Burrell 2004; Tempelman-Kluit & Ehrenberg 2003) and best practice recommendations based on experience and descriptions of specific initiatives (Chakraborty & Victor 2004; Cox & Pratt 2002; Crowther & Wallace 2001; Hickok 2002). Long & Culshaw (2005) provide all of the above in an exploration of the use of similar software for training programs within libraries. Many cite the pedagogical rationale of using video, such as the fact that it provides a multi-sensory learning experience, as opposed to what a text-based or solely audio tutorial would (Buehler, et. al. 2001; Xiao, et. al. 2004). There also exists a wealth of very well-researched criteria for effective online tutorials more generally, such as interactivity (Dewald, et. al. 2000) and "branching capabilities" or non-linear navigation options which many video programs do provide (Tancheva 2003, p. 9). Some researchers, including Hickok (2002), provide a comprehensive literature review outside librarianship, furthering the rationale for using streaming video in library instruction initiatives. Jackson (1999) provides the only data on users' evaluation of library instruction delivered via streaming video, showing that all six respondents in one survey would choose the method over text-based materials, thereby corroborating Cofield's (2002) broader data generated on a generic, non-library tutorial. Streaming video should provide an optimal alternative to live information literacy sessions, but there is no documentation in the literature that it does. What follows is a very preliminary investigation of engineering students' satisfaction with streaming video delivery of library instruction. It is intended as a beginning, and replication and expansion of this data is certainly necessary and recognized. Methods This study began as a simple feedback mechanism for the author in his delivery of library instruction lectures to three courses in CAETE's Engineering Management (EMP) and Interdisciplinary Telecommunications (ITP) Programs. Given early in the fall semester, 2005, the author created a simple online survey designed to gather input from students attending the lectures in an effort to improve upon the lectures. The questions were initially designed to measure the effectiveness of the instructor rather than the effectiveness of this type of video presentation. The responses proved to be meaningful statements regarding the video format, however, so approval to publish the data was sought and obtained through UCB's Human Research Committee. By surveying satisfaction levels of the classes, which included both in-class students and distance students attending via the video, the Library was able to begin assessing the viability of using such technology for library instruction tutorials that might be provided where live sessions could not. Four lectures were delivered to three different groups of students over a two month period, two EMP classes and one ITP class, the latter of which received two lectures one month apart. All students were directed to the online survey following the lectures by the courses' instructors via e-mail. In the case of the ITP class, the survey was only given to the students following the completion of the second lecture, the assumption being that the students would evaluate the two lectures as a whole. The lectures all consisted of introductory material, providing contact information for the Libraries and basic procedures, such as interlibrary loan and remote access to databases; one brief note on the conceptual underpinnings of research in the students' disciplines, and another on the reliability of sources; demonstrations of specific databases; and an introduction to bibliographic management software. One EMP class was given handouts, both in-class and distance students, prior to the lecture date. Students in the other EMP class and those in the ITP course were not given handouts. There was some discussion during the sessions, but no exercises that would constitute "active" learning. Eighty-four students were given the survey. The survey asked two identifying questions: what class they were in and how they had attended the session. It then asked students to respond to four positive statements (designed to measure satisfaction levels) about the session by indicating if they strongly agreed with the statement, agreed, felt neutral about, disagreed, or strongly disagreed. One outcome-based question asked students to indicate what they had learned from the session by choosing examples and/or indicating themselves what they had learned, and a final open-ended question asked for any additional feedback the student may wish to provide. The four positive statements (Q) meant to measure satisfaction were: Q1) The instructor's delivery of the material was valuable; I was engaged in the class. Q2) The material provided was adequate. I had everything I needed before, during, and after the lecture to assist my learning the material. Q3) The material itself was valuable. I will use what I learned in this lecture in my future studies and/or career. Q4) I feel more confident with doing research after receiving this lecture. The list of potential learning outcomes was: new database to use; new way to search; new way to think about research; new service from the library; new person to contact for help; and an open ended answer option. Results Twenty-eight of 84 students responded to the survey. This 33% response rate is common of e-mail surveys, which tend to average between 30% and 40% (Sheehan 2001). Fourteen respondents had attended the lectures via streaming video, 13 had attended in class, and 1 student indicated they had attended in both formats. This student may have been an on-campus student who viewed the video afterwards, but due to the uncertainty inherent in this answer, this student's answers were excluded from the current analysis (n=27). Both live and video students strongly agreed (SA) or agreed (A) with the four positive statements the majority of the time. Neutrality (N) was expressed very infrequently, and one student disagreed (D) with one statement. This overall similarity in responses between the two compared groups is also seen in the aggregate frequency of responses. Video students strongly agreed or agreed with the statements 98% of the time, and live students 97% of the time. Essentially, there was little or no difference in satisfaction between live and video students. In terms of satisfaction, then, this data would suggest that video is an adequate alternative to live instruction. Similarly to satisfaction, video students indicated they had learned as much as had live students. Students in both groups chose from the list of learning outcomes with very similar frequencies. The largest discrepancy between the video and live students occurred in the "new service" response, which video students chose more frequently. A possible explanation for this could be a discussion of UCB's virtual private network, necessary for remote access to databases, or a description of a document delivery service. An additional interesting result was that both live and distance students chose more practical outcomes ("new person to contact," "new database to use") with more frequency than conceptual outcomes ("new way to search"). It would appear these results corroborate research regarding engineering students' learning styles. Finally, two video students indicated an outcome not listed: they had discovered the use of a web-based bibliographic management tool. An additional comment was made by a student that reinforced the concept of providing video tutorials online. A live student wrote, "I would appreciate it if the topics covered in the lecture are available as a tutorial (maybe as a PDF file or small video tutorial) that the students can download from the university web site." A faculty member commented similarly in class. It is promising to surmise from this preliminary analysis of a small sample size that video tutorials posted online could adequately provide information literacy sessions where live instruction is unlikely or impossible, but much more investigation is needed. This survey analysis used full-length lectures delivered within students' courses, not short tutorials posted on library web pages, which most libraries, including UCB, are employing. Studies specific to these tutorials are necessary, as are pre- and post-test studies, which often determine students learn less from online tutorials than surveys indicate. Conclusion The use of streaming video is becoming more common in library services online, and though research in other fields leads one to believe this is a valuable service, one that would not only enhance conventional text-based web page tutorials but may provide an adequate alternative to attending such sessions live, there has been little research done in librarianship in this regard. And, considering the general findings of librarians relative to science and engineering students' preferences and learning styles, there is even less reason to assume that video tutorials are worthwhile ventures. The data given here is preliminary, but it does seem to suggest that video tutorials are indeed adequate alternatives to live lectures for engineering students. The video students in this survey reported similar rates of satisfaction, similar learning outcomes, and made comments that validated the idea of providing video tutorials. The promise of producing video tutorials for all students, at their point of need, and preserving the tutorials for future students, is tantalizing, and the UCB's Engineering Library will continue to produce them, believing they are valued and effective. References Arnold, J., et. al. 2003. Hands-on learning for freshmen engineering students. Issues in Science and Technology Librarianship 37 (Spring 2003). [Online]. Available: http://www.istl.org/03-spring/article3.html. [February 10, 2006]. Buehler, M., et. al. 2001. It takes a library to support distance learners. Internet Reference Services Quarterly 5 3): 5-24. Chakraborty, M. & Victor, S. 2004. Do's and don'ts of simultaneous instruction to on-campus and distance students via videoconferencing. Journal of Library Administration 41(1/2): 97-112. Cofield, J. 2002. An assessment of streaming video in web-based instruction. Paper presented at the Annual Meeting of the Mid-South Educational Research Association (Chattanooga, TN, November 6-8, 2002). Cox, C. 2004. From cameras to Camtasia: streaming media without the stress. Internet Reference Services Quarterly 9(3/4): 193-200. Cox, C. & Pratt, S. 2002. The case of the missing students, and how we reached them with streaming media. Computers in Libraries 22(3): 40-45. Crowther, K. & Wallace, A. 2001. Delivering video-streamed library orientation on the web: technology for the educational setting. College & Research Libraries News 62(3): 280-285. Dewald, N. et. al. 2000. Information literacy at a distance: instructional design issues. The Journal of Academic Librarianship 26(1): 33-44. Hickok, J. 2002. Web library tours: using streaming video and interactive quizzes. Reference Services Review 30(2): 99-111. Jackson, C. 1999. Computer-based video: a tool for information skills training? Aslib Proceedings 51(7): 213-223. Lawrence, D. & Weigand, S. 2005. Need and know-how: the perfect partnership for creating tools for distance students. Presented at the 2005 Colorado Association of Libraries Annual Conference. (Denver, CO, November 10-13, 2005). [Online]. Available: {http://cal-webs.org/handouts05/Knowhow.pdf} [December 16, 2005]. Lee, S. & Burrell, C. 2004. Introduction to streaming video for novices. Library Hi Tech News 21(2): 20. Long, H. & Culshaw, J. 2005. How we used demonstration authoring software to create tutorials. Computers in Libraries 25(8): 6-8. Quigley, B. D. & McKenzie, J. 2003. Connecting engineering students with the library: a case study in active learning. Issues in Science and Technology Librarianship 37 (Spring 2003). [Online]. Available: http://www.istl.org/03-spring/article2.html [December 16, 2005]. Sheehan, K. 2001. E-mail survey response rates: a review. Journal of Computer-Mediated Communication 6(2). [Online]. Available: {http://onlinelibrary.wiley.com/doi/10.1111/j.1083-6101.2001.tb00117.x/full} [December 16, 2005]. Sircar, J. 2000. Streaming media technology: laying the foundations for educational change. Syllabus 14(3): 56-57. Tancheva, K. 2003. Online tutorials for library instruction: an ongoing project under constant revision. Paper delivered at the Eleventh ACRL National Conference. [Online]. Available: {http://www.ala.org/acrl/sites/ala.org.acrl/files/content/conferences/pdf/tancheva.PDF} [December 16, 2005]. Tempelman-Kluit, N. & Ehrenberg, E. 2003. Library instruction and online tutorials: developing best practices for streaming desktop video capture. Feliciter 49(2): 89-90. Watson, J. 2004. Going beyond screen captures: integrating video screen recording into your library instruction program. Feliciter 50(2): 66-67. Williamson, J. 2003. Suiting library instruction to the Myers-Briggs personality types and Holland Vocational personality types of engineering students. Issues in Science and Technology Librarianship 37 (Spring 2003). [Online]. Available: http://www.istl.org/03-spring/refereed2.html [December 16, 2005]. Xiao, D.Y. et. al. 2004. Full stream ahead: database instruction through online videos. Library Hi Tech 22(4): 366-374. Previous   Contents   Next