College & Research Libraries News vol. 79, no. 1 (January 2018) C&RL News January 2018 16 Ashift in thinking in the library profession has moved us from the ACRL Informa- tion Literacy Competency Standards for Higher Education to the Framework for Information Literacy for Higher Educa- tion. This shift from basic retrieval skills to higher-level thinking has presented challenges among instruction librarians as they try to interpret and incorporate the Framework into their programs, while ensuring student’s success in classes where basic search skills are essential. As managers of instruction programs, we had the same questions at our libraries and hoped they could be answered through critical conversations about what we want students to know or be able to do when they leave our classrooms. Drawing on experiences in credit-bear- ing and one-shot instruction, we present a tested method for facilitating conversa- tions about developing student learning outcomes (SLOs) for entire instruction programs, specific courses, or one-shot sessions. The process we share in this article helped us guide librarians teaching both credit courses and one-shots in develop- ing new SLOs that incorporated aspects of the Framework. We hope that this process will help other instruction coordinators or depart- ment managers lead conversations about learning and the Framework. Our instruction programs University of Northern Colorado The University of Northern Colorado’s (UNC) University Libraries has long of- fered credit courses dating back to fall 2000. Since that time the program has grown to include eight distinct courses taught to undergraduate and graduate stu- dents. The earliest syllabus on record pres- ents student learning outcomes based on the Standards, and those outcomes were subsequently used for the development of all credit courses. With the drafting of the Framework, library faculty embraced the opportunity to overhaul the student learn- ing outcomes for all 100-level courses, of which there are four (LIB 150, LIB 151, LIB 160, and LIB 170). This was done through the University Libraries Curriculum Com- mittee, whose membership is typically made up of the six library faculty teaching credit courses throughout the academic year. Andrea Falcone is associate director of education and public services/head of education and outreach services at University of Colorado-Denver’s Auraria Library, email: andrea.falcone@ucdenver.edu, and Lyda McCartin is professor of university libraries/head of information literacy and undergraduate support at University of Northern Colorado, email: lyda.mccartin@ unco.edu © 2018 Andrea Falcone and Lyda McCartin Andrea Falcone and Lyda McCartin Be critical, but be flexible Using the Framework to facilitate student learning outcome development mailto:andrea.falcone%40ucdenver.edu?subject= mailto:lyda.mccartin%40unco.edu?subject= mailto:lyda.mccartin%40unco.edu?subject= January 2018 17 C&RL News Auraria Library The Auraria Library has a one-shot instruction program that serves three institutions—the University of Colorado-Denver, Metropolitan State University, and the Community College of Denver. As a result of the library’s mission to serve three unique institutions, library instruc- tion has been flexible to the needs of the varied curriculums. A number of years ago the library adapted the Standards to create four SLOs, which served primarily as a means of commu- nicating the program’s purpose through the library’s web- site. Following approval of the F r a m e w o r k , librarians in the Education and Outreach Services De- partment, who are responsible for delivering instruction to the three insti- tutions, came together to re- write the SLOs guiding their work. The process Student learning outcomes “focus on knowl- edge, skills, and values” and “describe the student behaviors that demonstrate their learning.”1 Student learning outcomes an- swer the questions What should students know? What should students be able to do? or What should students value? Facilitat- ing conversations about student learning is always challenging, because everyone in- volved has a unique teaching philosophy and values content differently. The three- step process outlined below helps combat that challenge by acknowledging these dif- ferences and ultimately bringing a team to consensus. Step 1: Brainstorm. Whether you are revis- ing existing SLOs or developing new ones, it’s important to start from scratch. If you have existing SLOs, we recommend putting them aside during brainstorming. Don’t be concerned. Worthwhile content in those current outcomes will come through in this process. This provides an open space for everyone’s ideas to be shared and given equal consider- ation. Brainstorming also familiarizes the team with what their colleagues’ value in information literacy instruction. The brainstorm starts with a “sticky note” activity, where each person writes down, on individual sticky notes, the skills and concepts they think students should have or know by the time they leave the instruction program (this c o u l d b e a credit course, a s e r i e s o f one-shots, or a single session). T h e n o t e s could include skills or concepts like peer review, selecting keywords, or writing a research question. Participants may also include assignments or activities, such as writing a literature review. During this activity remember to include the Framework concepts on the sticky notes. As the facilitator, you may need to consciously include the Framework language because the team may not do so naturally. Once everyone has created sticky notes, they should be displayed on a whiteboard or other large space to enable everyone to view all of the notes. Once the sticky notes are displayed, the team will work together to group them into overarching concepts. Then apply an overarch- ing concept to each batch of sticky notes. The image above shows the results of this step at Auraria Library. Some examples of overarching concepts include “evaluation” and “research process/search strategy.” Example of the brainstorm step at Auraria Library. C&RL News January 2018 18 Allow for the conversation about over- arching concepts to take quite a bit of time, as the meaning of individual notes will be questioned. This part of the process allows the team to norm language. For example, a sticky note labeled “citations” might be referring to teaching how to write citations, read citations, chase citations or any combi- nation thereof. Clarification and consensus is necessary to determine the related over- arching concept. Below is an example of the brainstorm from UNC. The heading is the overarching concept, and the bullet points represent individual sticky notes. The sticky notes under “evaluation” were easily grouped into a category. However, at the end of the conversation about overarching concepts, there were three sticky notes that were un- grouped, which included assignments and skills. Once someone moved the “scholar- ship as conversation” sticky note to the batch, it all fell into place. One thing to watch for during the brain- storm are outliers that focus on services like “virtual reference,” “interlibrary loan,” and “course reserves.” When services emerge during the brainstorm, remind people that this brainstorm is about creating measurable learning outcomes for your instruction pro- gram. Students can learn about the existence of services, like virtual reference, through various marketing and awareness outlets, like social media or the library’s website. Instructors may want to call attention to these services during a session. However, as the Framework talks about higher-order thinking skills and research behavior devel- opment, awareness of services would not be an assessed outcome of an instruction session. Step 2: Draft student learning outcomes. The next step in this process is to draft SLOs. The overarching concepts are the basis for the first draft of your team’s SLOs. Bloom’s Taxonomy is a wonderful tool to aid in the discussion. We suggest having copies of this available for everyone during this step. We also recommend drafting student learning outcomes one at a time so that the team remains focused. Here are some guiding principles for drafting student learning outcomes: • Write outcome statements that reflect the knowledge and skills students should acquire in a one-shot or credit course. • Write outcome statements that are observable and measurable. Use active verbs that focus on observable behaviors rather than what students think, understand, appreciate, etc. • Write outcome statements that are short and concise. • Write outcome statements in the form of “At the end of this one-shot students will be able to _____;” or “Students completing __________ will be prepared to _____.” Step 3: Be critical. Being critical of each drafted SLO is essential to setting the founda- tion for future student learning assessment and helps ensure you collect appropriate and useful data to assess your program, course, or session. We recommend using the Critical Checklist below to refine your SLOs. It is best to work through the critical checklist in pairs or small groups and then come back together for a larger conversation. 1. Is this SLO measurable? Does the SLO focus on observable behaviors rather than what students think, understand, appreciate, etc. If not, provide a revision for consideration. Evaluation Scholarship as Conversation • Publication cycle • Mini research study with literature review • Evaluation process • Skill of using bibliographies to find additional sources • Peer review • Literature review practice • Types of sources (popular, scholarly, trade, etc.) January 2018 19 C&RL News 2. Does this SLO focus uniquely on library research and our disciplinary exper- tise? Consider if other programs, depart- ments, or disciplines may be responsible for this content. 3. Does this SLO push beyond teaching basic procedural content? The Framework asks us to think about core ideas regard- ing information literacy and student’s roles within the information ecosystem, so an outcome related to accessing databases does not meet the aims of the Framework. 4. Is this SLO appropriate for a 100-level course? This checklist item will change based on the program. For example, this might be replaced with something like “Is this SLO apppropriate for the FYE work- shop?” 5. What is the importance of this SLO (high importance or low importance)? Unique SLOs should be used for different levels of instruction or types of programs. Be sure that the level of SLO is appropriate for what you are assessing. 6. Could this SLO be reworded for clar- ity and focus? If so, provide a revision for consideration. UNC developed SLOs for credit courses, first drafting each SLO, and then applying the critical checklist and having conver- sations to finalize wording. Below is the original and revised SLO for “scholarship as conversation.” • Original SLO: At the end of the course students will recognize that scholarship in a particular discipline is a conversation be- tween peers within the discipline. • Revised SLO: At the end of the course students will be able to participate in schol- arly discourse within a discipline or field of study. Ideally you will have developed three-to- five SLOs. If some SLOs were determined to be of low importance, focus on procedural content, or if the SLOs are more appropriate for another discpline, consider revising or removing them. Continue using the critical checklist with each iteration of SLO drafts until the team comes to consensus on the final wording. Conclusion This process is designed to be flexible and work within the context of any instruction program keeping in mind that developing and revising SLOs is a challenging yet necessary endeavor. Be- low are some suggestions for successfully imple- menting this process at your library. • Set aside time for reflection and editing. Rather than immersing your team in this pro- cess over the course of one or two meetings, space the conversations so there is adequate time to reflect on the drafts. Having an online editable document so people can edit the outcomes when inspired and able is helpful. • As the facilitator, continue to motivate and encourage individuals, especially when they contribute to the working document. If you notice an individual has been inactive, reach out and encourage their ideas. • Remember that this is a group effort. Facilitators must be careful not to dominate conversations and should not take sole re- sponsibility for writing the SLOs. • Remember that you do not have to incorporate every aspect of the Framework into your SLOs. Some parts of the Framework may be out of the scope for your instruction program and may be more appropriately in- tegrated in other teaching activities, settings, sessions, or library courses. As library instruction program coordina- tors and mangers, we have a responsibility to continuously evolve the learning experiences in our classrooms and to ensure that we teach students to be successful in our changing infor- mation landscapes. While SLOs are a powerful tool for demonstrating alignment with national expectations of information literacy, they, above all else, should guide our intentions for spend- ing valuable instructional time with students. Note 1. Mary J. Allen, Assessing Academic Pro- grams in Higher Education (San Francisco: Jossey-Bass, 2004), 28.