Academic Librarians in Data Information Literacy Instruction: A Case Study in Meteorology 536 Academic Librarians in Data Information Literacy Instruction: A Case Study in Meteorology Emily P. Frank and Nils Pharo Emily P. Frank is Research and Instruction Services Librarian at Louisiana State University; e-mail: efrank7@lsu.edu. Nils Pharo is a Professor at Oslo and Akershus University College of Applied Sciences; e-mail: nils.pharo@hioa.no. © 2016 Emily P. Frank and Nils Pharo, Attribution-NonCommercial (http:// creativecommons.org/licenses/by-nc/3.0/) CC BY-NC. E-science has reshaped meteorology due to the rate data is generated, collected, analyzed, and stored and brought data skills to a new promi- nence. Data information literacy—the skills needed to understand, use, manage, share, work with, and produce data—reflects the confluence of data skills with information literacy competencies. This research assessed perceptions of data information literacy and attitudes on its instruction for graduate students in meteorology. As academic librarians have traditionally provided information literacy instruction, the research determined if they were perceived as having a role in data information literacy instruction. The modified Delphi method was applied to obtain the perspectives of a panel of experts, representing students, librarians, professors, and researchers, for the purpose of forecasting and consensus-making. Through the consid- eration of the University of Oslo’s Department of Geosciences’ Meteorol- ogy Section, the research found that data information literacy skills were relevant to the work of meteorology students. Stakeholders perceived that academic librarians could play a future role in general instruction but that they would have to overcome obstacles to be involved in data information literacy instruction. For librarians to enter this domain, they would need to improve their technical skills, enhance their discipline-specific knowledge, or rely on collaborations. The significance of these findings was limited by the modest target population under examination; as a consequence, the results were strongly linked to the specific setting. Further studies would be necessary to determine their generalizability. cademic librarians have a history of facing changes in technology that vastly reshape their work. The digital age has brought incredible changes in the way information and data are produced, consumed, adapted, and shared, requiring a transformation of resources and services. At the same time, the meteorological community has undergone significant shifts as a consequence of e-science. E-science represents the unprecedented collection and analysis of data thanks to advances in high performance computational technology and networked environ- ments. The scale of these data has accelerated scientific innovation and discovery but doi:10.5860/crl.77.4.536 crl16-828 Academic Librarians in Data Information Literacy Instruction 537 has resulted in a deluge of data, some of which are messy and unstructured, and all of which are beyond the scope of manual control. Modern meteorology requires deal- ing with vast quantities of atmospheric measurements and running computation- and data-intensive models. For students in the field to engage with e-science data, they must have data skills to a degree not previously required. These skills—data informa- tion literacy skills—are the competencies needed to participate in the data-intensive research of e-science. We have explored perceptions of data information literacy skills and attitudes on their instruction for graduate students in meteorology to discover if academic librarians are perceived as having a role. Data information literacy is defined as the skills needed to understand, use, manage, share, work with, and produce data. This is addressed through the consideration of a specific case: the University of Oslo’s Department of Geosciences’ Meteorology Section. The motivation for this research is to assess if the instruction of data information literacy skills reflects an unmet need for graduate students in meteorology and, if so, to consider the role of academic librarians in sup- porting this need. Five research questions are investigated in our study: 1. Are data information literacy skills relevant to the work of graduate students in meteorology? 2. How do graduate students learn data information literacy skills? 3. Do graduate students achieve proficient data information literacy skills through this practice? 4. What is the forecasted role of the academic library in data information literacy instruction of graduate students in meteorology? 5. What is the consensus on perceptions of librarian involvement in data informa- tion literacy instruction for graduate students in meteorology? Literature Review Multiple terms have been used to describe literacy associated with data, including data literacy, research data literacy, and science data literacy.1 While all underscore the relationship between data and information literacy, data information literacy uniquely adapts ACRL information literacy competency standards for higher educa- tion to encompass data.2 It therefore offers a familiar language for librarians and was the foundation of the definition used here. If librarians seek to offer precise and specialized support services to meteorology, they must understand the users. In looking at information seeking among scientists, Anderson, Glassman, McAfree, and Pinelli stressed the importance of accessibility and personal contacts. Drawing on Zipf’s principle of least effort, they concluded that scientists “prefer to gain information from colleagues, particularly those within their or- ganization.”3 Not only was the library consulted after personal contacts, they found “the least preferred option was to consult with a librarian or technical information specialist.”4 Hallmark studied the information-seeking behavior of academic meteorologists and found a “strong sense of lack of time as he or she balances teaching, research, graduate student support and collaboration” with other responsibilities; heavy dependence on Internet resources for data and almost exclusive use of electronic formats; and growing infrequency of visits to the physical library despite having a positive and friendly attitude toward it and librarians.5 Research considering the habits of geoscientists produced similar results, revealing that they expected immediate access, preferred resources obtained electronically, and had decreased frequency of library visits.6 In an analysis of the information-seeking behaviors of meteorologists and other atmospheric scientists who looked at access and use of data, participants did not 538 College & Research Libraries July 2016 express problems with their ability to find datasets.7 They did underscore the restric- tions they encountered when seeking data, including costly fees imposed by some national meteorological agencies; inaccessibility of data if the owner delayed releasing it until after publishing findings; and issues of data formats, media incompatibility, and insufficient metadata.8 The geosciences and earth sciences are described as “strongly data driven, and large varied data sets are often developed by researchers and government agencies.”9 The community faces challenges in creating usable databases out of these data sets because “a number of databases exist, but almost none of them are truly complete, error free as is practical, easily accessible, and simple to use.”10 They also apply data in many ways. Considering meteorologists and other research- ers in environmental science, Murphy highlighted the reliance on visualization and mapping technologies, such as GIS [Geographic Information System—a software used for capturing, analyzing, managing, and visualizing geographical data].11 Carlson, Johnston, Westra, and Nichols found that faculty, research assistants, and graduate students revealed a lack of formal training in data management and a pro- pensity to learn practices through self-directed trial and error.12 Research completed in collaboration between Cornell University, the University of Minnesota, the University of Oregon, and Purdue University found that, for graduate students in STEM (science, technology, engineering, and mathematics) disciplines, education on maintenance and data upkeep came from outside the classroom: “either through experience in the laboratory or through word-of-mouth.”13 Additionally, “graduate students reported a noticeable lack of training in data management and curation skills as they relate to their research practices.”14 Likewise, research by JISC, the U.K. organization focusing on digital technology for education and research, found that “much more needs to be done to embed data management training into all postgraduate programmes.”15 Interviews completed by Jahnke, Asher, and Keralis with postdoctoral fellows, graduate students, and other researchers revealed little to no formal training on data management practices, a lack of satisfaction with data expertise, and little consideration given to the long-term preservation of data.16 To compensate for this lack for formal experience, the participants applied skills from methodology courses and learned by trial and error.17 Additionally, participants had little awareness of data services the library could offer.18 STEM researchers at Purdue University echoed a lack of skills, feeling unsure how to share data, lacking the time to organize it and needing help in description, manage- ment, and archiving.19 Faculty have described inadequate data proficiencies and need for improvement. In a survey of 82 faculty members within the College of Science and Mathematics at California Polytechnic State University, San Luis Obispo: Fully half (50%) of respondents report they are either not sure or not confident in their data management skills. Of these responders, only 7 percent report they always or frequently educate themselves on best practices for preserving data. Among the sample as a whole, only 20 percent report they always or frequently educate themselves on data curation best practices. For those individuals who occasionally, rarely, or never educate themselves, we see a strong desire by more than 70 percent of responders for more guidance and education on best practices.20 Considering how to improve data abilities, Strasser and Hampton asserted that students should learn data management and organization skills early in their sci- Academic Librarians in Data Information Literacy Instruction 539 ence education.21 Yet, despite the need for early student intervention, their survey of general ecology course instructors at 48 major U.S. universities came to this conclu- sion: “Instructors recognize the value of good data management, data sharing, and reproducibility in their own research, but are not transferring that knowledge to their undergraduate students.”22 Considering this gap, academic libraries are increasingly taking an active role in data instruction. Libraries have offered specialized courses covering narrower components, such as the Science and Engineering Library at Vanderbilt University’s course Visual Display of Quantitative Information.23 Others, such as the libraries at University of Massachusetts Medical School and Worcester Polytechnic Institute, have provided undergraduate and graduate courses on research data management.24 Efforts are not limited to STEM disciplines. The University of California Los Angeles offered a course for upper-level undergraduates in the social sciences to improve student skills in understanding and visualizing data.25 Yet this remains an emerging role, with many libraries just starting to formalize work in this domain.26 Initiatives have targeted librarian skills, although training opportunities and needs are in a state of flux. Humphrey described a project with 66 Canadian universities thus: “A decade ago data services were a very specialized area, but the growth of statistical and data resources has given it greater prominence…. Increasingly, the patrons of academic libraries expect to find data resources in their library.”27 Gore detailed a program within the University of Massachusetts system to improve li- brarians’ ability to provide data support by increasing their knowledge of e-science and enhancing discipline-specific skills.28 Yet, in a separate study, science librarians considering roles in data management planning communicated a lack of consensus on whether significant additional training would be necessary to take on these responsibilities.29 Students, researchers, and some faculty in meteorology grapple with data informa- tion literacy skills. Taking an active role in instruction is seen as an opportunity for librarians to remain relevant and provide expertise to an emerging domain, and there is a growing willingness among academic librarians to provide such support.30 Yet, despite increasing emphasis among librarians to improve their skills and stake a claim through service offerings, there is a lack of research aligning disciplinary perspectives to determine if librarians are understood as having a role in this evolving landscape. Method We used a modified Delphi method with two rounds.31 The Delphi method obtains per- spectives from a panel of experts to highlight points of convergence and divergence—of a shared and not shared reality. Participants construct a shared reality through an iterative process where they review and revise their subjective perspectives for the purposes of forecasting and consensus-making. Forecasting has been described as the “process of making probability assessments about future developments, risks, and opportunities on the basis of contemporary knowledge, but taking into account relative uncertainty about whether future con- ditions allow for the application of this knowledge.”32 The Delphi method acts as a forecasting tool for innovative situations where knowledge is uncertain and little historical information is available.33 A central assumption of the method is that the opinions and insights of a group are more valid and reliable than those of an individual. To create the group interaction, perspectives at the individual level must be sought to allow for the “creative explora- tion of ideas leading to the reliable forecast for the future of a given subject.”34 These perspectives are then synthesized and aggregated together through iterative rounds 540 College & Research Libraries July 2016 of review and revision. Through this structured process, the individual opinions of experts move toward a unified and credible forecast of the future. Consensus-making is another principal motivation for using the Delphi method. After each round of data collection, responses are compiled and presented to the group, allowing participants to reconsider and modify their perspectives in light of the perspectives of others. Gradually progressing, this process continues until a pre- determined level of consensus is reached. Accepted measurements of consensus vary. For this research, consensus was a simple majority with at least half of the participants holding shared perspectives. Informants and Research Design The sampling process began with the consideration of the target population, available resources, and research questions. We identified meteorology students at the University of Oslo and associated stakeholders who would have an awareness of student data information literacy skills and skill instruction. The panel of experts was composed of two meteorology professors at the University of Oslo’s Department of Geosciences’ Meteorology Section, two researchers from the Meteorological Institute (MET)—the national meteorological research institute and the primary data provider, three PhD students from the University of Oslo’s Department of Geosciences’ Meteorology Section, and three academic librarians from the University of Oslo’s Science Library. Face-to-face, semistructured interviews using open-ended questions were selected for the first round of data collection. A pilot test was conducted prior to the first round with an information scientist who had worked at MET. As this exploratory research focused on an emerging topic, face-to-face interviews were used to help ensure partici- pant understanding. All participants received the same questions with slight variations to accommodate responses. Participants were encouraged to ask for clarification and prompted to consider questions in light of their experiences. Responses varied in depth and the average interview lasted 30 minutes. All interviews were audiorecorded and manually transcribed. Following transcrip- tion, interviews were read and marked to note overall themes. These divisions were further reduced through a winnowing process that involved reading the transcripts closely and exercising judgment through analysis, interpretation, and sense-making.35 Content was grouped into categories to reveal areas of convergence and divergence. For round 2, we took the controlled feedback from round 1—summarized and cat- egorized responses and tables showing distribution for each question—and crafted the second round questionnaire. The questionnaire followed the structure and wording of the first-round interview and was e-mailed to participants. This allowed participants to see a range of responses within the context of a specific question. The second-round questionnaire asked participants to review the distilled first- round responses. Identities were anonymized and single letters substituted for names. Participants were informed of their letter to compare their first-round responses to those of others. The participants were prompted to reflect on these and to revise or add content in text boxes that followed each question. Additionally, the act of review- ing one’s own responses provided a level of quality assurance, allowing participants to correct an incorrectly interpreted statement. Participants returned the reviewed and revised questionnaires via e-mail. A 100 percent response rate was achieved. Five responded with modifications and five had no changes. The modifications submitted by the five participants were read and grouped. Once all content was categorized, we decided that performing a more in-depth analysis of the round 1 and 2 content would highlight the variety found within the existing categories. Transcripts with the round 1 content were adjusted to include the revised and new content Academic Librarians in Data Information Literacy Instruction 541 and then marked to consider narrower themes. Categories were broken down to reveal more nuanced areas of convergence and divergence, and participant statements were grouped into these categories. Results were tallied and presented in anonymized tables. Findings The findings are structured around the second round questionnaire. Question 1: Are data information literacy skills important to the work of graduate students in meteorology? All 10 panelists agreed that data information literacy skills were essential. Responses did not change between rounds 1 and 2. The majority of participants framed their responses by discussing the importance of these skills for academic success. While all 10 participants identified the relevancy of data information literacy skills in the work of students, four participants, made up of students, professors, and MET researchers, expressed having seen or heard students express confusion when interacting with data. One student explained that “Students learn different things and have different skills […] Sometimes we talk to each other and say ‘Have you been doing it this way? But you can do it this way!’ They say, ‘But I’ve been doing it this way for years!’” Question 2: How do graduate students learn to understand, use, manage, share, work with, and produce data? Formal learning was defined as being organized and structured, taking place intentionally, and occurring through classes, workshops, training sessions, or other formalized environments. In rounds 1 and 2 (see table 1), the responses of the three librarians were categorized as Don’t know, underscoring their peripheral role in meteorology. Nonetheless, two of the three librarians stressed the importance of formally teaching these skills as early as possible in the academic careers of students. For those who stated that skills were obtained informally and formally (four par- ticipants in round 1; five participants in round 2), all noted that only basic skills were taught formally. One professor described the role of informal learning: In undergraduate courses, students have obligatory exercises where they use data and they use MATLAB [a numerical computing environment] to work on the data. Informal learning is also very important. There are two different methods or ways of students developing their skills. One, we have IT personnel who help, and that is a huge help. Secondly, students teach each other and they are quite loyal to that. They use the skills of the senior students. When they themselves become the senior students, they have been very motivated to help the new students. TABLE 1 How Do the Students Learn to Understand, Use, Manage, Share, Work with, and Produce Data Total Number of Responses Students Professors MET Researchers Librarians Informal Process 2 0 1 1 0 Formal Process 0 0 0 0 0 Informal and Formal Process 5 3 1 1 0 Don’t Know 3 0 0 0 3 542 College & Research Libraries July 2016 The other professor described the lack of need for formal instruction: “We don’t have anything formal because the projects vary so much. We have a range of projects, a range of different models being used. So, in a way, it would be of limited value to have a course on how to do this because it’s on a case-by-case basis.” As seen in table 2, seven panelists identified three avenues for informal learning. All three students underscored that they gained minimal skills in undergraduate courses, emphasizing that skills were primarily learned independently and from other students. One noted, “Except from some basic exercises in math during the bachelor, it’s not something that we have learned in the classroom.” Another described, “Mostly, it’s teaching yourself and asking other students, senior students, how they did it. I think most students learn it on our own and go around and ask people.” Question 3: From your perspective, do graduate students in meteorology have proficient data information literacy skills? Responses were consistent between rounds 1 and 2. Librarians could not assess student skills within the discipline (see table 3). The responses of students, professors, and researchers fell on both sides. One student who identified students as having ok to good skills in round 1 qualified her response in round 2: “Yes, we handle data well, but could benefit from learning this at an earlier stage and seeing what we do in a wider [perspective].” All three participants who stated skills were ok to good nonetheless described the challenge of gaining and applying skills. One professor explained: TABLE 2 Sources of Informal Learning Total Number of Responses (could select more than one) Students Professors MET Researchers Librarians From Other Students 5 3 2 0 0 From Discipline Experts: Faculty, IT Staff, Supervisors, Researchers 6 3 2 1 0 Learned Independently 4 3 1 0 0 TABLE 3 Impression of Students’ Data Information Literacy Skills Response Total Number of Respondents Students Professors MET Researchers Librarians Ok to Good Skills 3 2 1 0 0 Skill Could Be Better/Need Improvement 4 1 1 2 0 Don’t Know/Unsure 3 0 0 0 3 Academic Librarians in Data Information Literacy Instruction 543 Yes, students seem to be able to handle this. It’s probably a struggle for students initially. Some students struggle more than others in terms of this. But most students are good about taking the initiative if there is something they do not know or are having a problem with. They’ll go find out. It seems like this is something they tackle. Question 4: Are the appropriate steps currently being taken to ensure data information literacy skills? All the respondents who stated yes, given the context—one MET researcher, one student, and one professor—provided qualified responses (see table 4). They revealed that, although the current system worked given the field of meteorology and types of data used, instruction would be an enhancement. The student said, I think it is ok that [it] goes this way because you can’t predict exactly what kind of data students will use and need. The faculty maybe could take a bit more time teaching this and including this in some of the courses you have to take. There are some big data sets many people use or know. Finding these data sets online, downloading them, handling them, and doing some things with them—that’s something that could have been included. You wouldn’t need a specific course in this, but you could include it a bit more in courses you already have to ensure this is something everyone can handle. The two students who felt that additional training would be beneficial offered insights on what it could include. One shared, It would be nice to have some training or some courses in the first year of the PhD program or in the master’s program. To learn how to deal with data, because that’s what you end up doing but you’re not prepared for it. The training could be included as practical exercises in the different courses. One librarian, who could not assess current training but was nonetheless in favor of it, highlighted the challenges associated with designing such an initiative: From the library’s perspective, we’re wondering where and how to draw the line between the supervisor’s role, when should a teacher from another place in the university come in and teach the overview of things, the ethics of things, when should we take the students out of their habitat and say “you’re not teach- ing the student well enough the data stuff,” how we can do that in a good way TABLE 4 Appropriate Steps Taken to Ensure Students’ Skills Total Number of Responses Students Professors MET Researchers Librarians Yes, Given the Context 3 1 1 1 0 No, Additional Training Would Be Beneficial 4 2 1 1 0 Don’t Know 3 0 0 0 3 544 College & Research Libraries July 2016 to strengthen the research and not add to the possible conflict. It’s complex but I think everyone agrees it can be done better, expect for some groups who are taking care of this on a really local basis. Question 5: Who are the stakeholders associated with data information literacy instruction? From round 1 to 2, three participants revised their comments, either adding new stakeholders or clarifying the roles they envision for existing stakeholders (see table 5). Many highlighted the role of expert knowledge, either by using that term or by identifying disciplinary professionals. One MET researcher explained, “It has to be an expert within the community. It is difficult for someone from outside to do this.” Only two participants identified librarians as having a role—one student and one librar- ian. Even in doing so, this librarian expressed uncertainty as to whether librarians currently possessed the skills to be involved: “I think it should be a combination of researchers, represented by the supervisors, but also I think the library can develop skills to provide support.” The student who identified the library as a stakeholder stated in round 1, “If I’m talking about going out and finding my data, a library wouldn’t be bad in this sense.” She clarified her comments in round 2, writing, “The library could provide the broad overview of the data but IT in each department should probably have the educative responsibilities.” Question 6: Considering the Science Library at the University of Oslo as it is currently, what do you think about the library having a role in data information literacy instruction? TABLE 5 Stakeholders Associated with Data Information Literacy Skills Total Number of Responses (could select more than one) Students Professors MET Researchers Librarians IT 7 3 2 0 2 Faculty 6 2 2 1 1 Data Providers (MET) 3 2 1 0 0 Researchers/ Supervisors 2 0 0 0 2 Librarians 2 1 0 0 1 Experts 2 1 0 1 0 Central Administration 1 0 0 0 1 TABLE 6 Library’s Role in Data Information Literacy Instruction Total Number of Responses Students Professors MET Researchers Librarians Librarians Cannot Play a Role Currently 6 3 1 1 1 Librarians Can Play a Role Currently 4 0 1 1 2 Academic Librarians in Data Information Literacy Instruction 545 A professor provided revisions for round 2 that appeared to move his comments from librarians cannot play a role to librarians can play a role (see table 6). He commented, “I’m sure there are general techniques that the library could teach, and also that this could be valuable. But since most of the data we handle is specific to meteorological applications (observations or models), this is still something the students have to learn in-house, at present.” A MET researcher stated she did not know what role librarians could play due to her own inexperience with libraries: I have no personal experience interacting with a library in this sense so it’s a bit hard to imagine how it would work out. Scientific libraries have been on the outskirts on my consciousness, I’ve not used them actively and they weren’t very visible to me in my student life, so they aren’t the first thing I think about when I think about these specific topics. Five of the six participants who believe that librarians could not currently play a role shared views of library services that centered on the provision of books and articles. Two academic librarians and one MET researcher described views that offered a role for librarians. Yet, their conceptions involved librarians participating through traditional service avenues, such as access to resources and bibliographic instruction. Comments included: “A lot of being a master’s student is seeing what other people are doing. Librar- ians could help the students develop skills to research the literature and to do that in an efficient way.” “Talking about research data, metadata, retrieving or storing data—the academic library should definitely play a role. But, at this point, it would have to be very general instruction.” “I think the situation for meteorology students could be better if they turned to the library more often. I have a feeling they are satisfied with what they find online but [the library] could show them a more practical way of finding information.” TABLE 7 Library’s Future Role in Data Information Literacy Instruction Total Number of Responses Students Professors MET Researchers Librarians Yes, If Librarians Teach General Skills and Have Experts Teach Specific, Technical Skills 8 3 0 2 3 Librarians Are Not Equipped with the Right Skills and Their Role in This Instruction is Uncertain 2 0 2 0 0 546 College & Research Libraries July 2016 Question 7: In the future, could libraries have a role in data information literacy instruction? Prior to asking this question in round 1, the interviewer shared the library’s estab- lished role in providing information literacy instruction and data support in terms of access to and preservation of information. The interviewer also acknowledged that librarians lack the disciplinary expertise and technological skills found in the me- teorology community. The determination to include this came as a result of the pilot interview. The interviewer found it was important for participants to have a sense of abilities and limitations in case participants had limited prior experience with or knowledge of academic libraries. In round 2, one participant added comments to clarify his perspective regarding areas where the library could be involved in the future. This did not result in a category change (see table 7). The majority of participants envisioned a potential role in data information literacy instruction with librarians teaching general concepts. Participants again emphasized literature-based skills. One MET researcher explained: It’s not a bad idea if the library is manned with people with these types of skills. Libraries could be a place for acquiring some competencies in the field that are general, not specific to one line of science. It makes sense in a way to have this located at the library. General data information literacy instruction done by the library with the specific details filled in by faculty. Good documentation and good know-how on standard data formats could be something the library could provide. If librarians are familiar with the most common data formats and how to extract and manipulate data, they could be helpful. It’s almost taken for granted [by] the faculty that you are familiar with that, but it’s not obvious how to acquire that knowledge, so it would be good to have some expertise [in the library]. All three librarians identified future roles and described them in terms that moved beyond traditional instruction. One explained, “The next stage will be students get- ting help, not only with basic things like finding books and journals, but to deal with their own data. I think that will be in the future for libraries.” The three librarians identified how such a role was contingent on librarians continuing to develop skills in the field. One detailed, I think it’s something librarians should work towards. Librarians have a great understanding of how to store, how to access, and how to find information. The understanding of how data could be and should be shared and used and access. I think the understanding is there but the experience and concrete skills are not there yet. It’s a matter of learning how to read data sets of various kinds, not only texts. Two participants’ comments were categorized as Librarians are not equipped with the right skills and their role in this instruction is uncertain. Both professors, one stated: Can libraries teach students something that would be useful for all of them? Me- teorology has a small number of students all doing very different projects. If the issues for meteorology students are similar across fields—in physics and biology, for example—then this instruction could be something librarians could develop. They could develop this type of competency and be a go-to place for students trying to gain proficiency with these skills. For interdisciplinary, collaborative work, if the library could help with that, terrific. But I don’t really know how. Academic Librarians in Data Information Literacy Instruction 547 The other professor echoed uncertainty regarding what support would look like while appearing open to the possibility of involvement: I have a hard time seeing that I should go to the library or librarian for data information literacy. Maybe this is a niche for a library and this is a service that could be useful for meteorology. They would have to operate quite differently than they do today. But, I’m very much in favor of getting loads off my shoul- ders. Any help, and persons to help my students with whatever, that is a bonus to me and my students. Maybe the library could tell me where different data are—maybe that would be fruitful. Question 8: If there are obstacles to library involvement, what are they? While the majority of participants could not identify obstacles or offered unclear responses when directly faced with this question, nine of the 10 participants identified obstacles at other points in the process (see table 8). One of the MET researchers was too unfamiliar with library services to identify obstacles. Half of the participants expressed misgivings about librarians’ skills, often begin- ning by sharing doubts and then providing clarification. One student said, “I’m not sure about the education there [for librarians], if they have the skills, if they work with computers and the data. That’s not where I’m thinking of [when thinking of data information literacy instruction].” Other participants qualified their responses to show how the obstacles could evolve. Two librarians stated that the lack of skills would be less of an issue moving forward. One explained: TABLE 8 Obstacles to Library Involvement in Data Information Literacy Training Number of Respondents (could select more than one) Students Professors MET Researchers Librarians Lack of Skills 5 1 1 0 3 Lack of Meteorological Knowledge/Skills 5 2 2 1 0 Lack of Technical Skills 4 1 1 1 1 Field is Changing/ Developing too Rapidly 4 1 1 1 1 Lack of Data Skills 3 1 1 1 0 Field is too Specialized 3 0 2 0 1 Physical Location of the Library on Campus 2 1 0 0 1 548 College & Research Libraries July 2016 When they hire new people, they demand higher and higher knowledge. It’s quite often people have the basic education in librarianship and then a bachelor’s and master’s in information science. That happens more and more, and it also happens that the library demands those things when they search for new people. I think that will be a development that will go by itself, there will be more and more skilled people working in the library. Half of the participants identified lack of meteorology expertise as a barrier. While the librarian panelists did not, two of the students, both professors, and one of the MET researchers highlighted it. Their comments included: “They do not know the field of meteorology.” “If that person has not been working with meteorology, he or she has not used the data sets I am looking for.” “A lot of the data we use are quite specialized, so it would even require some knowledge of meteorology.” “It is so specialized. It is something that would be difficult. If you have someone with that literacy—that meteorological literacy or oceanographic literacy or something—at the library, then you wonder, maybe they ought to be sitting with us because we’re the ones who are going to be doing that.” Some offered qualifications limiting this obstacle. One student shared, “For the librarians and skills—I think they would need some real connection with the sciences, someone who could understand what you need. But I guess this is something the library can do when it comes to literature, so I don’t know if I see large obstacles.” Considering disciplinary barriers, participants found the field to be too specialized for librarians to gain entry. One professor explained, I think it is too specific. What we typically use libraries for is to find scientific papers or book material—but that is stuff that is based on data, not the data itself. I think that this is too specialized for the libraries […]the librarians would have to go through quite some training to get on top of what is out there in data. [The interviewer asked what type of training would be needed to achieve this.] A computer science background would be useful. I’ve never thought about this so I’m not really sure. A lot of the data we use are quite specialized, so it would even require some knowledge of meteorology. Participants described how rapid developments in the field present an obstacle: “I think that this is too specialized for the libraries. And the thing is this is de- veloping very fast as well. New data sets are popping up and we pick that up at scientific conferences and reading scientific papers. I have a hard time seeing that libraries should play a role in this.” “[Librarians] would not be the first I would think of when it comes to the data I need. It changes quickly.” “The field is rapidly evolving (so that it’s difficult to follow up).” Academic Librarians in Data Information Literacy Instruction 549 Finally, two participants shared the obstacle of the Science Library’s physical loca- tion on campus versus the slightly off-campus location of the Meteorology Section. In speaking about the challenges of involvement, one librarian explained, “We get complaints that the library is too far away.” Similarly, when explaining why he did not see librarians as having a role in data information literacy instruction, one student shared, “we’re located here and they’re on-campus.” Conclusions and Discussion The findings were assessed to identify the importance and adequacy of meteorology graduate students’ data information literacy skills and to examine if librarians have an opportunity and role in instruction. Research question 1: Are data information literacy skills relevant to the work of graduate students in meteorology at the University of Oslo? Yes, data information literacy skills are relevant to the work of meteorology students at the University of Oslo. This question was addressed by way of a unique interview question in round 1. It was the only question to achieve unanimous consensus. All participants stated that these skills were relevant and provided an explanation detailing their importance. This yes/no research question was designed to be straightforward because relevancy of these skills was foundational to the research. Bias could have played a role in the overwhelming valuation of data information literacy skills by the panelists. In the request for participation, they were told that the research would look at data information literacy skills in the field of meteorology. Consequently, participants could have assumed that we perceived these skills to be relevant in meteorology and worthy of research. In an attempt to align their perspective with ours, participants may have overemphasized the importance of data information literacy skills. Yet, all participants offered more than a one-word answer, and many spoke in detail and with enthusiasm. We interpreted this to mean that participants had an acceptable understanding of the role that these skills played and that they perceived genuine relevancy in meteorology. Research question 2: How do graduate students learn data information literacy skills? Students primarily learn these skills informally from other students, discipline experts such as faculty and IT personnel, and independently. Librarians were not identified as playing a role in this learning process. Panelists’ emphasis on discipline- affiliated colleagues in this informal process reflected the findings of information- seeking research in the sciences.36 Research question 3: Do graduate students achieve proficient data information literacy skills through this practice? The majority of panelists believed that skills needed improvement. When asked if the appropriate steps were currently being taken to ensure data information literacy, the ma- jority of participants said they were not and highlighted the value of additional training. The majority of students found their skills and the skills of their peers to be ok to good. Psychological research shows that people tend to overestimate their own abili- ties and those of peers.37 While we do not know if these students overestimated their abilities, self- and peer assessments are poised for inaccuracy. The first three research questions were considered in unison to assess the perception of stakeholders on the current situation and to determine if an unmet need existed. Given the discrepancy between the prioritization of data information literacy skills and the perceived need for student skill improvement, we identified a gap. Research question 4: What is the forecasted role of the academic library in data information literacy instruction of graduate students in meteorology? 550 College & Research Libraries July 2016 Librarians can have a future role in instruction. They must gain significant skills related to the technical work in meteorology to achieve this role, or they must col- laborate with field experts, with librarians teaching general skills and experts teaching technical skills. Participants, including librarians, envisioned librarians as providing general instruc- tion in support of data information literacy due to a perceived lack of disciplinary skills. Research question 5: What is the consensus on the perceptions of librarian involvement in data information literacy instruction for graduate students in meteorology? The consensus of the expert panel was that, while future involvement would be possible, librarians were not perceived as having the skills needed to engage deeply in data information literacy instruction. Therefore, their level of involvement would be limited to the instruction of general skills unless librarians were able to overcome significant obstacles. While data information literacy instruction appears to be an unmet need for graduate students in meteorology, the academic library would need to develop and refine data skills and knowledge internally, or collaborate with discipline-based stakeholders to respond to this need. Participants revealed that they currently do not associate librarians with data infor- mation literacy instruction. When asked to identify the stakeholders associated with data information literacy instruction, only two participants identified librarians—a student and a librarian. When asked to consider the librarians’ role in data information literacy instruction, the majority of participants stated the librarians could not play a role currently. Nonetheless, the majority perceived the potential for librarians to play a role in the future. What will the future bring? The University of Oslo meets demands from funding bodies and society to make research data openly available, as seen internationally at universities and research institutions. To meet these requirements, the institution recently established a working group whose mandate was to: • Map current practices of research data management and the researchers’ needs for storage and sharing of research data. • Suggest principles and guidelines for research data storage and accessibility, with consideration given to researcher interests. • Suggest solutions that are in accordance with demands of funding organiza- tions and needs of researchers.38 A member of the secretariat for the working group came from the University of Oslo’s Science Library; one of the issues discussed in the report is the need for train- ing of graduate students in data management. Although the report does not specify any guidelines on data information literacy training, it states that different parts of the organization need to cooperate to provide adequate courses to improve the skills of faculty and students. The inclusion of the Science Library in this training is natural but requires that the subject specialists within the library work closely with faculty at their respective departments. Notes 1. Javier Calzada Prado and Miguel Ángel Marzal, “Incorporating Data Literacy into Informa- tion Literacy Programs: Core Competencies and Contents,” Libri 63, no. 2 (2013): 123–34; Tibor Koltay, “Data Literacy: In Search of a Name and Identity,” Journal of Documentation 71, no. 2 (Feb. 18, 2015): 401–15, doi:10.1108/JD-02-2014-0026; René Schneider, “Research Data Literacy,” in Worldwide Commonalities and Challenges in Information Literacy Research and Practice, eds. Serap Kurbanoğlu et al., Communications in Computer and Information Science 397 (Springer International Publishing, 2013), 134–40, available online at http://link.springer.com/chapter/10.1007/978-3-319-03919-0_16 Academic Librarians in Data Information Literacy Instruction 551 [accessed 23 May 2016]; Jian Qin and John D’ignazio, “The Central Role of Metadata in a Science Data Literacy Course,” Journal of Library Metadata 10, no. 2/3 (Aug. 31, 2010): 188–204, doi:10.108 0/19386389.2010.506379. 2. Jacob Carlson et al., “Determining Data Information Literacy Needs: A Study of Students and Research Faculty,” portal: Libraries and the Academy 11, no. 2 (2011): 629–57, doi:10.1353/ pla.2011.0022. 3. Claire J Anderson et al., “An Investigation of Factors Affecting How Engineers and Scientists Seek Information,” Journal of Engineering and Technology Management 18, no. 2 (June 2001): 152, doi:10.1016/S0923-4748(01)00032-7. 4. Ibid., 149. 5. Julie Hallmark, “Information-Seeking Behavior of Academic Meteorologists and the Role of Information Specialists,” Science & Technology Libraries 21, no. 1/2 (2001): 57–59, doi:10.1300/ J122v21n01_06. 6. Julie Hallmark, “Access and Retrieval of Recent Journal Articles: A Comparative Study of Chemists and Geoscientists,” Issues in Science and Technology Librarianship 40 (summer 2004), doi:10.5062/F40G3H3M. 7. Julie Hallmark, “Information-Seeking Behavior of Meteorologists and Other Atmospheric Scientists: Access and Retrieval of Cited References,” Issues in Science and Technology Librarian- ship 38 (summer 2003), doi:10.5062/F4348H9P. 8. Ibid. 9. Ann Gates et al., “Towards Secure Cyberinfrastructure for Sharing Border Information,” Departmental Technical Reports (CS) (Mar. 1, 2006): 5, available online at http://digitalcommons. utep.edu/cs_techrep/151 [accessed 23 May 2016]. 10. Ibid., 5–6. 11. Janet Murphy, “Information-Seeking Habits of Environmental Scientists: A Study of Interdisciplinary Scientists at the Environmental Protection Agency in Research Triangle Park, North Carolina,” Issues in Science and Technology Librarianship 38 (summer 2003), doi:10.5062/ F4PV6HBW. 12. Jake Carlson et al., “Developing an Understanding of Data Management Education: A Report from the Data Information Literacy Project,” Libraries Faculty and Staff Presentations (Jan. 1, 2013), available online at http://docs.lib.purdue.edu/lib_fspres/11 [accessed 23 May 2016]. 13. Sarah Wright et al., “A Multi-Institutional Project to Develop Discipline-Specific Data Literacy Instruction for Graduate Students,” Libraries Faculty and Staff Presentations (Dec. 1, 2012), available online at http://docs.lib.purdue.edu/lib_fspres/10 [accessed 23 May 2016]. 14. Ibid. 15. Laura Molloy and Kellie Snow, “The Data Management Skills Support Initiative: Synthesis- ing Postgraduate Training in Research Data Management,” International Journal of Digital Curation 7, no. 2 (Oct. 23, 2012): 102, doi:10.2218/ijdc.v7i2.233. 16. Lori Jahnke, Andrew Asher, and Spencer Keralis, “The Problem of Data” (Washington, D.C.: Council on Library and Information Resources, Aug. 1, 2012), available online at http:// digitalcommons.bucknell.edu/fac_pubs/52 [accessed 23 May 2016]. 17. Ibid., 14. 18. Ibid., 3–4. 19. D. Scott Brandt, “Librarians as Partners in E-Research Purdue University Libraries Promote Collaboration,” College & Research Libraries News 68, no. 6 (June 1, 2007): 365. 20. Jeanine Marie Scaramozzino, Marisa L. Ramírez, and Karen J. McGaughey, “A Study of Faculty Data Curation Behaviors and Attitudes at a Teaching-Centered University,” College & Research Libraries 73, no. 4 (July 1, 2012): 360. 21. C.A. Strasser and S.E. Hampton, “The Fractured Lab Notebook: Undergraduates and Ecological Data Management Training in the United States,” Ecosphere 3, no. 12 (Dec. 2012): 3, doi:10.1890/ES12-00139.1. 22. Ibid., 10. 23. Tracy Primich, “A Semester-Long Seminar in Statistical Visualization for Undergraduates as Taught by a Science and Engineering Librarian,” Science & Technology Libraries 29, no. 3 (2010): 181–82, doi:10.1080/0194262X.2010.497702. 24. Mary Piorun et al., “Teaching Research Data Management: An Undergraduate/Graduate Curriculum,” Journal of eScience Librarianship 1, no. 1 (Feb. 14, 2012), doi:http://dx.doi.org/10.7191/ jeslib.2012.1003. 25. Elizabeth Stephenson and Patti Schifter Caravello, “Incorporating Data Literacy into Undergraduate Information Literacy Programs in the Social Sciences: A Pilot Project,” Reference Services Review 35, no. 4 (Nov. 13, 2007): 525, doi:10.1108/00907320710838354. 26. Karen Antell et al., “Dealing with Data: Science Librarians’ Participation in Data Manage- ment at Association of Research Libraries Institutions,” College & Research Libraries 75, no. 4 (July 552 College & Research Libraries July 2016 1, 2014): 557–74, doi:10.5860/crl.75.4.557. 27. Charles Humphrey, “Collaborative Training in Statistical and Data Library Services,” Resource Sharing & Information Networks 18, no. 1/2 (2005): 179, doi:10.1300/J121v18n01_13. 28. Sally A. Gore, “Shaping Up: Boot Camp and Other Programs Addressing Professional Development Needs of Science Librarians,” Medical Reference Services Quarterly 30, no. 2 (2011): 130–40, doi:10.1080/02763869.2011.562781. 29. Antell et al., “Dealing with Data.” 30. Birte Christensen-Dalsgaard et al., “Ten Recommendations for Libraries to Get Started With Research Data Management” (The Hague: LIBER, 2012), available online at https://www. fosteropenscience.eu/content/ten-recommendations-libraries-get-started-research-data-manage- ment [accessed 23 May 2016]. 31. Felicity Hasson and Sinead Keeney, “Enhancing Rigour in the Delphi Technique Re- search,” Technological Forecasting and Social Change 78, no. 9 (Nov. 2011): 1695–1704, doi:10.1016/j. techfore.2011.04.005. 32. Christoph O. Meyer, “The Purpose and Pitfalls of Constructivist Forecasting: Insights from Strategic Culture Research for the European Union’s Evolution as a Military Power,” International Studies Quarterly 55, no. 3 (2011): 670, doi:10.1111/j.1468-2478.2011.00648.x. 33. Victoria Story et al., “Methodological and Practical Implications of the Delphi Technique in Marketing Decision-Making: A Re-Assessment,” Marketing Review 1, no. 4 (2000): 487–504, doi:10.1362/1469347002562579. 34. Blazej Feret and Marzena Marcinek, “The Future of the Academic Library and the Academic Librarian: A Delphi Study,” Librarian Career Development 7, no. 10 (Oct. 1, 1999): 38, doi:10.1108/09680819910301898. 35. I.E. Seidman, Interviewing as Qualitative Research: A Guide for Researchers in Education and the Social Sciences (New York: Teachers College Press, 2013). 36. C.J. Anderson et al., “An Investigation of Factors Affecting How Engineers and Scientists Seek Information,” Journal of Engineering and Technology Management 18, no. 2 (June 1, 2001): 131–55, doi:10.1016/S0923-4748(01)00032-7. 37. Justin Kruger and David Dunning, “Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments,” Journal of Personality and Social Psychology 77, no. 6 (1999): 1121–34, doi:10.1037/0022-3514.77.6.1121. 38. Universitetet i Oslo, “Dataeksplosjonen—En Stor Utfordring, Og En Gedigen Mulighet!: Rapport Fra Arbeidsgruppen «Lagring Og Deling Av Forskningsdata» Ved Universitetet I Oslo (UiO) 11.05.2015” (Oslo: Universitetet i Oslo, 2015).