Primo New User Interface: Usability Testing and Local Customizations Implemented in Response Blake Lee Galbreath, Corey Johnson, and Erin Hvizdak INFORMATION TECHNOLOGY AND LIBRARIES | JUNE 2018 10 Blake Lee Galbreath (blake.galbreath@wsu.edu) is Core Services Librarian, Corey Johnson (coreyj@wsu.edu) is Instruction and Assessment Librarian, and Erin Hvizdak (erin.hvizdak@wsu.edu) is Reference and Instruction Librarian, Washington State University. ABSTRACT Washington State University was the first library system of its 39-member consortium to migrate to Primo New User Interface. Following this migration, we conducted a usability study in July 2017 to better understand how our users fared when the new user interface deviated significantly from the classic interface. From this study, we learned that users had little difficulty using basic and advanced search, signing into and out of primo, and navigating their account. In other areas, where the difference between the two interfaces was more pronounced, study participants experienced more difficulty. Finally, we present customizations implemented at Washington State University to the design of the interface to help alleviate the observed issues. INTRODUCTION A July 2017 usability study by Washington State University (WSU) Libraries was the final segment of a six- month process for migrating to the new user interface of Ex Libris Primo called Primo New UI. WSU Libraries assembled a working group in December 2016 to plan for the migration from the classic interface to Primo New UI and met bi-weekly through May 2017. To start, the Primo New UI working group attempted to answer some baseline questions: What can and cannot be customized in the new interface? How, and according to what timeline, should we introduce the new interface to our library patrons? What methods could be used to assess the new interface? This working group customized the look and feel of the new interface to conform to WSU branding and then released a beta version of Primo New UI in March, leaving the older interface (Primo Classic) as the primary means of access to Primo but allowing users to enter and test the beta version of the new interface. In early May (at the start of the Summer semester), the prominence of the old and new interfaces was reversed, making Primo New UI the default interface but leaving the possibility of continued access to Primo Classic. The older interface was removed from public access in mid-August, just prior to the start of the Fall semester. The public had the opportunity to work with the beta version from March to May and then another two months experience with the production release by the time the usability study took place in July 2017. The remainder of this paper will focus on the details of this usability study. mailto:blake.galbreath@wsu.edu mailto:coreyj@wsu.edu mailto:erin.hvizdak@wsu.edu PRIMO NEW USER INTERFACE | GALBREATH, JOHNSON, AND HVIZDAK 11 https://doi.org/10.6017/ital.v37i2.10191 RESEARCH QUESTIONS Primo New UI was the name given to the new front end of the Primo discovery layer, which was made available to customers in August 2016. According to Ex Libris, “Its design is based on user studies and feedback to address the different needs of different types of users.”1 We were primarily interested in understanding the usability of the essential functionalities of Primo New UI, especially where the design of the new interface deviated significantly from the classic interface (taking local customizations into account). For example, we noted that the new interface introduced the following differences to the user (this ordinal list corresponds to the number labels in figure 1): 1. Basic Search tabs were expressed as drop-downs. 2. The Advanced Search link was less prominent than it was with our customized shape and color in the classic interface. 3. Main Menu items were located in a separate area from the Sign In and My Account links. 4. My Favorites and Help/Chat icons were located together and in a new section of the top navigation bar. 5. Sign In and My Account links were hidden beneath a “Guest” label. 6. Facet values were no longer associated with checkboxes or underlining upon hover. 7. Availability statuses were expressed through colored text. Figure 1. Basic search screen in Primo New UI. We also observed a fundamental change in the structure of the record in Primo New UI: the horizontally oriented and tabbed structure of the classic record (see figure 2) was converted to a vertically oriented and non-tabbed structure in the new interface (see figure 3). Additionally, the tabbed structure of the classic interface opened in a frame of the Brief Results area, while the same information was displayed on the Full Display page of the new interface. The options displayed in these areas are known as Get It and View It (although we locally branded our sections Availability and Request Options and Access Options, INFORMATION TECHNOLOGY AND LIBRARIES | JUNE 2018 12 respectively). Therefore, we were eager to see how this change in layout might affect a participant’s ability to find Get It and View It information on the Full Display page. Taking the above observations into account, we formulated the following questions: 1. Will the participant be able to find and use the Basic Search functionality? 2. Will the participant be able to understand the availability information of the brief results? 3. Will the participant be able to find and use the Sign In and Sign Out features? 4. Will the participant be able to understand the behavior of the facets? 5. Will the participant be able to find and use the Actions Menu? (See the “Send to” boxed area in figure 3.) 6. Will the participant be able to navigate the Get It and View It areas of the Full Display page? (See the “Availability and Request Options” boxed area in figure 3.) 7. Will the participant be able to navigate the My Account area? 8. Will the participant be able to find and use the Help/Chat and My Favorites icons? 9. Will the participant be able to find and use the Advanced Search functionality? 10. Will the participant be able to find and use the Main Menu items? (See figure 1, number 3.) Figure 2. Horizontally oriented and tabbed layout of Primo Classic. LITERATURE REVIEW 2012 witnessed a flurry of studies involving Primo Classic. Majors compared the experiences of users within the following discovery interfaces: Encore Synergy, Summon, WorldCat Local, Primo Central, and EBSCO Discovery Service. The study used undergraduate students enrolled at the University of Colorado and focused on common undergraduate searching activities. Each interface was tested by five or six participants who also completed an exit survey. Observations specific to the Primo interface noted that users had difficulty finding and using existing features, such as email and e-shelf, and difficulty connecting their failed searches to interlibrary loan functionality.2 PRIMO NEW USER INTERFACE | GALBREATH, JOHNSON, AND HVIZDAK 13 https://doi.org/10.6017/ital.v37i2.10191 Figure 3. Vertically oriented and non-tabbed Layout of Primo New UI. Comeaux noted issues relating to terminology and the display of services during usability testing carried out at Tulane University. Twenty people, including undergraduates, graduates, and faculty members, participated in this study, which tested five typical information-seeking scenarios. The study found several problems related to terminology. For example, participants did not fully understand the meaning of the Expand My Results functionality.3 Participants also did not understand that the display text “No full-text” could be used to order an item via Interlibrary Loan. 4 The study also concluded that the mixed presentation of differing resource types (e.g., books, articles, reviews) was confusing for patrons who were attempting known-item searches.5 Jarrett documented a usability study conducted at Flinders University Library. The aims of the study were to determine user perceptions regarding the usability of the discovery layer, the relevance of the information retrieved, and the user experiences of this search interface compared to other interfaces. 6 The usability portion of the study scored the participants’ completion of tasks in the Primo discovery layer as difficult, confusing, neutral, or straightforward. Scores indicated that participants had difficulty determining different editions of a book, locating a local thesis, and placing an item on hold. The investigators also observed that students had issues signing into Primo and distinguishing between journals and journal articles.7 INFORMATION TECHNOLOGY AND LIBRARIES | JUNE 2018 14 Nichols et al. conducted a usability test on a newly implemented Primo instance at the University of Vermont Libraries in 2012. Their research questions were designed to understand Primo’s design, functionality, and layout.8 The majority of the participants were undergraduate students. Similar to Comeaux, confusion occurred when participants had to find specific or relevant records within longer sets of results.9 Nichols et al. also noticed that test subjects had difficulty navigating and finding information in the Primo tabbed structure. Like Jarrett, Nichols et al. noted that participants had difficulty distinguishing between the journals and articles.10 Similar to Majors, participants in Nichols et al. had difficulty finding certain Primo functionality, such as email, the e-Shelf, and the feature to open items in a new window.11 The investigators concluded that these tools were difficult to find because they were buried too deep in the interface. The University of Kansas Libraries conducted two usability studies on Primo. The first study took place during the 2012–13 academic year and involved 27 participants, including undergraduate, graduate, and professional students, who performed four to five main tasks in two separate sessions. Similar to other studies, participants experienced great difficulty using the Save to E-shelf and Email Citation tools.12 Kliewer et al. conducted the second usability study in 2016, which focused primarily on student satisfaction with the Primo discovery tool. Thirty undergraduates participated in this study that collected both qualitative and quantitative data. In contrast to most usability studies of discovery services, this study allowed participants to explore Primo with open-ended searches to more closely mimic natural searching strategies. Results of the study indicated that the participants preferred Basic Search to Advanced Search, used facets (but not enough to maximize their searching potential), rarely moved beyond the first page of search results, and experienced difficulties using the link resolver. In response to the latter, a Primo working group clarified language on the link resolver page to better differentiate between links to articles and links to journals.13 Brett, Lierman, and Turner conducted a usability study at the University of Houston Libraries focusing primarily on undergraduate students. Users were able to complete the assigned tasks, but the majority did not do so in the most efficient manner. That is, the participants did not take full advantage of Primo functionality, such as facets, holds, and recalls. Additionally, some participants exhibited difficulty deciphering among the terms journals, journal articles, and newspaper articles. Another difficulty participants experienced was knowing what further steps to take once they had successfully found an item in the results list. For example, participants had trouble locating stacks guides, finding request features, and using call numbers. The researchers concluded that many of the issues witnessed in this usability study could be mitigated via library instruction.14 Usability testing of Primo New UI has recently begun to take a foothold in academic libraries. In addition to conducting usability testing on the Primo Classic in April 2015 (5 participants, 5–6 tasks), researchers at Boston University carried out both pre- and post-launch testing of the new interface in December 2016 and April 2017, respectively. Pre-launch testing with five student participants identified issues with “labelling, locating links to online services, availability statement links in full results, [and] My Favorites.”15 After completing fixes, post-launch testing with four students (2 infrequent users, 2 frequent) found that they were able to easily complete tasks, use filters, save results, and find links to online resources. Usage statistics for the new interface, compared to classic, also showed an increased use of facets after fixes, and an increase in the use of some features but decrease in the use of others, providing information on what features warranted further examination.16 PRIMO NEW USER INTERFACE | GALBREATH, JOHNSON, AND HVIZDAK 15 https://doi.org/10.6017/ital.v37i2.10191 California State University (CSU) libraries conducted usability studies on Primo New UI with 24 participants (undergraduate students, graduate students, and faculty) across five CSU campuses. Five standard tasks were required: find a specific book, find a specific film, find a peer-reviewed journal article, find an item in the CSU network not owned locally, and find a newspaper article. Each campus added additional questions based on local needs. Participants were overwhelmingly positive about the interface look and feel, ease of use, and speed of the system. The success rate for each task varied across the campuses, with participants having greater success on simple tasks such as finding a specific or known item and mixed results on more difficult tasks including using scopes, understanding icons and elements of the FRBR record, and facets. Steps were taken to relabel and rearrange the scopes and facets so that they were more meaningful to users, and FRBR icons were replaced. The authors concluded that Primo is an ideal solution to incorporate both global changes and local preference because of its customizability.17 University of Washington Libraries conducted usability studies on the classic and new Primo interfaces. The Primo New UI study observed 12 participants. Each 60-minute session included an orientation, pre- and post-tests, tasks, and follow-up questions. Difficulties were noted with terminology, the site logo, the inability to select multiple facets, unclear navigation, volume requesting, Advanced Search logic, the pin location in item details, and the date facet. A/B testing with 12 participants (from both the New and C lassic UI studies) revealed the need to fix the Sign-In prompt for My Favorites, enable libraries to add custom actions to the actions menu, add a sort option for favorites in the new interface, add the ability to rearrange elements on a single item page, and add Zotero support. Overall, participants preferred the new interface. Generally, participants easily completed basic tasks, such as known-item searches, searches for course reserves, and open searches, but had more difficulty with article subject searching, audio/visual subject searching, and print-volume searching, which was consistent from the classic to the new interfaces for student participants.18 METHOD We conducted a diagnostic usability evaluation of Primo New UI using eight participants, whom we recruited from the WSU faculty, staff, and student populations. In the end, we received a skewed distribution among the categories: three members of staff and five students (two undergraduate students and three graduate students). The initial composition of the participants comprised a greater number of undergraduate students, but substitution created the final makeup. All the study participants had some exposure to Primo Classic in the past. We recruited participants by hanging flyers around the libraries of our Pullman campus and the adjoining student commons area. We offered the participants $15 in exchange for their time, which we advertised as being a maximum of one hour. The usability test was designed by a team of three library staff, one from Systems (IT) and two from Research Services (reference/instruction). Two of us were present at each session, one to read the tasks aloud and the other to document the session. We used Camtasia to record each session so that we would have the ability to return to it later if we needed to verify our notes or other specifics of the session. We stored the recordings on a secured share of the internal library drive. We received an Institutional Review Board Certificate of Exemption (IRB #16190) to conduct this study. INFORMATION TECHNOLOGY AND LIBRARIES | JUNE 2018 16 This usability test comprised eleven tasks (see appendix A) to test the research questions described above. The tasks were drafted in consultation with the Ex Libris set of recommendations for conducting Primo usability testing.19 Each investigator drew their conclusions as to the participants’ successes and failures. We then met as a group to form a consensus regarding task success and failure (see appendix B). We met to discuss the patterns that emerged and to formulate remedies to problems we perceived as hindering student success. RESULTS For each of the ten research questions below, consult appendix B to see details regarding the associated tasks and how each participant approached and completed each task. Task set(s) related to research question 1: Will the participant be able to find and use the Basic Search functionality? This was one of the easier tasks for the participants to complete. Some participants did not follow the task literally to find their favorite book or movie, but rather completed a search for an item or topic of interest to them. All the participants completed this task successfully. Task set(s) related to research question 2: Will the participant be able to understand the availability information of the brief results? The majority of the participants understood that the availability text and its color represented important access information. However, there were instances where the color of the availability status was in conflict with its text. This led at least one participant to evaluate the availability of a resource incorrectly. Task set(s) related to research question 3: Will the participant be able to find and use the Sign In and Sign Out features? The participants all successfully completed this task. Participants used multiple methods to sign in: the Guest link in the top navigation bar, the Sign In link from the ellipsis Main Menu Item, and the Get It Sign In link on the Full Display page. All participants signed out via the User link in the top navigation bar. Task set(s) related to research question 4: Will the participant be able to understand the behavior of the facets? Almost all of the participants were able to select the Articles facet without issue. One person, however, misunderstood the include behavior of the facets. Instead of using the include behavior, this participant used the exclude behavior to remove all facets other than the Articles facet. Only two participants attempted to use the Print Books facet to complete the task, “From the list of results, find a print book that you would need to order from another library.” Instead, the other 75 percent simply scanned the list of results to find the same information. Five out of the eight participants attempted to find the Peer-Reviewed facet when completing the task to choose any peer-reviewed article from a results list: three were successful, while one selected the Newspaper Articles facet, and another selected the Reviews facet. Task set(s) related to research question 5: Will the participant be able to find and use the Actions Menu? The tasks related to the Actions Menu (copy a citation and email a record) were some of the most difficult for the participants: two were successful, three had some difficulty, and three were unsuccessful. Of those PRIMO NEW USER INTERFACE | GALBREATH, JOHNSON, AND HVIZDAK 17 https://doi.org/10.6017/ital.v37i2.10191 who experienced difficulty, one seemed not to understand the task fully; this participant found and copied the citation, but then spent additional time looking for a “clipboard.” The other two participants were both distracted by competing areas of interest: the Citations section of the Full Display and the section headings of the Full Display. Of those who were unsuccessful, one suffered from a technical issue that Ex Libris needs to resolve (the functionality to expand the list of action items failed), one did not seem to understand what a citation was when they found it, and another could not find the email functionality. This last subject continued searching in the ellipsis area of the Main Menu, in the My Account area, and the facets, but ultimately never found the Email icon in the scrolling section of the Actions Menu. Task set(s) related to research question 6: Will the participant be able to navigate the Get It and View It areas of the Full Display page? Three participants experienced substantial difficulty in completing this set of tasks. These participants were distracted by the styled Show Libraries and Stack Chart buttons on the Full Display page that were competing for attention with the requesting options. Task set(s) related to research question 7: Will the participant be able to navigate the My Account area? All of the participants completed this task successfully. Four participants located the back-arrow icon to exit the My Account area, while the other four participants used alternate methods: using the library logo, selecting the New Search button, and signing out of Primo. Task set(s) related to research question 8: Will the participant be able to find and use the Help/Chat and My Favorites icons? Participants encountered very little difficulty in finding a way to procure help and chat with a librarian, with one exception. Participant 2 immediately navigated to and opened our Help/Chat icon, but then moved away from this service because it opened in a new tab. This same participant, along with three others, had a more difficult time finding and deciding to use the Pin this Item icon than did the three participants who completed the same task with ease. The remaining participant failed to complete this task because they could not find the My Favorites area of Primo. Task set(s) related to research question 9: Will the participant be able to find and use the Advanced Search functionality? One participant had more trouble finding the Advanced Search functionality than the other seven. Another experienced a technical difficulty, in which the Primo screen froze during the experiment, and we had to begin the task anew. The remaining six people easily finished the tasks. Task set(s) related to research question 10: Will the participant be able to find and use the Main Menu items? The majority of the participants completed this task with ease, navigating to the Databases link in the Main Menu items. One participant, however, was confused by the term database but was able to succeed once we provided a brief definition of the term. The remaining two participants were further confused by the term and instead entered general search terms into the Primo search bar. These two participants failed to find the list of databases. DISCUSSION INFORMATION TECHNOLOGY AND LIBRARIES | JUNE 2018 18 Study participants completed four of our task sets with relative ease: using Basic Search (see research question 1 above), signing into and out of Primo (see research question 3 above), navigating their My Account area (see research question 7 above), and using Advanced Search (see research question 9 above). There was one exception: one participant experienced minor trouble finding the Advanced Search link, checking first among the drop-down options on our Basic Search page. Subsequent and unrelated to this study, WSU elected to eliminate the first set of drop-down options from our Primo landing page. Further testing might tell us if this elimination in the number of drop-down options has effectively made the Advanced Search link more prominent for users. Also, the ease with which participants were able to use items located underneath the “Guest” label contradicted our expectations. We predicted that this opacity would cause users issues, but it did not seem to deter them. From this, we concluded that the placement of the sign in options in the upper right corner is sufficient to maintain continuity. Participants encountered a moderate degree of difficulty completing two task sets: determining availability statuses and navigating the Get It area of the Full Display page. Concerning availability, participants were quick to understand that statuses such as “Check holdings” relayed that the item was not available. The participants were also keen to notice that green availability statuses implied access while non -green availability statuses implied non-access. However, per the design of the new interface, certain non-green links became green after opening the Full Display page of Primo. This was a significant deviation from the classic interface, where colors indicating availability status did not change. This design element misled one participant. Of note, we did not observe participants experiencing issues with the converted format of the Get It and View It areas (see figures 2 and 3) per se. However, we did notice that three of our participants were unnecessarily distracted by the Show Libraries link when trying to find resource sharing options because WSU had previously styled the Show Libraries links with color and shape. Therefore, our local branding in this area impeded usability and led us to rethink the hierarchy of actions on the Full Display page. Similar to comments made by DeMars, study participants also remarked that the layout of the Full Display was cluttered and difficult to read.20 We therefore took steps to make this page more readable for the viewer. Study participants displayed the greatest difficulty completing the remaining four task sets: selecting a Main Menu item, refining a search via the facets, using the Actions Menu, and navigating the My Favorites functionality. However, web design was not necessarily the culprit in all four areas. Three participants experienced difficulty finding the Databases link (a Main Menu item). After further discussion, it became apparent that this trouble related not to usability but to information literacy—they did not understand the term databases. Therefore, like Majors and Comeaux,21 we recognize the recurring issue of library jargon, and like Brett, Lierman, and Turner,22 we believe that this issue would best be mitigated via library instruction. In agreement with the literature, two participants selected the incorrect facet because they had difficulty distinguishing among the terms articles, newspaper articles, reviews and peer-reviewed.23 Further, one of these participants experienced even more difficulty because of not understanding the inherent functionality of the facet values. That is, this participant did not grasp that the facet value links performed an inclusion process by default. To the contrary, this person believed that they would have had to exclude all unwanted facet values to arrive at the wanted facet value. The change in facet behavior between classic and new interfaces likely caused this confusion. In Primo Classic, WSU had installed a local customization that provided checkboxes and underlining upon hover for each facet value. The new interface did not PRIMO NEW USER INTERFACE | GALBREATH, JOHNSON, AND HVIZDAK 19 https://doi.org/10.6017/ital.v37i2.10191 provide either one of these clues to the user. Additionally, we observed, similar to Kliewer et al. and Brett, Lierman, and Turner, that participants oftentimes preferred to scan the results list over refining their search via faceting.24 This finding also matches a 2014 Ex Libris user study indicating that users are easily confused by too many interface options and thus tend to ignore them.25 Regarding the Actions Menu, the majority of the participants attempted to find the Email icon in the correct section of the Full Display page (i.e., the “Send To” section). However, because of a technical issue in the design of the new interface, the Email icon was not always present for the participant to find. For others, it was difficult to reach the icon even when it was present as participants had to click the right arrow three to four times to navigate past all the citation manager icons. This observed difficulty in finding existing functionalities in Primo echoes that cited by Majors and Nichols et al.26 Participants also experienced significant difficulty deciphering between the similarly named functionalities of the Citation icon and the Citations section of the Full Display page. As a result of this observed difficulty, we concluded that differentiating sections of the page with distinct naming conventions would be beneficial to users. Like the results reported by Boston University, our study participants encountered significant issues when trying to save items into their My Favorites list.27 We noticed that participants had difficulty making connections between the icons named Keep this Item/Remove this Item and the My Favorites area. During testing, it was clear that many of the participants were drawn to the pin icon for the correctly anticipated functionality but then were confused that the tooltips did not include any language resembling “My Favorites.” From this last observation, we surmised that providing continuity in language between these icons and the My Favorites area would increase usability for our library patrons. Pepitone reported problems with the placement of the My Favorites pin icon,28 but we observed this being less of a problem than the actual terminology used to name the pin icon. Beyond success and failure, a 2014 Ex Libris user study suggested that academic level and discipline play a key role in user behavior.29 However, we were unable to draw meaningful conclusions among user groups because of our small and homogenous participant pool. DECISIONS MADE IN RESPONSE TO USABILITY RESULTS Declined to Change Facets. Although one participant did not understand the inclusion mechanism of the facet values, we declined to investigate a customization in this area. According to the Primo August 2017 release notes, Ex Libris plans to make considerable changes to the faceting functionality.30 Therefore, we decided to wait until after this release to reassess whether customization was warranted. Implemented a Change Labels Citations. We observed confusion between the Citation icon of the Actions Menu and the section of the Full Display page labeled “Citations.” To differentiate between the two items, we changed the Actions Menu icon text to “Cite This Item” (see figure 4) and the heading for the Citations section to “References Cited” (see figure 5). INFORMATION TECHNOLOGY AND LIBRARIES | JUNE 2018 20 Figure 4. Cite This Item icon of the Actions Menu. Figure 5. References Cited section of the Full Display page. My Favorites. There was a mismatch among the tooltip texts of the My Favorites icons. We changed the tooltip language for the “Keep this item” pin to read “Add to My Favorites” (see figure 6) and the tooltip language for the “Unpin this item” pin to read “Remove from My Favorites” (see figure 7). Figure 6. Add to My Favorites language for My Favorites Tooltip. Figure 7. Remove from My Favorites language for My Favorites Tooltip. Availability Statuses. Per the design of the new interface, certain non-green links became green after opening the Full Display page of Primo New UI. We implemented CSS code to retain the non-green coloring of the availability statuses after opening the Full Display. In this case, “Check holdings” remains orange (see figure 8). Figure 8. Availability status color of Brief Display, before and after opening the Full Display. PRIMO NEW USER INTERFACE | GALBREATH, JOHNSON, AND HVIZDAK 21 https://doi.org/10.6017/ital.v37i2.10191 Link Removal Full Display Page Headings. There was confusion as to the function of the headings on the Full Display page. These are anchor tags, but patrons clicked on them as if they were functional links. No patrons used the headings successfully. Therefore, we hid the headings section via CSS (see figure 9). Figure 9. Removal of headings on Full Display page. Links to Other Institutions. We observed participants attempting to use the links to other institutions to place resource sharing requests. Therefore, we removed the hyperlinking functionality of the links in the list, via CSS (see figure 10). Figure 10. Neutralization of links to other institutions. Prioritized the Emphasis of Certain Functionalities Request Options and Show Libraries Buttons. It is usually more important to be able to place a request than find the names of other institutions who own an item. However, the Show Libraries button was originally styled with crimson coloring, which drew unwarranted attention, while the requesting links were not. Therefore, we added styling to the resource-sharing links and removed styling from the Show Libraries button via CSS (see figure 11). Figure 11. Resource sharing link with crimson color, Show Libraries removed of styling. INFORMATION TECHNOLOGY AND LIBRARIES | JUNE 2018 22 E-mail Icon. We observed that the E-mail icon of the Actions Menu was difficult to find. Therefore, we decreased the number of icons and moved the emailing functionality to the left side of the Actions Menu (see figure 12). Figure 12. Email icon prioritized over Citation Manager icons. Contrast and Separation Full Display Page Sections. Participants noted that the information on the Full Display page tended to run together. To remedy, we created higher contrast between the foreground and background of the page sections via CSS. We also styled the section titles and dividers with color, among other edits (see figure 13). Figure 13. Separated sections of Full Display page (see figure 3 to compare to the New UI default Full Display page design). PRIMO NEW USER INTERFACE | GALBREATH, JOHNSON, AND HVIZDAK 23 https://doi.org/10.6017/ital.v37i2.10191 CONCLUSION While providing one of the first studies on Primo New UI, we acknowledge several limitations. Previous studies on Primo had larger study populations compared to this one (which had eight participants). However, we adhered to Nielsen’s findings that usability studies uncover most design deficiencies with five or more participants.31 Additionally, the scope of this study was limited to the usability of the desktop view. We recommend further studies that will concentrate on accessibility compliance and that will test the interface on mobile devices. Regarding the study design, the question arose as to whether the participants’ difficulties reflected poor design functionality or a misunderstanding of library terminology (as noted by Majors and Comeaux).32 The researchers did not carry out pre-tests or an assessment of participants’ level of existing knowledge. This limitation is almost always unavoidable, however, as a task list will always risk not fitting the skills or knowledge of every participant. The lack of some features’ use also might have been because of study design. While not using the facets may reflect that participants are unaware of them, it could also be from the fact that they never had to scroll past the first few items to find the needed resource. Users might have felt a greater need to use the facets had we asked more difficult discovery tasks. The study also contained an investigative bias in that the researchers were part of the working group that developed the customized interface, and then tested those customizations. This bias could have been reduced if the study had used researchers who were not a part of the same group that made these customizations. Despite these limitations, there are still key findings of note. Tasks that participants completed with the greatest ease mapped to those that we assume they do most often, which included basic searching for materials and accessing account information. Tasks beyond these basics proved to be more difficult. This raises the question of whether difficulties were really a function of the interface design or if they reflected ongoing literacy issues. Therefore, it is crucial that designers work with public services and instruction librarians to identify areas where users might be well-served by making certain functionalities more user- friendly and creating educational and training opportunities to increase awareness of these functionalities.33 Bringing diverse perspectives into the study is also crucial so that researchers can discover and be more conscious of commonalities in design and literacy needs, particularly regarding advanced tasks. INFORMATION TECHNOLOGY AND LIBRARIES | JUNE 2018 24 APPENDIX A: USABILITY TASKS Note: Search It is the local branding for Primo at Washington State University. 1) Please search for your favorite book or movie. a) Is this item available for you to read or watch? b) How do you know that this item is or isn’t available for you to read or watch? 2) Please sign in to Search It. 3) Please perform a search for “causes of world war II” (do not include quotation marks). a) Limit your search results to Articles. b) For any of the records in your search results list: i) Find the citation for any item and copy it to the clipboard. ii) Email this record to yourself. 4) Please perform a search for “actor’s choice monologues” (do not include quotation marks). a) From the list of results, find a print book that you would need to order from another library. 5) Please perform a search for a print book with ISBN 0582493498. a) This book is checked out. How would you get a copy of it? b) Pretend that this book is NOT checked out. Please show us the information from this record that you would use to find this item on the shelves. 6) Please navigate to your library account (from within Search It). a) Pretend that you have forgotten how many items you have checked out. Please show us how you would find out how many items you currently have checked out. b) Exit your library account area. 7) Please navigate to Advanced Search. a) Perform any search on this page. 8) Please show us where you would go to find help and/or chat with a librarian? 9) Please perform a search using the keywords “gender and media.” a) Add any source to your My Favorites list. Then open My Favorites and click on the title of the source you just added. b) Return to your list of results. Choose any peer-reviewed article that has the full text available. Click on the link that will access the full text. 10) Please find a database that might be of interest to you (e.g., JSTOR). 11) Please sign out of Search It and close your browser. PRIMO NEW USER INTERFACE | GALBREATH, JOHNSON, AND HVIZDAK 25 https://doi.org/10.6017/ital.v37i2.10191 APPENDIX B: USABILITY RESULTS Note: Search It is the local branding for Primo at Washington State University. Research Question 1: Will the participant be able to find and use the basic search functionality? Associated task(s): 1. Please search for your favorite book or movie. Participant Successful? Commentary 1 Yes Searches for “the truman show” from the beginning. 2 Yes Searches for “pet sematary” from the beginning. 3 Yes Searches for “additive manufacturing” from the beginning. 4 Yes Signs in first, navigates to New Search, searches for “PZT sensor design.” 5 Yes Searches for “the notebook” from the beginning. 6 Yes Searches for “das leben der anderen” from the beginning. 7 Yes Searches for “Legally Blonde” from the beginning. 8 Yes Searches for “Jurassic Park” from the beginning. Research Question 2: Will the participant be able to understand the availability information of the brief results? Associated task(s): 1b. How do you know that this item is or isn’t available for you to read or watch? 4a. From the list of results, find a print book that you would need to order from another library. Participant Successful? Commentary 1 Yes Differentiates between green and orange text; uses the “Check holdings” availability status. Clicks on “Availability and Request Option” heading and then clicks on the resource sharing link. 2 Yes, with difficulty. Says that green “Check holdings” status indicates ability to read the book. Selects book with “Check holdings” status and locates resource sharing link. INFORMATION TECHNOLOGY AND LIBRARIES | JUNE 2018 26 Participant Successful? Commentary 3 Yes, with difficulty Unclear. Initially, goes to a record with Online Access; redoes search, eventually locates resource sharing link. 4 Yes Says the record for the item reads “In place” and the availability indicator = 1. The record for the item reads “Check holdings.” 5 Yes Says that status is indicated by statement “Available at Holland/Terrell Libraries.” The record for the item reads “Check holdings.” 6 Yes Says that status is indicated by statement “Available at Holland/Terrell Libraries” and “Item in place.” Clicks on “Check holdings”; says that orange color denotes fact that we don’t have it. 7 Yes Hovers over “Check holdings” status, and then notes that “Availability” statement reads “did not match any physical resources.” The record for the item reads “Check holdings.” 8 Yes Says that status is indicated by statement “Available at Holland/Terrell Libraries.” Says the record for the item reads “Check holdings.” Research Question 3: Will the participant be able to find and use the Sign In and Sign Out features? Associated task(s): 2. Please sign into Search It. 11. Please sign out of Search It and close your browser. Participant Successful? Commentary 1 Yes Navigates to “Guest” link, signs in. 2 Yes Navigates to ellipsis, signs in. Navigates to “User” link, signs out. 3 Yes Navigates to “Guest” link, signs in. Navigates to “User” link, signs out. 4 Yes N/A—already signed in. Navigates to “User” link, signs out. PRIMO NEW USER INTERFACE | GALBREATH, JOHNSON, AND HVIZDAK 27 https://doi.org/10.6017/ital.v37i2.10191 Participant Successful? Commentary 5 Yes Navigates to “Guest” link, signs in. Navigates to “User” link, signs out. 6 Yes Navigates to “Guest” link, signs in. Navigates to “User” link, signs out. 7 Yes Uses Sign In link from Full Display page. Navigates to “User” link, signs out. 8 Yes Navigates to “Guest” link, signs in. Navigates to “User” link, signs out. Research Question 4: Will the participant be able to understand the behavior of the facets? Associated task(s): 3a. Limit your search results to Articles. 4a. From the list of results, find a print book that you would need to order from another library. 9b. Return to your list of results. Choose any peer-reviewed article that has the full text available. Participant Successful? Commentary 1 Yes Selects Articles facet. N/A—does not use facets (however, participant investigates the Library and Type facets, returns to results lists). 2 Yes Selects Articles facet. N/A—does not use facets. 3 No Uses “Exclude” property to remove everything but Articles. Uses “Exclude” property to remove everything but Print Books. Looks in facet Type for Articles; selects Newspaper Articles instead. 4 Yes, with difficulty Selects Articles facet. Selects Print Books facet. Selects Articles under Type facet, clicks on “Full-text available” status, selects Peer-reviewed Articles facet. 5 No Selects Articles facet. N/A—does not use facets. Screen freezes (technical issue) and participant is forced to redo search. N/A— does not use facets. When further prompted to find only peer- reviewed articles, participant searches pre-filter area and then selects Reviews facet. INFORMATION TECHNOLOGY AND LIBRARIES | JUNE 2018 28 Participant Successful? Commentary 6 Yes Selects Articles facet. Clicks on “Check holdings.” Participant hovers over “Online Access” text and then selects Peer-reviewed facet. 7 Yes Looks in drop-down scope, then moves to Articles facet. N/A— does not use facets. N/A—does not use facets. 8 Yes Hovers over Peer-Reviewed Articles facet, and then selects Articles facet. N/A—does not use facets. Selects Peer-reviewed facet. Research Question 5: Will the participant be able to find and use the Actions Menu? Associated task(s): 3.b.i. For any of the records in your search results list, find the citation for any item and copy it to the clipboard. 3.b.ii. For any of the records in your search results list, email this record to yourself. Participant Successful? Commentary 1 Yes Briefly looks at Citation icon, scrolls to bottom of page and looks at Citations area, returns to Citation icon. Scrolls to bottom of page, returns to Actions area, scrolls with arrow to find Email icon, emails to self. 2 No Initially clicks on citation manager icon (Easybib), then clicks on Citation icon and copies to clipboard. Could not find Email icon (technical issue with Search It). Although further discussion reveals that participant expects to see email function within “Send To” heading. 3 No Opens Full Display page of item, scrolls to bottom of page. Clicks on the Citation icon but doesn’t see what looking for. Finds Email icon and emails to self. 4 No Opens Full Display page of item, clicks on the Citation icon, double-clicks to highlight citation. Could not find Email icon. Searches in ellipsis. Attempts the Keep This Item pin. Navigates to My Account. Searches in facets. 5 Yes, with difficulty Finds Citation icon, but then leaves the area via Citations heading and winds up at Web of Science homepage. Hovers over “cited in this” language. Finds the copy functionality. PRIMO NEW USER INTERFACE | GALBREATH, JOHNSON, AND HVIZDAK 29 https://doi.org/10.6017/ital.v37i2.10191 Participant Successful? Commentary Attempts Sent To heading twice, looks through Actions icons, scrolls to right, finds Email icon. 6 Yes Finds Citation icon, copies to clipboard. Scrolls down page, returns to Actions Menu, scrolls to Email icon, emails record to self. 7 Yes, with difficulty Copies citation from the Brief Result, and then spends some time trying to find “the clipboard.” Navigates to the Email icon. 8 Yes, with difficulty Scrolls to bottom of Full Display page, clicks on Citing This link, clicks on title to record, and then copies first 3 lines of record. Scrolls until finds Email icon, but then moves to Sent To heading, and then back to Email icon, and sends. Research Question 6: Will the participant be able to navigate the Get It and View It areas of the Full Display page? Associated task(s): 5.a. This book is checked out. How would you get a copy of it? 5.b. Please show us the information from this record that you would use to find this item on the shelves. 9.b. Click on the link that will access the full text. Participant Successful? Commentary 1 Yes Clicks on “Check holdings” availability status, clicks on Availability and Request Options heading, clicks on Request Summit Item link. Refers to call number in Alma iframe. Clicks “Full-text available” status, clicks database name. 2 Yes Opens record, locates resource sharing link. Refers to call number; opens stack chart to find call number. Clicks on title, clicks database name. 3 Yes Locates request option. Locates call number in record. Clicks “Full-text available” status, clicks database name. 4 Yes, with difficulty. Clicks on Show Libraries button, then finds request option after searching page. Locates call number in record. Clicks “Full-text available” status but does not click on database name. 5 Yes, with difficulty. Moves to Stack Chart button, then to Show Libraries button, and then to Availability and Request Options heading, clicks on Stack Chart, clicks on Show Libraries, moves into first library listed and INFORMATION TECHNOLOGY AND LIBRARIES | JUNE 2018 30 Participant Successful? Commentary back out, and finally to ILL link. Finds call number on Full Display page. 6 Yes Finds Request Summit option. Identifies call number and Stack Chart as means to find book. Clicks on database name. 7 Yes, with difficulty. Looks at Status statement, scrolls to bottom of page, then Show Libraries button, then Request Summit option. Identifies call number and Stack Chart as means to find book. Attempts to use “Full-text available” link, then clicks on database name. 8 Yes Finds Summit Request option. Identifies call number and Stack Chart as means to find book. Attempts to use “Full-text available” link, then clicks on database name. Research Question 7: Will the participant be able to navigate their My Account area? Associated task(s): 6. Please navigate to your library account (from within Search It). 6a. Pretend that you have forgotten how many items you have checked out. Please show us how you would find out how many items you currently have checked out. 6b. Exit your library account area. Participant Successful? Commentary 1 Yes Navigates to My Account from “User” link. Navigates to Loans tab. Uses back arrow icon. 2 Yes Navigates to My Account from “User” link. Navigates to Loans tab. Uses back arrow icon. 3 Yes Navigates to My Account from Main Menu ellipsis. Navigates to Loans. Uses back arrow icon. 4 Yes Navigates to My Account from Main Menu ellipsis. Navigates to Loans. Uses to back arrow icon. 5 Yes Navigates to My Account from “User” link. Navigates to Loans. Signs out of Search It. 6 Yes Navigates to My Account from “User” link. Navigates to Loans. Uses Search It logo to exit. PRIMO NEW USER INTERFACE | GALBREATH, JOHNSON, AND HVIZDAK 31 https://doi.org/10.6017/ital.v37i2.10191 Participant Successful? Commentary 7 Yes Navigates to My Account from “User” link. Navigates to Loans. Uses New Search button to exit. 8 Yes Navigates to My Account from “User” link. Navigates to Loans. Uses Search It logo to exit. Research Question 8: Will the participant be able to find and use the Help/Chat and My Favorites icons? Associated task(s): 8. Please show us where would you go to find help and/or chat with a librarian? 9.a. Add any source to your My Favorites list. Then, open My Favorites and click on the title of the source you just added. Participant Successful? Commentary 1 Yes, with difficulty Navigates to Help/Chat icon. Navigates to Keep This Item pin, hesitates, navigates to ellipsis, returns to and clicks on pin. Moves to My Favorites via animation. Clicks on title. 2 Yes, with difficulty Initially navigates to Help/Chat icon, but thinks it is the wrong button because chat is not directly available within Search It. Navigates to Keep This Item pin, hesitates, looks around, selects pin. Moves to My Favorites via animation. Clicks on title. 3 Yes, with difficulty Navigates to Help/Chat icon. Navigates to ellipsis, Actions Menu, and Tags section. Finds Keep This Item pin. 4 No Navigates to Help/Chat icon. Navigates to ellipsis, Keep This Item pin, My Account, and facets Quits search. 5 Yes, with difficulty Navigates to Help/Chat icon. Adds Keep This Item pin after investigating 12 other icons. Moves to My Favorites via animation. Clicks on title. 6 Yes Navigates to Help/Chat icon. Adds Keep This Item pin and moves to My Favorites via animation. Clicks on title. 7 Yes Navigates to Help/Chat icon. Checks Actions menu, adds Keep This Item pin and moves to My Favorites via animation Clicks on title. 8 Yes Navigates to Help/Chat icon. Adds Keep This Item pin and moves to My Favorites via animation. Clicks on title. INFORMATION TECHNOLOGY AND LIBRARIES | JUNE 2018 32 Research Question 9: Will the participant be able to find and use the Advanced Search functionality? Associated task(s): 7. Please navigate to Advanced Search. 7a. Perform any search on this page. Participant Successful? Commentary 1 Yes Navigates to Advanced Search. Performs search. 2 Yes Navigates to Advanced Search. Performs search. 3 Yes, with difficulty Navigates to Basic Search drop-down, then to New Search, then to Advanced Search. Has trouble inserting cursor into search box. 4 Yes, with difficulty Navigates to Advanced Search. Builds complex search, then Search It freezes and we have to restart the search tool. 5 Yes Navigates to Advanced Search. Performs search. 6 Yes Navigates to Advanced Search. Performs search. 7 Yes Navigates to Advanced Search. Performs search. 8 Yes Navigates to Advanced Search. Performs search. Research Question #10: Will the participant be able to find and use the Main Menu items? Associated task(s): 10. Please find a database that might be of interest to you (e.g., JSTOR). Participant Successful? Commentary 1 Yes Navigates to “Databases” link of Main Menu. 2 Yes Navigates to “Databases” link of Main Menu. 3 No Types query “stretchable electronics” into search box, but unsure how to find a database in the results lists. 4 No Types query “reinforced concrete” into search box, but unsure how to find a database in the results lists. PRIMO NEW USER INTERFACE | GALBREATH, JOHNSON, AND HVIZDAK 33 https://doi.org/10.6017/ital.v37i2.10191 Participant Successful? Commentary 5 Yes, with difficulty Is confused by term database. Enters “IEEE” in search box. 6 Yes Navigates to “Databases” link of Main Menu. 7 Yes Searches within drop-down scopes, then facets, then moves to “Databases” link of Main Menu. 8 Yes Navigates to “Databases” link of Main Menu. 1 “Frequently Asked Questions,” Ex Libris Knowledge Center, accessed August 28, 2017, https://knowledge.exlibrisgroup.com/Primo/Product_Documentation/050New_Primo_User_Interface /010Frequently_Asked_Questions. 2 Rice Majors, “Comparative User Experiences of Next-Generation Catalogue Interfaces,” Library Trends 61, no. 1 (2012): 186–207, https://doi.org/10.1353/lib.2012.0029. 3 David Comeaux, “Usability Testing of a Web-Scale Discovery System at an Academic Library,” College & Undergraduate Libraries 19, no. 2–4 (2012): 199, https://doi.org/10.1080/10691316.2012.695671. 4 Comeaux, “Usability Testing,” 202. 5 Comeaux, “Usability Testing,” 196–97. 6 Kylie Jarrett, “FindIt@Flinders: User Experiences of the Primo Discovery Search Solution,” Australian Academic & Research Libraries 43, no. 4 (2012): 280, https://doi.org/10.1080/00048623.2012.10722288. 7 Jarrett, “FindIt@Flinders,” 287. 8 Aaron Nichols et al., “Kicking the Tires: A Usability Study of the Primo Discovery Tool,” Journal of Web Librarianship 8, no. 2 (2014): 174, https://doi.org/10.1080/19322909.2014.903133. 9 Nichols, “Kicking the Tires,” 181. 10 Nichols, “Kicking the Tires,” 184. 11 Nichols, “Kicking the Tires,” 184–85. 12 Scott Hanrath and Miloche Kottman, “Use and Usability of a Discovery Tool in an Academic Library,” Journal of Web Librarianship 9, no. 1 (2015): 9, https://doi.org/10.1080/19322909.2014.983259. 13 Greta Kliewer et al., “Using Primo for Undergraduate Research: A Usability Study,” Library Hi Tech 34, no. 4 (2016): 576, https://doi.org/10.1108/lht-05-2016-0052. https://knowledge.exlibrisgroup.com/Primo/Product_Documentation/050New_Primo_User_Interface/010Frequently_Asked_Questions https://knowledge.exlibrisgroup.com/Primo/Product_Documentation/050New_Primo_User_Interface/010Frequently_Asked_Questions https://doi.org/10.1353/lib.2012.0029 https://doi.org/10.1080/10691316.2012.695671 https://doi.org/10.1080/00048623.2012.10722288 https://doi.org/10.1080/19322909.2014.903133 https://doi.org/10.1080/19322909.2014.983259 https://doi.org/10.1108/lht-05-2016-0052 INFORMATION TECHNOLOGY AND LIBRARIES | JUNE 2018 34 14 Kelsey Brett, Ashley Lierman, and Cherie Turner, “Lessons Learned: A Primo Usability Study,” Information Technology & Libraries 35, no. 1 (2016): 21, https://doi.org/10.6017/ital.v35i1.8965. 15 Cece Cai, April Crockett, and Michael Ward, “Our Experience with Primo New UI,” Ex Libris Users of North America Conference 2017, accessed November 4, 2017, http://documents.el- una.org/1467/1/CaiCrockettWard_051017_445pm.pdf. 16 Cai, Crockett, and Ward, “Our Experience with Primo New UI.” 17 J. Michael DeMars, “Discovering our Users: A Multi-Campus Usability Study of Primo” (paper presented, International Federation of Library Association and Institutions World Library and Information Conference 2017, Warsaw, Poland, August 14, 2017), 11, http://library.ifla.org/1810/1/S10-2017- demars-en.pdf. 18 Anne M. Pepitone, “A Tale of Two UIs: Usability Studies of Two Primo User Interfaces” (slideshow presentation, Primo Day 2017: Migrating to the New UI, June 12, 2017), https://www.orbiscascade.org/primo-day-2017-schedule/. 19 “Primo Usability Guidelines and Test Script,” Ex Libris Knowledge Center, accessed October 28, 2017, https://knowledge.exlibrisgroup.com/Primo/Product_Documentation/ New_Primo_User_Interface/Primo_Usability_Guidelines_and_Test_Script. 20 DeMars, “Discovering Our Users,” 9. 21 Majors, “Comparative User Experiences,” 190; Comeaux, "Usability Testing," 198–204. 22 Brett, Lierman, and Turner, “Lessons Learned,” 21. 23 Jarrett, “FindIt@Flinders,” 287; Nichols, “Kicking the Tires,” 184; Brett, Lierman, and Turner, “Lessons Learned,” 20–21. 24 Kliewer et al., “Using Primo for Undergraduate Research,” 571–72; Brett, Lierman, and Turner, “Lessons Learned,” 17. 25 Miri Botzer, “Delivering the Experience that Users Expect: Core Principles for Designing Library Discovery Services,” white paper, Nov 25 2015, 10, http://docplayer.net/10248265-Delivering-the- experience-that-users-expect-core-principles-for-designing-library-discovery-services-miri-botzer- primo-product-manager-ex-libris.html. 26 Majors, “Comparative User Experiences,” 194; Nichols et al., “Kicking the Tires,” 184–85. 27 Cai, Crockett, and Ward, “Our Experience with Primo New UI,” 28–29. 28 Pepitone, “A Tale of Two UIs,” 29. 29 Botzer, “Delivering the Experience,” 4–5; Christine Stohn, “How do Users Search and Discover? Findings from Ex Libris User Research,” Library Technology Guides, May 5 2015, 7–8, https://librarytechnology.org/document/20650. https://doi.org/10.6017/ital.v35i1.8965 http://documents.el-una.org/1467/1/CaiCrockettWard_051017_445pm.pdf http://documents.el-una.org/1467/1/CaiCrockettWard_051017_445pm.pdf http://library.ifla.org/1810/1/S10-2017-demars-en.pdf http://library.ifla.org/1810/1/S10-2017-demars-en.pdf https://www.orbiscascade.org/primo-day-2017-schedule/ https://knowledge.exlibrisgroup.com/Primo/Product_Documentation/%20New_Primo_User_Interface/Primo_Usability_Guidelines_and_Test_Script https://knowledge.exlibrisgroup.com/Primo/Product_Documentation/%20New_Primo_User_Interface/Primo_Usability_Guidelines_and_Test_Script http://docplayer.net/10248265-Delivering-the-experience-that-users-expect-core-principles-for-designing-library-discovery-services-miri-botzer-primo-product-manager-ex-libris.html http://docplayer.net/10248265-Delivering-the-experience-that-users-expect-core-principles-for-designing-library-discovery-services-miri-botzer-primo-product-manager-ex-libris.html http://docplayer.net/10248265-Delivering-the-experience-that-users-expect-core-principles-for-designing-library-discovery-services-miri-botzer-primo-product-manager-ex-libris.html https://librarytechnology.org/document/20650 PRIMO NEW USER INTERFACE | GALBREATH, JOHNSON, AND HVIZDAK 35 https://doi.org/10.6017/ital.v37i2.10191 30 “Primo August 2017 Highlights,” Ex Libris Knowledge Center, accessed November 2, 2017, https://knowledge.exlibrisgroup.com/Primo/Product_Documentation/Highlights/ 027Primo_August_2017_Highlights. 31 Jakob Nielsen, “How Many Test Users in a Usability Study?,” Nielsen Norman Group, Jun 4, 2012, https://www.nngroup.com/articles/how-many-test-users/. 32 Majors, “Comparative User Experiences,” 190; Comeaux, “Usability Testing,” 200–204. 33 Brett, Lierman, and Turner, “Lessons Learned,” 21. https://knowledge.exlibrisgroup.com/Primo/Product_Documentation/Highlights/%20027Primo_August_2017_Highlights https://knowledge.exlibrisgroup.com/Primo/Product_Documentation/Highlights/%20027Primo_August_2017_Highlights https://www.nngroup.com/articles/author/jakob-nielsen/ https://www.nngroup.com/articles/how-many-test-users/ Abstract Introduction Research Questions Literature Review Method Results Task set(s) related to research question 1: Will the participant be able to find and use the Basic Search functionality? Task set(s) related to research question 2: Will the participant be able to understand the availability information of the brief results? Task set(s) related to research question 3: Will the participant be able to find and use the Sign In and Sign Out features? Task set(s) related to research question 4: Will the participant be able to understand the behavior of the facets? Task set(s) related to research question 5: Will the participant be able to find and use the Actions Menu? Task set(s) related to research question 6: Will the participant be able to navigate the Get It and View It areas of the Full Display page? Task set(s) related to research question 7: Will the participant be able to navigate the My Account area? Task set(s) related to research question 8: Will the participant be able to find and use the Help/Chat and My Favorites icons? Task set(s) related to research question 9: Will the participant be able to find and use the Advanced Search functionality? Task set(s) related to research question 10: Will the participant be able to find and use the Main Menu items? Discussion Decisions made in response to usability results Declined to Change Implemented a Change Labels Link Removal Prioritized the Emphasis of Certain Functionalities Contrast and Separation Conclusion Appendix A: Usability Tasks Appendix B: Usability Results Research Question 1: Associated task(s): Research Question 2: Associated task(s): Research Question 3: Associated task(s): Research Question 4: Associated task(s): Research Question 5: Associated task(s): Research Question 6: Associated task(s): Research Question 7: Associated task(s): Research Question 8: Associated task(s): Research Question 9: Associated task(s): Research Question #10: Associated task(s):