Making LibGuides Work: Student Interviews and Usability Tests

ABSTRACT This article discusses student use of and feedback on LibGuides in an academic library. The authors conducted interviews and usability tests with on-campus and distance students to determine their preferences and expectations, as well as to identify specific pain points. Based on the findings, the authors changed LibGuides best practices at their institution and overhauled the LibGuides index page.


Introduction
The University of Wyoming has an enrollment of approximately 12,500 students. The only public four-year institution in a sparsely-populated state, UW maintains a high undergraduate population and has offered distance education services for over a century (Johnson, 1987).
Serving our many distance students and faculty has long spurred the UW Libraries to take advantage of the Internet to facilitate access to research assistance, including online guides. UW Libraries developed a homegrown content management system exclusively for research guides in 2004 before adopting the open source Library a la Carte (Nichols, 2009). In 2010, the Libraries began utilizing Springshare's LibGuides (https://www.springshare.com/libguides/).
Librarians create multiple types of guides targeted to researchers: course guides help students complete specific assignments; subject guides pull together resources by discipline and often include strategies for using them; and other guides answer questions about particular library services or policies. Responsibility for design practices and currency of guides has moved among various individuals or task forces. The UW Libraries have a cross-departmental Web team that frequently does usability testing of the library website, but we had never extended that testing to research guides.
The authors were primarily interested in what design elements and amount of content in our guides would be most useful for students. Other researchers have found that students prefer guides with simple, clean designs (Hintz et al., 2010). The amount of information that students prefer on a guide varies widely (Arnold, Csir, Sias, & Zhang, 2004;Staley, 2007). Reeb and Gibbons (2004) found that students struggle with the concept of academic disciplines that form the basis for many subject guides and prefer guides targeted to courses. On the other hand, Murphy and Black (2013) found that high use guides were more often department or college-level guides rather than course guides.
An upgrade to LibGuides was another impetus to learn about our students' preferences. The new version offered the ability to create side menus for navigation as an alternative to tabs across the top of the page. Many research guides are organized using tabbed navigation, which presents usability issues if there are more tabs than fit on a single row (Ouellette, 2011). Now that LibGuides feature responsive design, the number of tabs in a single row is less clear-cut. Some recent studies have found that users struggle with tabs in general (Ouellette, 2011;Pittsley & Memmott, 2012;Sonsteby & DeJonghe, 2013), although Web usability expert Steve Krug (2013) argued that tabs-done correctly-are self-evident forms of navigation.
Because the authors were interested in learning about design preferences as well as what elements might inhibit using content in LibGuides effectively and efficiently, we chose to do usability testing and interviews with target audiences for guides: undergraduate and graduate students. Usability testing has become common in academic libraries (Chen, Germain, & Yang, 2009;Chen, Germain, & Rorissa, 2011). Plenty of guidance for website usability and testing is available (Krug, 2010;Nielsen, 2012;Rubin & Chisnell, 2008;US Department of Health & Human Services, 2006), including guidance specifically for libraries (Blakiston, 2014;Winters & Norlin, 2001).

Methodology
We recruited 11 students for usability testing and 8 students for interviews. Interview participants were enrolled in distance or face-to-face programs in education or nursing. Usability testing participants could be enrolled in distance or face-to-face programs in any college. Since more than a quarter of University of Wyoming students take distance classes (US Department of Education, 2014), we wanted to be sure these students were included in our assessment efforts. Two students participated in both usability testing and interviews. Participants were offered Amazon gift cards as an incentive.
We developed a script for usability testing with open-ended tasks like "You're in an accounting class and need to find a SWOT analysis" (Appendix A). We wanted the tasks to be as authentic as possible so we could determine whether students who were aware guides existed and knew what they wanted to find could use guides to get to the desired materials. In-person sessions were recorded with Camtasia, while distance sessions were recorded with Join.Me (http://www.join.me).
We also developed a script for interviews with students in education and nursing programs (Appendix B). These were more flexible-participants were asked about their experience with guides and preferences before comparing versions of a guide using tabbed and side navigation. The interviews gave more in-depth information about guide design. Again, in-person sessions were recorded with Camtasia while distance sessions were recorded with Join.Me.
After all usability tests and interviews were transcribed, one investigator initially coded the transcriptions using an open coding approach. When this was completed, the research team reviewed the transcriptions together and made changes to the codes as necessary.

Results
Out of 19 participants in either usability studies or interviews, 8 indicated they had used research guides before; 11 were not familiar with them. Two of the 8 interviewees, 5 of 11 usability participants, and one student who participated in both activities indicated they had used the guides before.
Student performance in the usability tests was mixed, as shown in Figure 1. All participants were able to find instructions for locating a Strengths Weaknesses Opportunities and Threats (SWOT) analysis using course guides, and most were easily able to find a guide listing citation managers. Students had the most difficulty finding instructions on how to request a book for off-campus delivery.
The research team analyzed comments to identify problems that interfered with efficient and effective use of LibGuides for completing tasks. We used the same open coding for the interviews and found comments by students fell into six broad categories:

Content Design
Organizational failure Page length Guide rejection Praise Half of interviewees and 3 of the 11 usability participants made comments indicating the content or amount of content was appropriate-either for themselves or another population. For example, a graduate student participant said the guide would be useful for undergraduates. Half of the interviewees and two of the usability participants said the content was different from what they had anticipated based on tab labels. Two interviewees said more descriptive information about links to resources would be useful, while two others noted that the amount of information about links was appropriate. Two interviewees and one usability participant specifically mentioned that they liked the inclusion of contact information for librarians.
We coded comments mentioning visual aspects or readability in the design category. We also included as design issues students' difficulty or confusion in identifying links.
Five interviewees identified visual aspects of guides that appealed to them: boxes, images, symmetry, or a "clean" appearance. Two of these five also pointed to aspects that could be improved, such as moving images to the center of the page instead of a side column or editing colors to make a box more visible. A couple of participants said that the guides reminded them of Facebook or news sites, or that content in a narrow side column looked like an ad.
Six interviewees commented positively about aspects of guides related to readability, such as text size, bolded text, limited text, or content being easy to find because of its placement on the page. Four interviewees made suggestions for improving readability: replacing paragraphs with bullets or lists, reducing the amount of text, or changing the size or color of headings.
Issues with readability came up in five of the usability tests where participants either suggested improvements or overlooked links or content that seemed obvious to us. Four students tried to interact with a static image of a search interface. Three had difficulties identifying links in the text. Four of the usability participants commented on positive aspects related to readability.
We coded actions as organizational failure when terminology or the organization of guides seemed to hinder students from completing usability tasks.
One usability task was to find information about health concerns of a specific immigrant group for a nursing class. The "correct" answer was to use a course guide on cultural diversity in nursing, but students had difficulties adjusting their thinking from a narrow concept to a broader category. Eliminating all errors like this would be difficult, but usability testing confirmed widespread problems with a category for service-type guides developed by librarians. The most direct path using guides to find instructions for requesting books from off-campus was through a category called "User Groups." Participants who completed this task found the guide through alternative paths; others said they would simply contact the library for help. Students were more successful seeing the prominent link to Citation & Style Guides or finding the target citation guide under our category, Research Tools & More.
We saw difficulties with jargon in nine usability tests and one interview. Five students were unfamiliar with the terminology "guides," or assumed we were asking about databases. Four students hesitated at the terminology "off-campus access" as they tried to find guides for distance users. One interviewee commented that a tab labeled "Articles" should be labeled "Search Databases" because the page did not offer a list of articles.
Another aspect of organizational failure involved the built-in search function in LibGuides. Several students used the search box to attempt to complete usability tasks, but were frustrated to get a list of guides for search results rather than answers to the prompts. Students assumed the LibGuides search box on an individual guide would search only that guide (which was not possible at the time of the study); instead results appeared from pages across LibGuides. The LibGuides search algorithm delivered results that were too broad, with less relevant results listed before pages that may have helped participants complete tasks.
Participants in interviews were asked about the length of pages. Six of the 8 mentioned scrolling-either that they liked that they did not have to scroll or disliked that they did have to scroll down to see information on a page. One said she did not mind scrolling. Three of the interviewees and two usability participants noted too much text on a page, and one suggested adding popups to minimize text.
One design question we hoped to answer through this study was whether students preferred tabs across the top of the page in LibGuides or guides with a navigation menu on the side. We included this question in our interviews. Four of the eight interviewees preferred tabs; two preferred side navigation; two liked tabs on one guide and side navigation on another. We did not ask usability participants about their preference more broadly, although one mentioned preferring side navigation on websites in general. We set up our test guides before LibGuides allowed multiple columns within side navigation guides, so our side navigation guides required more scrolling. Based on our evidence it is impossible to determine whether students genuinely preferred tabbed navigation or whether their primary concern is limited scrolling, with side versus tabbed navigation making less of a difference for them. Figure 2 shows tabs on LibGuides at the University of Wyoming appear more button-like, which makes them easier to see than tabs on other instances of LibGuides.
Eight of our 11 usability participants either used an alternative path or said they would normally choose an alternative strategy to answer the prompts we gave them. To find information on a topic, they would use the library catalog, a database, or the discovery tool search box on the library home page instead of a guide. A couple of students already knew how to request books off-campus; the others said they would seek out help from a librarian regarding this service rather than using a guide. We coded these comments as "guide rejection." One interviewee said he would not use guides again, and a second said she had seen guides before and found them frustrating.
Subject categories, terminology, the LibGuides search algorithm, overlooking links or tabs, and preferring an alternative to guides were the more prominent obstacles that prohibited students from efficiently completing usability tasks with our LibGuides (see Figure 3).
Participants also gave what we coded as praise for guides. Seven of our eight interviewees complimented some aspect of guides, including five who noted  content in the guides. Three liked the inclusion of video tutorials. Five usability participants praised guides or their content; one said that the guides were easy to use. Students also commented positively about links to chat and contact information for librarians. Six of the interviewees and three of the usability participants spontaneously praised librarians and library services.

Discussion
While our study was intended to help us improve usability of our own instance of LibGuides, many of our findings align with recommendations from other researchers and may be of interest to a broad audience of academic librarians who create online research guides.
Students expressed preferences for shorter pages with less scrolling; they also noted that bullets, reduced amounts of text, and bolded or varying text size made pages easier to read. While students stated that most guides they viewed in the study had an appropriate amount of information, a few ran across text-heavy pages that they said were overwhelming. After completing this study, we formed a LibGuides group at our library to revise best practices, improve consistency among guides, and take responsibility for guides which were no longer regularly updated due to personnel changes.
We asked students whether they preferred navigation menus on the side of the page or navigational tabs across the top of the page. More than half stated a preference for tabs on some or all pages, but student responses made it unclear whether they preferred the tabs themselves or the shorter page lengths associated with tabbed navigation in our sample pages. In usability testing, some students overlooked tabs or took time before finding the tab to navigate to another page in the guide. We plan to do additional interviews and usability testing to revisit our question about tabs, removing page length as a variable.
In addition to students overlooking tabs, the LibGuides search box functionality proved an impediment to completing usability tasks. Some students expected the search box to deliver articles rather than links to guides, echoing findings at the University of Washington (Tawatao et al., 2010). Because of the widespread confusion about what the search box actually searched and the overly broad results, we initially removed the search box from all individual guides so that it only appeared on the LibGuides index page. LibGuides has since improved the ability to search within a guide and the search box has been restored to individual guides.
Subject categories were another barrier to completing usability tasks. Some students had to consider for a moment before finding an accounting course guide under the broader category of business, but a usability task that asked participants to find a guide with instructions for requesting books from off-campus proved so frustrating for students that many gave up.
We revisited our subject categories, paying extra attention to three subjects that were not directly connected to an academic discipline: Library Information, Research Tools & More, and User Groups. Since this happened at the same time as a website redesign, the information in User Groups is leaving LibGuides for a more prominent spot on the main library website. This new placement should make it easier for users to find what they need. The guides under Library Information and Research Tools & More mostly contain information about branch libraries and specific services that will be linked to from the new services for faculty/students/the public pages. We recommend that other librarians also consider revisiting their online guide categories, especially those that serve as catch-alls. Are they intuitive to users and new student workers?
We identified jargon as another barrier during usability tasks and interviews. The most frequent problem with jargon had to do with the guides themselves; students equated the term "research guide" with library databases. This led to our most dramatic change: redesigning the home page for Lib-Guides. Figure 4 shows the LibGuides Subject view that we previously used as the landing page for guides. Given that our participants had trouble finding what they needed to complete tasks using this homepage, we wanted to try a different approach. After looking at the Oregon State (http://diy.library.oregon state.edu/) and Portland State (http://library.pdx.edu/diy/) installations of Library DIY and Northwestern University's Where Do I Start (http://libguides.northwest ern.edu/wheredoistart), we developed a similar resource in LibGuides (see Figure 5). In Summer 2016, a link to this guide was added to a universal navigation bar of the library website. While these changes were all made in response to the issues we discovered while conducting interviews and usability tests, we hope to pursue another round of testing to learn whether changes have really addressed these problems.

Conclusion
The authors set out to learn what design elements were useful or problematic for students viewing LibGuides or using them to complete research-related tasks. We recruited 19 graduate and undergraduate, campus and distance students for interviews and usability studies.
Student feedback and performance on usability tasks provided us with helpful information that we are using to revise local best practices for creating LibGuides. We established a new cross-departmental LibGuides team that meets at least once a month to discuss issues related to individual guides or overall navigation and appearance of guides. The biggest change we made as a result of this study was a revamp of the LibGuides home page to more of a task-focused menu.
Several students in the study mentioned positive experiences with individual librarians and library services. Over half of our participants had not used our guides before, but both experienced and inexperienced users expressed that the guides seemed useful and were pleased to learn more about them. We believe that guides have the potential to supplement interaction with a librarian and that they are worth the time to develop and improve through further usability testing.