Submitted by Shaun Adamson on November 1, 2002

---

Summary & Recommendations

Usability testing for the Stewart Library homepage included several rounds of testing and took place during October 2002.  Quantitative data included mostly demographic data for users who completed the Web-based survey (MS Word).  Qualitative data included opinions about the Web site from the survey and the student focus group.  Functionality of the site still remains to be tested via task testing. 

As a whole, users view the new site as an improvement over the old.  As a result of current testing and as per conversation with the WAG chair, the Usability Committee will make the following suggestions and recommendations:

• A more uniform and standardized process will be developed for more time-intensive testing activities (ie. user testing).  More authentic and robust questions will be developed.
• Standard processes and calendars will be developed in concert with WAG to ensure that usability testing is used to improve the Web site in accordance with WAG policies.  More stable timelines will be developed for various usability tasks and WAG/Usability roles and relationships will be addressed in detail.
• No changes will be made to the new site at this time. Web survey testing will be continued in Spring semester 2003, with the following considerations:
  • Some questions will be rewritten so that quantitative data (ie. demographic data) may be more easily analyzed using SPSS statistical software.  
  • Additional questions may be added to address system concerns.  
  • Pop-up menus and 'clutter factor' will be priority items to consider, as there were many mentions of these items in the qualitative data, but no clear positive or negative direction to the comments.
• If Spring 2003 usability testing data is consistent with current data, the following recommendations should be considered:
  • Change the current color scheme to reflect WSU school colors (purple).
  • Change the image on the homepage from belltower to ??
  • Get rid of scrolling text.
  • Continue to revise terminology as needed to increase site intuitiveness. 

Detailed data from the various usability tests are outlined below:

 

Quantitative Data from Survey

A total of 84 individuals completed the Web-based survey (MS Word).   This survey was posted to the new library site during October 2002 and asked for demographic data and user opinions of the new site.  Survey questions were reworded during the analysis phase so that data could be run through SPSS.   The survey sample was not a representative sample of our users.

While the demographic data was not a representative sample and did not inform the committee about Web page improvements, it did give us some interesting statistics on some of our users.  Out of the 84 participants who took the survey, there were 9 faculty/staff, 1 community member, 10 did not respond to the affiliation question, and the rest were students- mostly sophomores and juniors.   Respondents came from a variety of departments on campus, and approximately 91% of them use the Web daily or a few times per week.    When asked if they liked the way the new page looked, the overwhelming majority said yes.  

See graphical data for a more detailed representation.  For a full report, including exact frequencies, please see Shaun Adamson, or click HERE for the data (.SPO).

 

Qualitative Data from Survey & Focus Group

The focus group data (Kathy P.'s 2201 class) concentrated on student opinions and first impressions of the new site. Students were asked what their first impressions of the page were, whether they liked the way the new page looked, if there was anything about the new page that they didn't like, a series of task-related questions to get an informal picture of the site functionality, and queried on the terminology. The class stated that they were familiar with the old page, although they hadn't used all functions at this time.

All but two members of the class thought the new site was an improvement over the old. There were three complaints about the new site, and these corresponded with the data from the Web survey: 

• two members of the class thought the new site was more complicated/cluttered
• a few didn't like the scrolling text
• one student could not find encyclopedias

Some of the terminology on the page was addressed, including 'Subject Resources', 'Articles & Databases', 'Services for Students', and 'How do I'. Students seemed to have a fairly good idea of what might be included under each link. Detailed focus group responses can be found on the focus group notes page.

The qualitative survey data addressed student opinions of the new Web site, and included questions 7, 9, and 10 of the survey:

• 7.    What is your first impression of the library's homepage?
• 9.    Is there anything in particular about the homepage that you don't like?
• 10.  Are there changes you can suggest to make the homepage easier to use?

This data was divided into positive, negative, and mixed comments for question 7, and compiled for questions 9 and 10.  Five prominent themes were noted from this data. These included comments related to:

• the belltower graphic
• the color scheme of the new homepage
• the 'clutter' factor
• the pop-up menus,
• the scrolling text

As a whole, students did not like the belltower graphic (many requested other images) or scrolling text. Most of the color comments asked for school colors (purple) instead of the blue color scheme. There were a mix of comments on the pop-up menus and 'clutter' For detailed information, see the qualitative survey data page.