Submitted by Shaun Adamson
on November 1, 2002
Summary & Recommendations
Usability testing for the Stewart Library homepage included
several rounds of testing and took place during October 2002. Quantitative
data included mostly demographic data for users who completed the Web-based
survey (MS Word). Qualitative data included opinions about the
Web site from the survey and the student focus group. Functionality
of the site still remains to be tested via task testing.
As a whole, users view the new site as an improvement
over the old. As a result of current testing and as per conversation
with the WAG chair, the Usability Committee will make the following suggestions
and recommendations:
- A more uniform and standardized process will be developed for more
time-intensive testing activities (ie. user testing). More authentic
and robust questions will be developed.
- Standard processes and calendars will be developed in concert with
WAG to ensure that usability testing is used to improve the Web site
in accordance with WAG policies. More stable timelines will be
developed for various usability tasks and WAG/Usability roles and relationships
will be addressed in detail.
- No changes will be made to the new site at this time. Web survey testing
will be continued in Spring semester 2003, with the following considerations:
- Some questions will be rewritten so that quantitative data (ie.
demographic data) may be more easily analyzed using SPSS statistical
software.
- Additional questions may be added to address system concerns.
- Pop-up menus and 'clutter factor' will be priority items to consider,
as there were many mentions of these items in the qualitative data,
but no clear positive or negative direction to the comments.
- If Spring 2003 usability testing data is consistent
with current data, the following recommendations should be considered:
- Change the current color scheme to reflect WSU school colors (purple).
- Change the image on the homepage from belltower to ??
- Get rid of scrolling text.
- Continue to revise terminology as needed to increase site intuitiveness.
Detailed data from the various usability tests are outlined
below:
Quantitative Data from Survey
A total of 84 individuals completed the Web-based
survey (MS Word). This survey was posted to the new library
site during October 2002 and asked for demographic data and user opinions
of the new site. Survey questions were reworded during the analysis
phase so that data could be run through SPSS. The survey sample
was not a representative sample of our users.
While the demographic data was not a representative
sample and did not inform the committee about Web page improvements, it
did give us some interesting statistics on some of our users. Out
of the 84 participants who took the survey, there were 9 faculty/staff,
1 community member, 10 did not respond to the affiliation question, and
the rest were students- mostly sophomores and juniors. Respondents
came from a variety of departments on campus, and approximately 91% of
them use the Web daily or a few times per week. When
asked if they liked the way the new page looked, the overwhelming majority
said yes.
See graphical data for
a more detailed representation. For a full report, including exact
frequencies, please see Shaun Adamson, or click HERE
for the data (.SPO).
Qualitative Data from Survey
& Focus Group
The focus group data (Kathy P.'s 2201 class)
concentrated on student opinions and first impressions of the new site.
Students were asked what their first impressions of the page were, whether
they liked the way the new page looked, if there was anything about the
new page that they didn't like, a series of task-related questions to get
an informal picture of the site functionality, and queried on the terminology.
The class stated that they were familiar with the old page, although they
hadn't used all functions at this time.
All but two members of the class thought the new site was an improvement
over the old. There were three complaints about the new site, and these
corresponded with the data from the Web survey:
- two members of the class thought the new site was more complicated/cluttered
- a few didn't like the scrolling text
- one student could not find encyclopedias
Some of the terminology on the page was addressed, including 'Subject
Resources', 'Articles & Databases', 'Services for Students', and 'How
do I'. Students seemed to have a fairly good idea of what might be included
under each link. Detailed focus group responses can be found on the focus
group notes page.
The qualitative survey data addressed student opinions of the
new Web site, and included questions 7, 9, and 10 of the survey:
- 7. What is your first impression of the library's
homepage?
- 9. Is there anything in particular about the homepage
that you don't like?
- 10. Are there changes you can suggest to make the homepage
easier to use?
This data was divided into positive, negative, and mixed comments for question
7, and compiled for questions 9 and 10. Five prominent themes were
noted from this data. These included comments related to:
- the belltower graphic
- the color scheme of the new homepage
- the 'clutter' factor
- the pop-up menus,
- the scrolling text
As a whole, students did not like the belltower graphic (many requested
other images) or scrolling text. Most of the color comments asked for
school colors (purple) instead of the blue color scheme. There were a
mix of comments on the pop-up menus and 'clutter' For detailed information,
see the qualitative
survey data page.
