Phydeaux
Usability Study
Usability Testing & Testing • UX Research
A semester-long project during graduate school evaluating a local pet store’s e-commerce website by using usability testing and evaluation research methods.
Overview
This was a semester-long, group-based project for my Usability Testing & Evaluation class at UNC.
My team evaluated the usability of an e-commerce pet retail store, Phydeaux, by designing, observing, and moderating a summative usability test. Our analysis helped inform our proposed design recommendations for future iterations of the site.
Course:
Usability Testing & Evaluation
(INLS 719, UNC-Chapel Hill)
Duration:
4 months
(Sep. 2018 - Dec. 2018)
Team:
Dottie Blyth
Nura Hill
Rachel~Anne Spencer
Jeffrey Robbins
Phydeaux: A Specialty Retail Pet Store
As pet owners and animal lovers, my team chose to evaluate the e-commerce page of a locally-owned pet store, Phydeaux.
We were eager to analyze this e-commerce platform because while the pet store has received both national (e.g. Inc. 5000) and local recognition (e.g. Indy Weekly), we were curious to see whether their success would translate online.
Ultimately, we felt that this was an important interface to evaluate since it can contribute to the future sustainability of an independent, locally owned store.
Screenshot of the interface we evaluated
Evaluation Goals
Usefulness
Are users able to understand that Phetch is the e-commerce site for Phydeaux?
Effectiveness
Are users able to effectively utilize the e-commerce capability of Phetch?
Research Design & Environment
Testing environment (click image to expand)
We applied research design principles we learned in class and administered a within-subjects usability study, which included:
A think aloud protocol
Pre- and post-test questionnaire
4 pre-defined tasks (all available in the full report linked at the bottom of this page)
Each member on the team had the opportunity to moderate a session (~30-40 minutes long), observe and take notes on a session, and provide tech support (i.e., setting up Camtasia software).
Usability Tasks
After setting up the test design and environment, we then created four usability tasks, which directly correlated to our evaluation goals. We tried to come up with tasks that could point us to new usability issues and where more user feedback would be helpful. In other words, we tried to stay away from already-known usability issues on the site.
The tasks for this test were:
1. Determine whether Phydeaux provides a way to purchase items online.
2. Find an item on Phetch without search.
3. Delete an item from the Phetch shopping cart.
4. Locate an '“out-of-stock” item in an adjacent store.
Data Collection
During and after each session, we also gathered performance metrics and self-reported metrics, which included:
Performance Metrics
Time on task
Number of pages viewed
Success rate
Self-Reported Metrics
Ease of use
User expectations
System intuitiveness
User confidence
User satisfaction
Findings from the Study
Once we were finished conducting our four usability tests, we then analyzed the data we gathered, with a focus on quantitative data. We identified three major findings from this evaluation study. An in-depth analysis can be found in our report linked below.
1) Participants struggled to understand and navigate to Phydeaux’s e-commerce page
50% of participants were able to determine that Phydeaux provides a way to purchase items online
100% of participants were reluctant to click the navigation button that would take them to Phetch
2) Not all user expectations were met while on the site
Most users expected a delete button when trying to delete an item from their shopping cart
3) There are fundamental navigational and information architecture issues
Participants kept clicking “Small Animal Foods” rather than “Small Animal Treats”
Locating an “out-of-stock” item was the most time-intensive task (average time = 4:30, whereas every other task was completed in <2 minutes).
System Usability Scale
We also applied another metric learned in class — the System Usability Scale (SUS)— to help communicate our findings via a universal scale. We calculated the mean responses and calculated an average SUS score of 48.875. To put this into comparison, any score of 68 is considered "average.”
Design Recommendations
After our analysis, we provided several UX recommendations in an attempt to improve the usability of Phydeaux’s e-commerce website. We ranked each recommendation with a level of severity (critical vs. minor) based on our observations and expertise help our stakeholders could prioritize them.
(Click image to expand)
Reflection
This project proved to be one of the more intensive projects I’ve worked on and I was satisfied with how my team and I were able to execute our study. This project also allowed me to focus on the details of the test design (e.g., how the moderator would greet the participant, where the moderator would sit, etc.) which I believe have strengthened my skills as a researcher.
— With final reports, sometimes less is more! —
Although a lot of hard work went into this project, one of the biggest challenges my team and I faced was surprisingly… A page limit.
We had to constrict the entirety of our report (excluding the Appendix) to 11 pages, which isn’t much when you want to include graphs, charts, and pictures! In the end though, I think deciding between essential vs. non-essential information to include in our report was a key takeaway from this project.