Usability Testing Protocol and Insights
Rating System
Purpose and Goal
the product was not fully functional and so we ask our users to use this rating system to rate any service in order to test their desire to use the system
To see if users are able to give honest and meaningful ratings and reviews
Protocol
Hey, I’m ____. Thank you for agreeing to participate in the usability testing for our project. Our team is working to redesign the rating and evaluation system for freelancing platforms Fiverr, Upwork or 99designs for a project in our UX Design class. Our design is consistent with Upwork’s interface .
Today, we will be showing you some screens and, giving you a few tasks to try and accomplish. We are not testing you, we are testing the tool, so don’t be stressed and feel like you are doing anything wrong if you can’t do the tasks. As you navigate through the process, please share all of your thoughts and intentions. Try to imagine that you are using this tool for the first time. Interact with it, and think aloud while you do, especially if you find anything confusing. Inform us about everything you see, everything you touch on the screen. For example, “I am currently looking at the search page, I click on the search button and this page appears”.
Since this is a prototype, certain features are not entirely functional. Whenever you encounter these areas, simply act like you are interacting with it and move on to the next thing you would do if this prototype was entirely functional.
First, we will begin with a few demographics questions to better understand your background.
Questions
Scenario 1
Imagine that you are looking for a freelancer to design a landing page for your website on Upwork. You go to the search page to find the freelancer who is a perfect match for you.
Tasks
From this screen (search page), how will you find the freelancer for your job?
How will you view the freelancer’s page?
How will sort the search results based on the communication score?
Tell us if the information displayed on the screen helped you make a quick, better informed decision.
Scenario 2
Imagine that you have worked with a freelancer, the job is done and now you have to rate him on Upwork. The freelancer did a great job with his design but was arrogant and did not provide timely updates for the design. He produced the final deliverables on time but missed some deadlines in between.
Tasks
Rate the freelancer based on your experience.
Interact with the next screens that appears. How will you provide additional feedback here?
A/B Testing
We provided three users with both current and new rating system and asked them the following questions-
Which one took less time to rate?
Which one took less effort to rate?
Which one was more difficult to interpret?
Which one would you prefer?
Takeaways and Results
Search
The overall score was small and not near the three scores for skills, time management, and communication.
Liked being able to filter on specific criteria they were most interested in
Evaluation
Saw there were overall scores elsewhere, but they never saw what overall score they gave the person they were evaluating.
Was a little confused that they clicked submit after finishing the sliders and then had more information added. At first they thought it was a new page with three new sliders.
Generally preferred the three specific categories over the star ratings.
All the users we tested, preferred the new rating (3 categories- 3 levels) over the old rating system (scoring on a scale of 10 or 5). They were quite happy with the three categories of rating as they didn’t have to evaluate the entire experience to rate on an overall basis.