Freelance Life: The Fault in Our Stars
A redesign of Fiverr’s rating system to provide better feedback to freelancers
Problem Background
MEET ALEX
Freelancer
Alex works as a freelancer. She tries to do her best and wants to increase her clientele.
But, in order to do that, she needs to improve and be the best version of herself and she can’t do
that because her clients don’t leave her informative reviews and just give a simple star rating.
She’s also pretty confused because her clients either give her either a very low rating (1 or 2 stars) or 5 stars.There’s never an in-between. So, is she good or not? Is the problem with the clients? How will her future clients know? How will the algorithm rate her compared to others based on her rating?
Let’s address Alex’s concerns and help other freelancers like her benefit from an improved reviews and ratings system.
Jump to Solution
Timeline
3 weeks (Sept 2022)
Team
UX Designer: Maham Ghazanfar, Maira Mansoor
UX Writer and Documentation: Umar Khan
UX Methods Used
Research
Literature Survey
User Interviews
Mental Model Diagramming
Design
Ideation
Sketching
Low-fi Prototyping
Concept & A/B Testing
My Role
I conducted desk research and mapped the research protocol, conducting pilot and semi-structured user interviews. I also work on creating the sketches, prototyping and testing.
Project Objective
The primary goal of this project was to change how freelancers are evaluated, in order to ensure that clients have a better understanding of the user journey of working with the freelancer in question, freelancers receive fair ratings and reviews that are unskewed, and freelancers are able to use these ratings to improve and illustrate this improvement over time.
Goal Achievement Pipeline
Understanding the Problem
Preliminary Understanding
As none of the team members had experience with freelancing platforms, we started building our knowledge in this domain and understanding how freelancers use these platforms by conducting 2 unstructured online interviews with freelancers who had been using Fiverr and Upwork for at least 1 year and had completed several projects, on how they use the platform, get clients and the role of reviews and ratings.
FINDING 01
Fake Reviews
FINDING 02
Getting Clients
Offering a unique service assists in getting more clients
FINDING 03
Bad Ratings
Even a single bad rating (1 or 2 stars) can significantly affect the rating of the freelancer
Since these interviews were open-ended, some of the findings, although interesting takeaways, were not relevant to our problem except the bad ratings which showed the impact of the rating issue on freelancers.
Deeper Dive
To develop a deeper understanding of this problem space, and see how the rating system works in other related applications and the current problems faced by freelancers, we conducted secondary research, reviewed freelancing platforms’ (Fiverr and Upwork) community forums, and conducted primary research in the form of in-depth semi-structured interviews with 4 freelancers.
Identification of Issues
Affinity Mapping
Analyzing the data collected from the interviews and community forums, affinity mapping was done to identify the major themes
Mental Model Diagram
Since a mental model diagram provides a rigorous way to to align people’s behaviors and emotions while attempting to complete a task, based on the findings of the affinity mapping, we created a mental model diagram. This allowed us to develop a bottom-up approach to identify the issues pertaining to different themes and clearly list possible solutions to address each issue.
Developing the Problem Statement
Based on the research and analysis conducted, several major problem areas were identified that we decided to tackle in regard to the reviewing system on freelancing platform
1
Lack of trust: there is no progress visibility, no complete communication between clients and freelancers, and freelancers feel the platform prefers clients.
2
Reviews are subjective based on the client’s perception of what makes a good review and a client may not leave reviews since it is not mandatory.
3
Freelancers may improve but the bad ratings they once got, even way earlier impact their future performance
4
Algorithms are ambiguous and obscure in terms of how they rank freelancers’ performance
Note: we did not consider reviewing clients by freelancers and fake reviews since due to the time constraints of the projects, we could not cater to these perspectives. Additionally, The interviews showed that the freelancers were not concerned with other freelancers using fake reviews since they believe that they are only effective in the beginning but not later on.
Areas of Solution Improvement
We focused our solution ideas on three main areas: enabling freelancer improvement over time, improving the communication between freelancers and clients, and trying to revamp the review and rating system.
Sketches
After a session of brainstorming, we created quick sketches of all the ideas we shortlisted to visualize how our ideas would look and iterate on those ideas.
The Solution: Fiverr Example
Since, our focus on majorly based on Fiverr, the final design was inspired by the Fiverr platform to showcase how the reviewing system on Fiverr ( on thus, other freelancing platforms can be improved)
We made paper prototypes that were digitized and converted to a Figma prototype to enable quick changes since for some functionalities we designed multiple variations to see which would work better, and to also allow interactivity.
Task Progress
To tackle the problem of lack of trust between the client and freelancers, we decided to make the task progression clear and make several iterations of its interface, some including file attachments for the completed sections.
Communication
Our research showed that poor communication was a factor in unsatisfactory reviews and ratings even if the work done met the requirements.
Consequently, we included the ability to archive parts of a chat with specific names for future reference to avoid scrolling or forgetfulness.
Reviews
We saw that some platforms mandate the reviews while some do not. For our case, we made leaving reviews mandatory for the clients.
However, to avoid possible inconvenience, we included quantitative and qualitative elements in the reviews and provided semi-auto-filled options. We also wanted to make the star rating more objective and not open to everyone’s interpretation so the number of stars selected prompted the selection of ‘experience words’ to select from.
Moreover, the chat tone of the communication between the client and freelancer was meant to be determined by AI. As a result, if the client chose a very low star rating, showing an inconsistency between the communication and rating, it would prompt the client to explain what went wrong
We also wanted to explore different alternatives to evaluate later by considering changing star ratings to emojis to see if that would to judge performance instead of arbitrarily deciding how many stars to give.
Growth and Algorithm Feedback
To not punish freelancers for their old work and avoid them creating new profiles when given bad reviews, we decided to provide the option to delete old bad reviews if there had been substantial good reviews in the aftermath to indicate improvement.
There would also be an analytics dashboard to give feedback to the freelancer on their performance to provide a guideline for getting better. This would be done using word cloud histories for summary information and an aggregate qualitative average rating from the private feedback clients give based on which the algorithm rates them, to improve algorithmic invisibility and foster growth.
Evaluation
To test how our ideas would be perceived by freelancers, we conducted usability testing with 3 users where we presented the prototype to interact with and perform the specified flows. We also asked them to choose between the alternative designs where applicable and provide the reasoning for their preference. Moreover, we discussed which functionalities would be perceived as useful and which would not.
Method
The users were given a background of the project and were instructed on how to interact with the Figma prototype and asked to interact with it by performing some user scenarios.
Insights
Users found the chat logs feature particularly convenient, as it eliminated the need to scroll through previous conversations.
The task progress tracker was also well-received, with most participants favoring the list format with attachments over the Kanban board, which was considered to include too much unnecessary detail.il.
The combination of auto-generated and custom reviews was appreciated for its balance, but there was concern that it might discourage users from writing personalized reviews, resulting in fewer unique responses
Participants valued the ability to monitor chat tone for consistency, which they felt would benefit freelancers. However, some expressed concerns over potential privacy implications.
Feedback on using emojis instead of stars for rating was mixed. Some liked the personal touch emojis brought, while others worried it might make it harder to gauge the freelancer’s overall rating.
Final Decisions
DECISION 01
Task Progress
We chose the alternative with the list of tasks with a checkbox to denote completion along with an expandable section to view attachments to prevent visual clutter.
DECISION 02
Emoji vs Stars
We chose the star rating since we had taken measures using the word clouds to maintain consistency between what each star rating meant. Moreover, a study by Alismail and Zhang (2018) showed that users found interpreting what the emojis mean to require more cognitive load and the use of emojis is friendly so it resulted in people choosing more positively than they felt.
REFLECTION
Limitations
Access to interviewees was limited, and we were not able to find clients, i.e. people who hired freelancers, to interview.
Our design included both the client side and the freelancer side, but our testing and feedback were limited to freelancers only. Therefore, there was no way of gathering information and insights on the client-side experience and usability.
Our interviewees were limited to a certain demographic - young South Asians from India and Pakistan.
REFLECTION
Retrospection
Initially, after conducting the interviews, we unearthed many different problems and uncovering ‘problems to solve’ made us try to solve for them. However, later we realized that while these problems are valid, it would move too far away from the project brief. This made me appreciate problem statement development since understanding which problems to focus on and fix and which are beyond the scope (in a project, or a sprint) is also paramount.
References
Alismail, Sarah & Zhang, Hengwei. (2018). The Use of Emoji in Electronic User Experience Questionnaire: An Exploratory Case Study. 10.24251/HICSS.2018.427.