UX Strategy | Design Research | Product Design
Review contribution experience
To comply with my non-disclosure agreement, I have omitted and obfuscated confidential information in this case study.
The information in this case study is my own and does not necessarily reflect the views of Chewy.
Problem
84% of millennials say user-generated content (UGC) influences their purchasing decisions, and it creates authentic evaluation of products.
UGC on Chewy primarily exist as content from Reviews. In 2023, Chewy’s new review volume was about 480K per year. The new review to order ratio is declining despite orders growing (from 2.55% in 2016 to 0.63% in 2022).
Solution
There is currently no single solution to this project, I led and delivered on multiple review contribution projects to optimized the review journey today for Chewy reviewers. We also came up with a vision on moving towards engaging new new reviewers to engage with Chewy’s review contribution system, and amplify their review experience. Chewy is still executing toward the roadmap set out from the vision today.
Role
Project management and coordination
Established a comprehensive work-back plan by coordinating with content and research partners, ensuring accountability for deliverables and sign-offs. Proactively accelerated the final deliverable to be completed three days ahead of schedule.
Define scope and timeline
Defined project scope and timelines in collaboration with product and engineering teams. Successfully delivered on short-term objectives while co-developing a year-long roadmap with the Product Manager.
Design strategy and storytelling
Created low-fidelity prototypes to collaboratively finalize a storyboard that articulates the team’s broader vision. This approach translated a complex review-contribution space into clear, actionable steps.
Design and execution
Created hand-off Figma files work with the design system team to perform accessibility checks. Participated in multiple rounds of QAT to ensure the final design met quality and user experience standards.
Impact
Optimized Collection Channel
Improve review form and remove friction
Scaling to other channels
Email Template Update
Redesigned the email template to improve content, layout and order logic
Result: Increased CTR from 2.5% to 6.2%, resulting in a 68% increase in daily review submissions. Encouraged 15K first-time review submissions within two weeks.
Review my purchases page
Post Launch, Reviews peaked in September (> 800k per month) and have normalized downward since then as people catch up with reviewing previous purchases. We had 761k in Oct and 663k in Nov. There was a 170% increase two weeks in launch, which far exceeded the projected 80% increase in review volume.
Expand review collection channel
One-tap rating MINT and tie pet to purchase: aim to bring personalization to the review experience.
additional notification-based initiatives to meet customer at the right time are currently being rolled out.
Context
I acted as the design lead, acting as partner to product manager in both defining the roadmap for 2-3 years as well as breaking down roadmap into sizable deliverables. I also worked closely with cross-functional partners in various teams (PDP, Polaris Marketing, Design systems, Delivery, Sampling programs, Customer Service)
When the team started I ran a workshop for the longer term vision planning which end up generating over 90 ideas across 4 themes.
This helps get a preview of all the possibilities for innovation and then ground us back in our current experience.
01
Discovery
Current State
How might we boost the quantity of customer reviews while ensuring that both reviewers and readers maintain confidence in the quality and honesty of those reviews?
02
Low-Fi Concept Design
Explorations on Landing Experience
In the landing experiences, I initially started with 3 divergent concepts in low-fidelity to move quickly with testing these concepts with users. Through concept testing and previous insights, I recognize that there’s a risk with the Product manager’s priority to focus solely on increasing ratings of the product, but emphasizing the importance of text reviews and photos is also important for customer trust and engagement, especially when evaluating a product. We end up iteration on the Miniform, to stike a balance between text reviews, and numerical ratings.
Stepped Review Cards Stack
Full form with a stepped process but with small footprint—a way to consider how to increase review volume for text and photos
One-Tap Rating cards
Cards that can be shuffled, to give a sense of gamified experience and lower the bar for rating an item.
Focus on collecting Ratings, but have reviews text, photos later to edit, etc
Mini Form:
A simplified form to not include things like review title
Only expanded to form after interacting with star-ratings
Post Review submission
Our initial ask included the legacy net promoter score in the post review process. However, this is not the right place to collect this. While I explored a few options for how we might potentially surface this, the real issue was in why we needed to include the NPS in the review process itself. From previous User Research (UXR) findings NPS score was perceived as a "trust buster" by customers. After speaking with Marketing, Voice of Customers, Customer Insights, and PR team, we finally were able to come to a consensus on removing this and monitor performance in the new experience.
I facilitated discussions these discussions, getting support from UXR, presenting evidence that the NPS score was legacy and no longer meaningful.
03
Design Iterations
Increase fidelity to address usability issues
In our earlier concept test with 6 people, some participants were confused as to whether further action is needed to "save" a star rating after clicking a rating in the email and lands on the review form. In addition, it was not clear whether the fields remaining were required or not. and some participants failed to click the submit review button after typing out their review text. So there's a need to distinguish the two steps.
Making Rating-saved more obvious
Option 1:
Using the confirmation tag from the design system. While this worked, it added more to the visual complexity of the form that made it easy to miss.
Option 2:
This option make editing star rating less accessible. While it provided benefit of emphasizing the ratings saved message, we want to still give the autonomy and remove any friction for updating star rating.
Option 3:
The simple option of tying confirmation closer to the star ratings. After talking to a small sample of customers, we feel this is the option we are most confident in moving forward with.
Product Card Widget
Personalized Review Writing prompts
Scaling this design to other ingresses
The widget designed for the new launch extends its reach beyond just the review page. Built as a React component, it is highly versatile and scalable. As part of the next steps, we integrated the pet profile team’s existing MINT framework to enhance review collection. This integration is projected to increase review volume by 30%, while also enabling the pet data collected to improve personalization efforts and provide deeper insights into pets’ favorite foods
04
Forward looking
This is just the beginning
Although the first phase achieve some success, we want to look ahead, as this is only the first step, there are still many questions to be answered to get all of the features to completion. A lot of the ideas coming out of this MVP is currently on the roadmap for the next two years.
Known Insights
Customers expect their reviews to be acknowledged.
“Neither Chewy or Amazon have sent me a review confirming that my reviews have been published and it’s been about a week.” - Chris who may have been using a shared account.
The primary channel for review submission is via Email (comprised of 97% of all reviews submitted)
only 2% of active customers submit a review and of those that submit, they only submit one review per order.
Having validation and reward through the process will be encouraging to customers.
“And Chewy, I think it took longer to publish my review, and I wasn't notified. And those are little, tiny things that don't affect the review itself, but it sort of can make a difference to the reviewer... so anything that you can do to make that process rewarding... would be a good thing.” – Deborah
Acknowledge and Appreciate
Creating a sense of validation and reward with a simple point system in the review space that brings a sense of delight and sense of accomplishment for their contributions.
Ability to Edit and Revise
Participants proactively recognized the need to be able to see past ratings/reviews previously provided.
This also came from customer calls around customers having to try the food over time, and that some brands the food changes and would warrant another evaluation on the product after a period of usage.
Engagement
One of the biggest themes from our workshop is to create engagement and building towards sharing to the community with visual media. With our mobile-first experience, encourage customers to upload photos is a great way to build engagement with the community.
In our diary study, particiapnt also found photo and videos to be compelling “Cats.. they are not paid actors, and they don’t lie.”
Quality of Review
Even though the primary goal is to increase review volume, part of what customers value about chewy is the community of authentic pet parents. So while we were working on the short-term solution for minimizing friction, I’ve also come up with some ways to help improve review writing to ensure that the process is easy, and that customers are able to provide quality reviews.
AI-powered Review writing process
We’ve heard from research that people are unwilling to leave reviews or that they need to sift through reviews to find what is helpful for them. In order to optimized the review contribution, I met with our 3-rd party vendor BazaarVoice to learn about their advanced AI-feature that could support helpful prompts to guide review writing. In addition, by leveraging AI to generate a title, it removes the burden on the customer to think of a review title, especially when they have spend time providing long feedback.
05
Impact
Within just two months of launching this feature, we achieved a +800k in the first month from the baseline of 480k review / year. This created a 5-fold increase in review volume/month, and 3-fold increase in review per customer.
New product review volume is up by 15K in first two weeks. This success not only accelerated the 'Tie Pet to Purchase' initiative through one-tap rating but also paved the way for expanding the Reviews widget to other product areas.
Reflection
Start with a plan, stay flexible.
Even without all the details upfront, having a plan provides direction. Having a work back plan help guide the process The first solution won’t be perfect, but understanding that the team is building towards a broader vision will help us stay on track.
Question the status quo.
Just because something has always been done a certain way doesn’t mean it’s right. Thoughtful questioning and focusing on user needs lead to better outcomes.
Small changes = big impact.
Targeted, strategic improvements can drive meaningful impact faster than large-scale overhauls. While we don’t have the technology to implement Ai-features to help review process, having dynamic content as a first step to refining the prompts allowed us to see progress toward the broader goal.