Yes Health Mobile App
Enhancing User Trust in AI Coaching
The Yes Health app offers online health coaching to assist individuals in preventing diabetes and losing weight. Users can submit pictures and a text description of their meals, and receive feedback from an AI or human health coach. My work helped the team shape feature requirements and generate effective and simple solutions to improve user experience with the AI coach. I collaborated closely with the design lead, product manager, developers, and coaches to accomplish these goals.
Background
Yes Health is invested in scaling their personal health coaching services using AI-powered responses. The AI coach ‘Cala’ recognizes texts and images from user nutrition logs, and is able to provide relevant feedback to improve the user’s dietary habits. Before expanding the AI feedback service, Yes Health considered the potential negative impact on user engagement. My goal was to first investigate users’ impressions of Cala as an AI health coach, and leverage that information to improve the user experience.
My role
-
UI/UX Design
-
User Research
-
User Testing
Tools
-
Figma
-
Jira
Project timeline
5 Months
Understand & Research
I first met with the product lead and machine learning engineer to define our goals. Although the intention of the feature was clear, the specifications had not been detailed. Understanding the perspective of the users as well as their interactions with Cala was my first priority.
What we know about our users
-
Typically between 40-60 years old, around 70% female and 30% male.
-
Many users have struggled with weight gain, and still struggle despite the use of other programs.
-
Some are beginners in their journey of health-consciousness.
-
Users typically seek both accountability and connection through Yes Health programs.
I created these model user profiles based on our previous research to help our team visualize and identify with our users and their needs
User Feedback
Our support and coaching team have a close relationship with our users. Here is what I learned after collecting user feedback:
Receiving non-human responses made the users feel they were not held accountable anymore.
“I feel like the comments I get for my meal logging are canned responses and not a real person anymore. Has something changed ?”
“ I think I started feeling I didn’t have to be held accountable anymore. I want to be successful with this program”
Research on AI-generated messages
AI Chatbot
I looked into conversational design resources and researched existing solutions from other competitors.
Our AI coach, Cala
I interviewed the machine learning engineer to understand our trial process and Cala's capabilities. How did we want Cala to interact with our users? And what was our plan for Cala’s involvement in future health coaching?
Here are my findings from some of the competitors I researched. I wanted to see how AI was introduced to the user, as well as the visual and text interface used in the screen.
Key findings
-
Many users look for human connection in the app.
-
Informing the user at the beginning of the conversation that the user is communicating with an AI can make users more forgiving. This transparency also guides the user to keep the conversation within the bounds of the AI’s capabilities.
-
Adding a slight delay to the AI’s responses makes them more human-like and therefore more credible.
-
The AI requires a complicated set of criteria to be fulfilled in order to yield a response for the user.
Based on these findings, I was able to shape feature requirements and define our criterion for success.
Success Criteria
The implementation of this feature should result in little to no impact on the number of activities and meal logs after the trial.
Brainstorm & Design
To gain a full picture of the important stakeholders, I first mapped out the interactions between users, human coaches, and the AI coach. Then, I started brainstorming and exploring different ways to introduce Cala in a more active role. We planned to get feedback from users based on the two most feasible designs.
Mapped out the interaction
I aimed to conceptualize the workflow of the application, starting with the user opening the application, and ending with the user’s response to a health coach’s feedback.
I also examined how users tend to log their meals, and view feedback.
After considering the time and resources, we found it more feasible to limit the design scope within this screen.
Listed out questions and ideas
How might we...
-
help users differentiate AI coaches from human coaches?
-
tell users about the usage of AI? How?
-
make users feel that they are still being cared for by human coaches?
I also sketched low-fidelity wireframes to flush some ideas out.
Should we differentiate Cala by showing her ability to respond quicker than human coaches?
If Cala is capable of responding right away, why navigate users away from the meal logging screen? Can we simplify the flow by keeping users on the same screen to log and view their feedback?
Although Cala's feedback can be instant, it doesn't converse with the users. Plus, the idea would alter the existing flow and inevitably complicates the implementation. After discussing it with the team, I eventually discarded this idea.
Focus on the meal logging screen
My approach was to simplify the user’s interactions and the interface design to achieve our goals. I discussed this with the product manager, and was able to convince the team to disregard option B (see below). The icon was too easy to miss, and the size of the touch target was too small.
Keeping the screen clean and simple runs the risk of users missing the important information.
It occupies considerable vertical space. Content length is more limited. But, it doesn’t require interaction to convey important messages
Down to two candidates
We wanted to get the ideas tested as soon as possible so the feature could be released soon after the trial started. Although new design features have been planned to ease the accessibility of the app, the following mockup was tested with the old design to minimize distraction.
User Interview/ Test
I collaborated with the design lead to create an interview plan and guide. The design lead facilitated the sessions while I observed, took notes, and consolidated our findings after the interviews.
We conducted one-on-one Zoom interviews with ten current users who have received meal feedback from Cala. There were two parts to this interview: learning about the user’s experience, and presenting our design mockups for feedback.
Discoveries
-
Some users were not aware that their responses were coming from an AI coach.
-
8/10 Users didn’t mind receiving meal feedback from Cala although many prefer human coaches when they needed additional help.
-
Users were unaware that human coaches were monitoring at the background.
-
Users did not identify a significant advantage between receiving an instant response versus receiving one a few hours later.
All users preferred design A
This effectively communicated to users the distinction between AI and human coach messages. This feature assured users that a human was monitoring their experience and that they could easily reach a human coach if needed.
In contrast, users showed little interest in clicking on the info box to learn more about the product. Additionally, senior users were particularly cautious and avoided exploration in fear of “breaking things.”
Opportunities for Improvement
-
The message is too long in Design A.
-
Users often ignore Cala’s messages. the avatar was also used to send other daily announcements that users considered “canned and spammy.”
-
Users need more personalization- many reflect on the need for different coaching styles.
Iterate and Deliver
I trimmed down the length of the wording, and incorporated the updated design patterns to enhance accessibility and consistency. We were able to hand off the design to both our IOS and Android developers and release the update in a short period of time.
Result
After the feature was launched, I worked with the product manager to closely monitor data and user feedback. A month after the release, we found that user engagement with logging activities and meals remained the same as before the trial. The team has continued to roll out AI-generated responses.
Learnings and Refleciton
Design for senior adults
Senior adults can be power users of our health digital products. It's important to take into account their concerns about “making mistakes”. To address this, the design should instill confidence and encourages exploration.
Tradeoffs of Simplicity
Designing the mobile interface involves balancing layout simplicity and content delivery. It's crucial to weigh the risk of not displaying certain information when making these tradeoffs.
Challenges with remote interviews
Remote user testing revealed some participants were not tech-savvy and had trouble with Zoom calls. To overcome these challenges, it's important to have a backup plan, such as a phone call.
More explorations
The design exploration phase was limited by time constraints. Given more time, I would have explored additional solutions to the problems.
Next Steps
The team will continue to monitor the data to see whether there are any changes in users' engagement. After gathering insight from users, we have identified opportunities to integrate Cala earlier in the onboarding process. We also want to explore ways to help users gain more control over their coaching preferences.
