How do we capture memories: Designing Conversational Motion States for a VUI
By integrating motion cues and abstract metaphors, the design enhances user interaction, making actions like recording, reminiscing, and deleting intuitive and immersive through natural gestures and dynamic visual feedback.
2 weeks
Digital Health & Wellness
2 Product Designers
Voiceflow, Figma, Google Suite
My Role
UX Design and Interaction Design
Conversational Design Strategy
Designing Intuitive Conversational States
Prototyping and Testing
Impact created
Demonstrated increased user engagement and ease of use by aligning conversational states with natural language cues, making actions like recording, reminiscing, and deleting intuitive for users.
Enhanced the user experience by creating smooth, natural transitions for each action while interacting with the mobile app.
5
Conversational main states conceptualised
30+
mid-motion states designed
Overview
In this project, I focused on creating high-fidelity prototypes for a VUI for a voice-assisted mobile app by designing six distinct user actions, each with over five motion states. The states are based on a conceptual mobile 'Memory' app that records, saves, deletes memories - in the form of an audio, video, images, etc or can be integrated with an existing Photos app with added functionalities.
Gestural Interaction Mapping: Observed and analyzed natural physical gestures to inform digital action states, ensuring each conversational design mirrored real-world gestures, creating a seamless bridge between user intentions and app responses.
Motion State Design for Intuitive UX: Developed multiple motion states per action to refine interactions like record, reminisce, and delete. This approach allowed for deeper engagement, where each action unfolded in fluid, contextually relevant steps, enhancing user immersion.
High-Fidelity Prototyping: Delivered detailed prototypes to capture the interactive nuances and motion flows of each state. This precision supported testing and demonstrated how experiential design could elevate the app’s usability and emotional connection with users.
We selected this topic as it gave us a chance to explore critical emotions associated with the action of recording memory and the emotionally charged state people are in when remembering. People take photos and use their camera rolls to remember key moments in their lives, but are now overwhelmed with the sheer number of photos to sift through. For a voice assistant function, we felt it made sense as this is an action a user can need to perform on the go/ instantly before it ends or before the user forgets, and the voice assistant can assist in the same. We felt designing voice states for the topic would be challenging, but worthwhile.
Defining the functions, traits and states of the Agent
Purpose of the personal assistant: The agent is focused on recording memories of a user. It can be used to quickly record, store, access memories in text, voice, visual formats, location, time, and event. This might add value to a user for efficiently recording important valuable moments, shared or personal. Users can feel more motivated, grateful and help them reflect on their life.
Defining Design metaphors of association - Function, Behaviors, Structure and State
Purpose of the personal assistant: The agent is focused on recording memories of a user. It can be used to quickly record, store, access memories in text, voice, visual formats, location, time, and event. This might add value to a user for efficiently recording important valuable moments, shared or personal. Users can feel more motivated, grateful and help them reflect on their life.
Metaphors of Crystal Ball, Pensieve and inside out references…
The wisps + mysticism of a Pensieve* + the globes + emotive colors of Inside Out** + the form, image viewing capability and glass structure of a Crystal Ball
*Pensieve from the Harry Potter world: A device used to collect 'wisps' of memories from the brain to collect and see/ experience them when needed
**Inside Out: A Disney Movie that shows the workings of a brain with memories collected through 'memory globes' that have different colors like yellow (joy), blue (sad), purple (fearful).
These metaphors are chosen as they are direct representations of examples of how memories are captured. They come with their own metaphors and logic that can work in a design of a VA for this topic.
Metaphors of How an eye works and behaves…
The eye was studied - how it works, moves and behaves. The concept was applied to this iteration as the form of the iris and pupil, the pupil's behavior to dilate and expand and action of blinking.
The same is applied as sequences of how the VA could respond, like intake stage of capturing a voice recording is how the pupil enlarges within the iris.
This metaphor is chosen as it is resonant of how an eye 'captures' a memory, because of which some actions like 'blinking' added to the VA design would be something users can understand visually.
Metaphor of a Camera Aperture's working
A camera aperture's working was studied, and its properties and behaviors were added to the design iteration. This metaphor was chosen as the user can visually resonate the design with the act of 'capturing' like a camera captures a picture. Subtle references to the working of an eye can also be seen as how the circle turns black in the deleting memory stage, like a pupil.
User Testing of concepts to understand more common associations
" The blink as an idea to 'capture' something makes perfect sense to me."
" The emptying of the globe for deleting memory is pretty good. I could understand that"
By understanding positive and negative connotations we moved forward with concepts that resonated emotionally with users the most.
The agent's concept was decided to take the form of a symbolic globe, a form recurring in many emotionally and physically associated metaphors.
As a team, we chose to focus on the bubble metaphor but redesigned the states to convey more abstract rather than literal bubble behaviors, retaining only the "pop" effect for the delete action. Most states are now represented by wisps within the bubble to signify different interactions.
We expanded the number of states to create clear distinctions between actions, such as when the agent is listening versus recording a memory. Additional states, including speaking, loading, and default, were added to enhance the user's understanding of each interaction stage.
Agent States were detailed keeping user behavioral styles and gestural associations in mind.
User Feedback
"This resonates with me personally, the deleting states seems like a therapeutic experiential journey to me."
"This has the potential to be an extremely impactful experience for me."
Meet Memaura!
My key take-aways
State differentiation improves user comprehension
Expanding the number of states (listening, recording, speaking, loading, and default) highlighted how crucial it is to distinguish stages in conversational interactions. This approach helped clarify each interaction’s purpose, guiding users effectively and minimizing confusion, particularly in a voice-interactive setting.
Abstract Metaphors can enhance usability
Working with an abstract bubble metaphor allowed us to create interactions that were both visually engaging and intuitively understood by users.
Observing real-world gestures inspires intuitive digital actions
Translating physical gestures into digital interactions underscored the importance of grounding design in natural human behavior.
