This is not a real project, but its components are real. The research and the tests were conducted, the designs were created and tested.

Project background

In an era marked by rapid advancements in wearable technology and augmented reality, there exists a unique opportunity to revolutionize the way we interact with our living spaces. This project aims to leverage the potential of smartwatches and augmented reality to create an innovative solution for seamless smart home control. Upon successful implementation, this smartwatch-based AR application has the potential to redefine how users interact with their smart homes and smart watches. It promises a seamless, intuitive, and immersive control experience, setting a new standard for smart home management. By combining the convenience of wearable technology with the power of augmented reality, this project represents a significant leap towards the future of smart home automation and user-centric interfaces.

My role

As a UX/UI designer with a specialized focus on Virtual Reality (VR), Augmented Reality (AR), Mixed Reality (MR), and Extended Reality (XR), I play a pivotal role in shaping the user experience and interface of our innovative smartwatch-based AR application for smart home control.

My responsibilities encompass leveraging my extensive experience in UX design to craft an intuitive and user-centric interface that seamlessly integrates the use of the smartwatch with the augmented reality environment. I lead the design efforts to create a compelling and immersive user experience, drawing upon my expertise in VR/AR/MR/XR design. This includes designing intuitive gestures and touch-based controls that allow users to interact seamlessly within the augmented reality space, also taking into consideration body poses which do not make the users uncomfortable.

I am adept at creating wireframes, prototypes, and mockups that visually communicate the application’s flow, interactions, and visual design elements. This aids in the iterative development process and ensures alignment with project objectives. Additionally, I actively engage in usability testing, gathering valuable feedback from users to refine and enhance the application’s interface. This iterative approach ensures that the design meets and exceeds user expectations.

My expertise in UX/UI design, particularly within the realm of VR/AR/MR/XR, is instrumental in creating an application that offers an intuitive interface for smart home control. My ability to balance aesthetics with functionality ensures that the application not only looks visually appealing but also provides a seamless and effective means of interaction.

The problem

From the user’s perspective, traditional smart home control applications often present several usability challenges. These apps typically demand more time and attention to navigate, requiring users to sift through menus and buttons. Additionally, they are tethered to a specific device, which might be located on the opposite end of the house, leading to inconvenience and potential delays in device activation. Moreover, connection issues can further impede the seamless operation of these apps.

Furthermore, the reliance on traditional apps may not align with the dynamic and on-the-move lifestyle of users. Having to locate and unlock a device every time a light needs to be adjusted can disrupt the flow of daily activities. Using their smartwatch as a strigger basically gives them the opportunity to ahve everything just one tap apart.

The goal

The primary goal of our application is to revolutionize the way users interact with their smart home environment by seamlessly integrating wearable technology and augmented reality. By combining the capabilities of a smartwatch with an immersive AR interface, our objective is to provide an intuitive, efficient, and immersive control system for managing smart home devices.

This application aims to address the limitations of traditional smart home control apps, which often require more time to navigate and are bound to a specific device’s location. Additionally, it mitigates connectivity issues and caters to users’ dynamic, on-the-move lifestyles. The solution seeks to offer a user-friendly interface that eliminates the need for physical proximity to devices, enhancing the overall smart home experience.

Specifically, our initial focus will be on enabling users to effortlessly control lighting within their smart home. This foundational feature will serve as a stepping stone towards expanding the application’s capabilities in the future.

Ultimately, our application aspires to set a new standard in smart home management, providing a seamless, intuitive, and integrated experience that aligns perfectly with the way users live and interact with their environments.

Meet our Personas

The Research

As a researcher for the Smartwatch-Based AR Application for Smart Home Control project, my approach involves a multifaceted methodology to gather insights and validate design decisions.

Personal Interviews:
Engaging with potential users, including younger individuals with tech-savvy backgrounds and older users less familiar with advanced technologies, was a fundamental step. These interviews allowed me to understand their daily routines, pain points in smart home management, and their expectations from a more intuitive control solution. Valuable qualitative data was collected, providing a deep understanding of user needs.

Market and Technology Research:
In addition to interviews, extensive research was conducted using various resources. This involved studying the competitive landscape of smart home control apps, wearable technology, and emerging trends in AR applications. Insights on the latest developments in wearables, AR, and smart home automation were gathered. This research was crucial in identifying areas for innovation and ensuring our application aligned with current industry trends.

Leveraging ChatGPT for Knowledge Expansion:
The utilization of ChatGPT played a pivotal role in expanding my knowledge base. It assisted in answering specific questions, generating ideas, and providing additional context for the research. By leveraging this AI language model, I was able to gain valuable insights on relevant topics, further enriching the research process.

Usability Testing:
Representative users were engaged to conduct usability tests with interactive prototypes of the application created in Bezi. Observing how users interacted with the interface allowed for the collection of valuable feedback on usability, navigation, visualizations, and overall user experience. This iterative process was essential in refining the design, identifying areas for improvement, and validating design decisions.

Analysis and Synthesis:
Data gathered from personal interviews, market research, ChatGPT interactions, and usability tests were analyzed and synthesized. This process identified common themes, patterns, and user preferences. These findings provided a comprehensive understanding of user needs, challenges, and expectations in the context of smart home control.

Using the research insights, I iterated and refined the design of the Smartwatch-Based AR Application for Smart Home Control to have an MVP. By incorporating user feedback and addressing usability concerns, the aim is to create a user-centered solution that meets the specific needs of individuals seeking a more intuitive and efficient way to manage their smart home environment.

Through this research process, I’ve gathered valuable insights, validated design decisions, and ensured the development of an application that aims to be both efficient and user-friendly for smart home management.

The most important pain points

– Gesture Recognition Accuracy:
Achieving precise and reliable gesture recognition in the AR environment can be challenging. Factors such as varying lighting conditions, hand positioning, and occlusions may affect the accuracy of gesture recognition algorithms.

– User Interface Design Complexity:
Balancing the visual aesthetics of the user interface with its functional efficiency in both the smartwatch and AR environment is a critical consideration. Designing an intuitive interface that effectively communicates control options while avoiding clutter or confusion is a potential challenge.

– Optimizing for AR Performance:
AR applications require careful consideration of performance optimization to ensure smooth and responsive user interactions. Balancing graphical fidelity with real-time tracking and responsiveness can be a complex task.

– Usability Testing and User Feedback:
Conducting effective usability testing for an AR application presents unique challenges. Ensuring a diverse range of test scenarios and gathering feedback on gesture recognition, touch controls, and overall usability in an AR context may require specialized methodologies.

– Accessibility Considerations:
Ensuring that the application is accessible to a wide range of users, including those with disabilities, presents a challenge. Providing alternative interaction methods and considering accessibility features will be important.

My problem statements

Sofia (user) is a working mother with a demanding schedule (user characteristics), who needs to efficiently manage her smart home (user need) because she has limited time and complex traditional apps (insight).

Pierre (user) is a hardworking bakery owner (user characteristics), who desires a practical and straightforward method to manage his home environment alongside his business (user need) because he has a preference for solutions that enhance efficiency without adding unnecessary complexity (insight).

Usability study

Study type: In-person usability testing

Location: Conducted in my home with my family and friends.

Participants: Younger and older family members and friends, who are less or more familiar with these technologies. Aim for a sample size of 5-6 participants.

Duration: The usability study is estimated to last approximately 15-25 minutes per participant, including setup, tasks, and post-task interviews.

Results: The usability study aims to identify usability issues, gather user feedback, and obtain valuable insights to refine the design of the application, ultimately improving its usability and user experience.

Low-fidelity design (wireframes)

High-fidelity designs (Figma)

High-fidelity designs (Bezi)


Reasoning behind decisions

Hand Posture:
We opted to prioritize hand posture in the AR environment to ensure natural and comfortable interactions for users. By considering ergonomic factors and designing gestures that align with how users naturally position their hands, we aim to minimize physical strain and enhance the overall user experience. This approach also promotes intuitive interactions, allowing users to seamlessly navigate the AR environment without conscious effort. Because we intuitively know how to hold our phones we aimed to use the same position when the application is visible. To minimize collosion errors we flipped the initial direction – this also helped to trigger the watch in a friendlier way.

Smartwatch as a Trigger:
We chose to utilize the AR watch as a trigger for several reasons. Firstly, a smartwatch is a device that users typically wear throughout the day, making it readily accessible and convenient for initiating interactions with the AR environment. This familiarity enhances user engagement and ensures that the control interface is always within arm’s reach. Additionally, leveraging the smartwatch as a trigger aligns with our goal of providing a seamless and integrated smart home control experience. Of course for this the technology for AR glasses has to develop far better, because the watch is just the trigger, the most of the work is done by the AR glasses or headset.

Mobile-Like Interface and Gestures:
The decision to incorporate a mobile-like interface and gestures stems from the ubiquity and familiarity of smartphones in today’s digital landscape. By mirroring elements of mobile interfaces, we aim to provide users with an intuitive and recognizable interaction model. This approach reduces the learning curve for users and leverages existing muscle memory associated with smartphone interactions, facilitating a smoother transition to our AR application.

Direct Touch:
Direct touch interactions were integrated to enhance precision and tactile feedback in the AR environment. Enabling users to interact directly with virtual elements fosters a more immersive and intuitive control experience. This approach empowers users to engage with their smart home devices in a manner that closely resembles physical interactions, bridging the gap between the digital and physical worlds. Additionally, direct touch supports a more responsive and immediate feedback loop, further enhancing the overall user satisfaction and control efficiency.

Seamless Integration of Wearable and AR Technologies:
The decision to seamlessly integrate wearable technology with augmented reality serves to leverage the strengths of both platforms. By utilizing the smartwatch as a trigger and incorporating AR interactions, we create a symbiotic relationship that enhances user convenience and control capabilities. This integration allows for continuous tracking of the smartwatch’s movement, enabling real-time responsiveness and a heightened sense of presence within the AR environment.

User-Centric Design Approach:
Throughout the decision-making process, a user-centric design philosophy was at the forefront. By considering factors such as hand posture, familiarity with smartwatches, mobile-like interactions, and direct touch, we aimed to prioritize user comfort, ease of use, and efficiency. This approach ensures that our application aligns with user needs, preferences, and daily routines, ultimately delivering a more personalized and satisfying smart home control experience.

Accessibility considerations

By incorporating these accessibility considerations and best practices into the design of the AR application, we could create a more inclusive and user-friendly experience for individuals with diverse abilities and needs.

Text and Font Size:
We ensure that text within the AR interface is legible and easily readable.

Contrast and Color Choices:
We use high contrast color combinations to improve visibility for users with low vision or color vision deficiencies. We avoid relying solely on color to convey information; incorporated other visual cues as well.

Navigation and Wayfinding:
We implement clear and intuitive navigation cues to guide users through the AR interface.

Avoid Clutter and Overcrowding:
We kept the AR interface uncluttered to prevent visual overload, which can be particularly challenging for users with cognitive or attention-related disabilities.

Accessibility Guidelines and Standards:
Adhere to established accessibility guidelines such as the Web Content Accessibility Guidelines (WCAG) and incorporated them into AR design principles.


In conclusion, the Smartwatch-Based AR Application for Smart Home Control represents a groundbreaking leap forward in the realm of smart home management. By seamlessly integrating wearable technology with augmented reality, we have redefined the way users interact with their environments. The combination of intuitive hand gestures, direct touch interactions, and the familiarity of a smartwatch trigger creates a natural and immersive control experience.

Through extensive research, user interviews, and iterative design feedback, we have ensured that the application addresses the specific needs and preferences of a diverse user base.

As we move forward, our ongoing dedication to user feedback, continuous improvement, and adherence to accessibility standards will further refine and enhance the Smartwatch-Based AR Application for Smart Home Control. We believe that this innovative solution will not only revolutionize smart home management but also serve as a testament to the power of technology in enhancing our everyday lives. With this application, we are poised to set a new standard in smart home control, offering a seamless, intuitive, and immersive experience for users around the world.

Tools used


Try the Prototype