Origami Sensei
AI-embedded extended reality tool for learning creative task using hands.
This project has been accepted
for ACM CHI 2025 in Yokohama, Japan.
ROLE
Lead Researcher, UI Designer
DESIGN TOOLS
Figma, HTML/CSS
CLIENT
Embodied Computation Lab and
XRTC Lab at Carnegie Mellon University
TIMELINE
12 Weeks
Project Overview
Traditional learning methods for hands-on creative tasks, like origami, lack real-time feedback, hindering beginners' progress as learners are not able to track their progress and identifying areas for improvement. Origami Sensei project addresses this gap by leveraging modern computer vision and mixed reality technologies.
Origami Sensei empowers users to craft their own origami pieces from a collection of pre-existing designs. Leveraging advanced computer vision algorithms, the system recognizes the user's current origami step and delivers instant visual projections and verbal feedback to aid in successfully executing the subsequent steps. In this project I lead the exploratory design process to understand the design principle of creating learning tools using mixed reality.
RESEARCH PROCESS
What are the design principles in designing creative task learning tools in mixed reality environment?
I became a part of this project in August 2023, joining a team that had already laid the foundation for Origami Sensei. At that point, the team had successfully developed an early prototype featuring computer vision model. In this context, I then helped conduct series of research to determine appropriate concept.
Ergonomics Principles
Explorations
Before delving into the user's perspective, I decided to explore how principles from an ergonomic standpoint could enhance user experiences and interactions with Origami Sensei. However, finding existing design principles in the literature proved challenging. Despite spending hours reading papers, it appears that, given the tool's novelty, comprehensive design principles references are not widely available yet.
I shifted focus to understanding the ergonomics of the average person in relation to computer screens. This allowed me to grasp common-sense visibility and movement principles and affordances.
Optimal Viewing Zone
The optimal viewing zone for screens and technology interfaces refers to the ideal distance for comfortable and immersive viewing. Research suggests that the optimal zone falls between 20 and 28 inches away from the eyes.
Additionally, the angle at which your eyes interact with the interface matters. Ideally, your eye angle rotation should be no more than 30 degrees to avoid neck strain and fatigue.
Optimal Body Position
The optimal body position zone for screens and technology interfaces refers to the ideal distance for comfortable movement. The only thing research suggests that the optimal zone is that the screen / tools to be positioned within the limb length.
Additionally, we need to be mindful differences in people’s height because it would also affect their optimal viewing zone.
DESIGN PROCESS
How might we design an AI embedded XR learning tool to enable users in learning skills in creative hands-on tasks like origami?
Exploratory Making
Having established various measurements based on an ergonomic model, I proceeded to experiment with diverse physical setups for the mixed reality environment. Given the continuous demand of the computer vision model to detect origami movements, I opted for a table setup and I also decided to test out UI component.
PRINCIPLE 1
Readability
Projection’s visual component (text, graphic, line) should be clearly readable by user. Maintain a minimum paragraph spacing of 10px and utilize letter spacing. Choose Sans Serif fonts for clarity and reduced eye strain on projection.
PRINCIPLE 2
Legibility
Ensure a contrast between text/line and background colors for increased visibility. Avoid subtle color combinations that may not display well on projectors.
Place text, lines, and graphics where they won't be covered by hand movements for clear visibility.
PRINCIPLE 3
Intuitive Reading
The instruction area is placed on the left, and the next step is on the right side to facilitate more intuitive reading (from left to right: reading the instruction first, then seeing what they're going to make).
PRINCIPLE 4
Nudge for Progression
Instruction should include progress bar / step number to help user know where they are in the journey and nudge them to proceed with the process.
PRINCIPLE 5
Easy to Understand
Language used in the instruction content should be clear and concise. All signs, lines should also be introduced to the users before.
PRINCIPLE 6
Easy Error Recovery
The instructions should help users recognize when they've made a mistake and provide them with ways to recover.