Chien-Pang Lin, Cheng-Yao Wang,Hou-Ren Chen, Wei-Chen Chu, Mike Y. Chen

Published in MM 2013(short paper)

We present RealSense, a technology that enables users to easily share media files with proximate users by detecting the relative direction of each other only with built-in orientation sensors on smartphones. With premise that users are arranged as a circle and every user is facing the center of that circle, RealSense continuously collects the directional heading of each phone to calculate the virtual position of each user in real time during the sharing.

RealSense users can simply share photos to others by swiping and throwing the photo to another user’s direction without remembering or searching the name and even the device id of a specific receiver. We evaluated the orientation sensor error and the minimal arc degree for selection, and compared RealSense with linear menu, pie menu and NFC. Our results show that the limitation that requires participants to face toward the circle center is rather acceptable and participants preferred RealSense than other sharing interactions especially they were unacquainted with each other.

Cheng-Yao Wang,Wei-Chen Chu, Hou-Ren Chen, Chun-Yen Hsu, Mike Y. Chen

Published in CHI 2014(full paper)

We present EverTutor, a system that automatically generates interactive tutorials on smartphone from user demonstration. For tutorial authors, it simplifies the tutorial creation. For tutorial users, it provides contextual step-by-step guidance and avoids the frequent context switching between tutorials and users' primary tasks. In order to generate the tutorials automatically, EverTutor records low-level touch events to detect gestures and identify on-screen targets. When a tutorial is browsed, the system uses vision-based techniques to locate the target regions and overlays the corresponding input prompt contextually. It also identifies the correctness of users' interaction to guide the users step by step.

We conducted a 6-person user study for creating tutorials and a 12-person user study for browsing tutorials, and we compared EverTutor's interactive tutorials to static and video ones. Study results show that creating tutorials by EverTutor is simpler and faster than producing static and video tutorials. Also, when using the tutorials, the task completion time for interactive tutorials were 3-6 times faster than static and video tutorials regardless of age group. In terms of user preference, 83% of the users chose interactive type as the preferred tutorial type and rated it easiest to follow and easiest to understand.

Cheng-Yao Wang,Wei-Chen Chu, Po-Tsung Chiu, Yih Harn Chiang, Mike Y. Chen

We present PalmType, which uses palms as interactive key- boards for smart wearable displays, such as Google Glass. PalmType leverages users’ innate ability to pinpoint specific areas of their palms and fingers without visual attention (i.e. proprioception), and provides visual feedback via the wear- able displays. With wrist-worn sensors and wearable dis- plays, PalmType enables typing without requiring users to hold any devices and does not require visual attention to their hands. We conducted design sessions with 6 participants to see how users map QWERTY layout to their hands based on their proprioception.

To evaluate typing performance and preference, we conducted a 12-person user study using Google Glass and Vicon motion tracking system, which showed that PalmType with optimized QWERTY layout is 39% faster than current touchpad-based keyboards. In addition, PalmType is preferred by 92% of the participants. We demonstrate the feasibility of wearable PalmType by building a prototype that uses a wrist-worn array of 15 infrared sensors to detect users’ finger position and taps, and provides visual feedback via Google Glass.

Cheng-Yao Wang,Po-Tsung Chiu, Min-Chieh Hsiu, Chiao-Hui Chang, Mike Y. Chen

Excellent Work,The 8th Acer Long-Term Smile Innovation Contest, 2014

We present EyeWrist, which uses palms as the gesture interface for smart wearable displays such as Google Glass. With abundant tactile cues and proprioception on palms, EyeWrist can also be leveraged for device-less and eyes-free remote for smart TVs. EyeWrist embeds a micro-camera and an IR laser line generator on the wristband and use computer vision algorithms to calculate the finger’s position on the palm. Without requiring the line of sight of users’ fingertips on palms, the camera height could be lower, making the whole device more portable. We also implemented a gesture recognizer to distinguish different symbols, letters or touchscreen gestures(e.g. swipe, pinch) on palms. The recognition result would be sent to smart devices via Wi-Fi for gesture-based interaction.

Submitted to ACM MobileHCI 2015, Full Paper

With abundant tactile cues and proprioception on palms, the palm can be leveraged as an interface for eyes-free input which decreases visual attention to interfaces and minimizes cognitive/physical effort. In this paper, we explored eyes-free gesture interactions on palms, which enables users to interact with devices by drawing stroke gestures on palms without looking at palms. In two 24-person user studies, we observed that (1) users preferred using the whole palm region as the gesture interface with 3 categories of hand orientations. (2) users tended to draw different gestures from the same start region due to proprioception on palms. (3) the accuracy of eyes-free Graffiti input and multi-stroke gesture recognizer on palm interface reached as high as 98% and 95% respectively. (4) the tactile feedback of palms can significantly help participants build up spatial memory of gestures. Also, we implemented EyeWrist that turns the palm into a gesture interface by em- bedding a micro-camera and an IR laser line generator on the wristband, and three interaction techniques that takes advantages of palm characteristics are proposed. The preliminary evaluation revealed that EyeWrist enabled users to draw graffiti letter and multi-stroke gestures with 90% above accuracy, and that the concept of eyes-free palm-based gesture interaction was appealing to users.

Cheng-Yao Wang,Po-Tsung Chiu, Min-Chieh Hsiu

12/03 Final,MobileHero – user experience design competition, 2014

We present EyeWatch, which uses back of the hand as gesture interface for smart watches. EyeWatch not only overcomes the Big smartwatch problem: occlusion and fat finger problem, but also enables more powerful and natural interaction such as drawing a symbol quickly to open an application, or intuitively handwriting on back of hand to input message. Our proof-of-concept implementation consists of a micro-camera and an IR laser line generator on the smart watch, and computer vision algorithms are used to calculate the finger’s position on the back of hand.

Cheng-Yao Wang,Min-Chieh Hsiu, Chin-Yu Chien, Shuo Yang

ACM UIST 2014 Student Innovation Contest

Imagining you were shrunk, you can explore your big house, play with big pets and family. To fulfill the imagination and provide users with incredible shrinking adventures, we use a robotic car and a google cardboard which turns a smartphone into a VR headset. We build a robotic car and attach a smartphone on the pan/tilt servo bracket. stereo images are generated from smartphone’s camera and are streamed to the other smartphone inside of the google cardboard. When users see the world through the smartphone on robotic car, they feel they were shrunk.