Motion Laboratory

Sightseeing spots in Japan are becoming more popular with foreign tourists. In this research, I developed a virtual experience system so that such tourists can learn the manners of Japanese shrines with HMD and Leap Motion to deepen their understanding of ritual in Japanese culture.This system allows users to visit a virtual shrine by HMD. By gestures that are input using Leap Motion and HMD, users can virtually experience three actions: chozuya (purification), hairei (worship), and omikuji (written fortune) at the virtual shrine. In the chozuya and omikuji experiences, Leap Motion detects the motions of hands and users can virtually experience these actions with particular gestures based on customs. For the hairei experience, Leap Motion recognizes the clapping gesture, and HMD finds the incline of the user's head. Users can play nirei-nihakushu-ichirei (two bows, two claps, and one bow) as a virtual experience of Japanese shrine manners.To examine the operability and usability of this system, I asked one foreign teacher and nine Japanese students in our university to try it. Our result suggests that system operability slightly improved their understanding of Japanese shrine manners.

Even though team play is more important than individual skill in handball, self learning is difficult because there are few documents and videos of teaching materials. I propose learning support contents for handball team play for beginners using CG animation to understand the movements of team play by operating part of attack patterns. This study addresses a three-player attack pattern called Yugoslavia, which includes team-play motions of position changes between two players, passes between players, passes to a shooting player, and shots. I recorded the main motions of handball using a motion capture system to reproduce these motions with CG animation. The user operates one of the three players : center, left, and right. The other two characters are controlled by AI or respond to the user input. Handball beginners and experienced players operated these contents and answered a questionnaire to evaluate the learning support contents. Most beginners and experienced players manifested interest in team play. In addition, the experienced players understood the content of the learning support system, but many beginners believed that it was difficult.

A Buddhist ceremonial procession called Nerikuyo is an outdoor event where people walk while wearing religious costumes on a bridge in lines.This procession is difficult to accurately record and transmit because visible things change by time and place.I developed a system using 3DCG and interactive techniques to simulate the human animation of Nerikuyo for its digital archiving and simulation.In this research, whose target is the Nerikuyo of the Taima temple, this system reproduces the appearances of people who are walking on a bridge in lines in 3D space.Users can edit the number of people walking, the distances among them, their walking patterns in the text file, and simulate the lines of people, the human movements, and the elapsed time.The created Nerikuyo 3DCG can be previewed from different viewpoints using HMD and Leap Motion.With a collision detection between a button displayed on the 3D space and hands recognized by Leap Motion, the camera position is changed to a predefined location, and the camera orientation is controlled by the head tracking of HMD.Ten students used this system and answered questionnaires.The questionnaire results revealed that 70% of the users believed that this system effectively represented the configuration of Nerikuyo.This result confirms that this system is an effective simulation scheme for Nerikuyo.

The basic form of boxing, center gravity movements, and wrist rotations are important factors to improve boxing skills, but they are difficult to understand because they are not visible. In this study, I calculated the physical feature values of punch motions and analyzed them. First, I measured motions of a jab and a straight punch using an optical motion capture system. I calculated the following as features from 3D positions that acquired markers: speed and distance punches, angles that open between the body and the arm, and the wrist rotations when making a jab, and the center of gravity and the supporting leg when making a punch. Since the opening angle between the body and the arm is based on whether the side is open when making a jab, I calculated the angle from the marker positions of the left hip, the shoulder, and the left elbow. My analysis results of punch motions using calculated feature values revealed that the speeds of both a jab and a straight punch might be maximized by elongation arms. The wrist rotation is delayed by the opening angle between the body and the arm. In addition, the center of gravity position is moved more in a straight punch than in a jab. To determine the validity of my obtained features, inexperienced persons, beginners, and professional boxers evaluated them. As a result, in terms of the effectiveness of the opening angles and the wrist rotations, the evaluation of professional boxers was high, and the evaluation of most inexperienced persons was low. Feature values are difficult to imagine for inexperienced persons. However, in terms of effectively learning boxing, the evaluation of novices was the highest.

Although Japanese temples have many sliding-door paintings that are valuable cultural properties, displaying all of them at museum is difficult. In this study, I developed a simulation system of sliding-door arrangements using 3DCG and interactive techniques to support museum exhibitions. The system represents by 3DCG the state of the sliding doors and the rooms, which are actually difficult to see, to support the understanding of the actual exhibits by interactive operations in such virtual exhibits as fitting sliding doors to the current arrangement or trying a new arrangement. This system can arrange sliding doors by touch input and display the result in 3D space. Considering visibility and operability, the sliding doors are lain as a development view and arranged in a perspective from the top. Functions include automatic correction and rotation by size and the creation of puzzles by the random selection of sliding doors based on difficulty. This system, which was exhibited for three days as a special exhibition at the Ryukoku museum, was evaluated by 152 visitors. Approximately 90% deeply understood the arrangement of the sliding doors and the state of the rooms, suggesting that our system provides simple operability and can be part of exhibition support systems in museums.

Shomyo is an ancient style of Japanese Buddhist chant that is over 1000 years old. This study supports understanding Shomyo scores that are generally shown in unique notation to stimulate interest in Shomyo by people who are unfamiliar with it. I developed a creation-and-play system for Shomyo scores for tablet computers. In the actual scores of Shomyo chants, the melody is described by a combination of black ink lines and symbols that represent the tone's pitch. In this system, Shomyo scores are reproduced on a tablet based on the input of users who create scores by dragging ink line images and lyrics. Sound can also be generated that corresponds to the score to confirm that the user is actually chanting a Shomyo. The ink lines, which are arranged by users, automatically lead and guide users based on the actual notation of the Shomyo score. I made sound files that correspond to the lyric's syllabary and reproduced sounds that correspond to the input by changing the pitch within a range of five lower and upper levels. In this system, users can input and play eleven sounds that are written in ink lines.From questionnaire results about our system's usefulness, I obtained high evaluations about understanding Shomyo musical scores. I also identified improvements in the sound play that I must address in future work.

In this study, I simply created three-dimensional animations of the human body at low cost. I developed a choreography synthesis system by inputting body-part motions using a Kinect. During the playback of the existing motion data, the motions of both arms inputted by Kinect were synthesized to the motion data in real time. Users combine steps from 40 hip-hop dance steps that were prepared in four counts in time series and input the arm motions to the created sequence. The inputted arm motions are blended to the corresponding parts of the hip-hop steps. This system also includes such features as the adjustment of the frame rate and the playback speed and smoothing for noise reduction. The utility of the system was evaluated by ten students who were interested in motion creation. 80% answered that the system was simple and able to create motion data easier than CG production software. This system easily created 3D animation at low cost.

Although there are many sliding door paintings in Japanese temples, it is difficult to display all of them in a conventional museum. In this study, I developed a walk-through system for a temple using HMD and Leap Motion to support museum exhibitions adopting interactive techniques. Rooms that cannot be seen in an actual museum exhibition are represented by 3DCG. This study proposes a virtual experience as if they were in the actual place. I reproduced certain sliding doors of the temple in 3D space. Users can walk through the 3D space while looking around wearing HMD. In this system, the viewpoint in 3D space is changed automatically to fit the head direction of the HMD user moving forward. Three types of interaction by gesture recognition are used in Leap Motion. Based on the difference in the number of fingers, the camera moves forward or backward and stops. In the state where a hand is recognized, the user can open and close the sliding doors by swiping hand, ignite the candle by touching it, and rearrange the sliding doors. Evaluation experiments are conducted to verify the system's operability and effectiveness and the reproduction of rooms by 3DCG. Most users answered a post-experimental survey that 3DCG was faithfully reproduced.