Motion Laboratory

Tablets, which can be operated by touching their screens with fingers in multi-touch systems, are widely being used due to their high portability. In this study, I developed a system for simulating the layout of furniture by freely placing it in 3D space using a tablet. I provide a practical and easy to use system to help people consider the arrangement of their furniture. The system creates a virtual room by arranging furniture from a furniture list after determining the size of the room. Users can place the furniture by a touch operation, which corresponds to such operations as furniture placement. To align the furniture easily, the room is divided into invisible squares along which the furniture is placed. After placing the furniture, users can delete it, change it by a tap, or move it by a drag operation. The system has 25 types of furniture including different size, and a wide variety of indoor spaces can be reproduced. Ten students used the system and evaluated its utility, usability, and problems. The results identified room for improvement in operability and functionality.

Shomyo is sutra with melody, and it is sung by monks in Buddhist temples. In this study, I have proposed a visualization system for the Shomyo scale based on voice input to check the scale of an inputted voice. The purpose of this research is to arouse the interest of people who are unfamilar with Shomyo. When the user inputs a voice with a microphone, the pitch is converted into a scale, and then notation of the Shomyo score corresponding to the scale is displayed in the lower-left area of the score image. Moreover, it is possible to visualize the level of sound by moving a sphere and an oscilloscope. The system is intended for Tendai Shomyo, Meyasu Hakase, and the scale of the Shomyo called Ritsusen, and it recognizes the basic five sounds of Kyu, Sho, Kaku, Chi, and U corresponding to D, E, G, A, and C in the Western notation. I conducted a questionnaire study to evaluate the system. As a result, this system obtained high evaluations in operability, ability to sensuously understand the high and low ends of the sound, and understanding of Shomyo notation by reference to the Western notation.

In classical ballet, such characteristics as pelvic obliquity and the state of the angle of the hip joints are critical. However it is difficult for learners to imagine them because they are unseen. We have developed a system that simplifies the understanding of dance techniques for learners by visualizing human characteristics with CG. Characteristics are visualized in many methods to confirm the optimum visualization method for each characteristic. The system calculates pelvic obliquity, the hip joint angle, and the center of gravity per frame using the 3-dimensional coordinate data of markers captured by an optical motion capture system and visualizes them as CG objects. Pelvic obliquity is shown as a plane object composed of right and left hip markers and a lumbar marker. The angles of the right knee, the left knee, both knees, and the tiptoes are calculated and shown as 3D/2D fan-shaped objects, bar, and numerical values. The center of gravity is calculated by synthesizing the partial center of gravity of 15 divided human body parts calculated by their lengths. To evaluate an optimum method to visualize the characteristics, 10 examinees used our proposed system. As the results of questionnaires on the understandability of the plural visualization method, we confirmed that a 2D fan-shaped object is the optimum method to understand the angles.

In this study, I have developed a system which operates a CG character by voice to original support and interesting dance. Human motions are captured by a motion capture system and played with CG character, the users can make dance movement by putting those motions together. The dance motion corresponding to the specific sentence is assigned and user can change the motion of the CG character by inputting voice from a microphone. Julius is used for recognizing Japanese. The system starts with a Julius server and accepts voice inputting from a microphone. Animation of a CG character is handled by body parts. The variations of dance motions are prepared for each segment such as a head, the right arm, the left arm, both legs foot and it can be merged. I had ten people use this system to evaluate the usefulness of this system. As a result, the interest to a dance lasted, but was the evaluation that there was not despite original support.

Although motion data have been widely used for recent digital content, such data are generally intended only for human body motions. In this study I reproduced the 3DCG animation of clubs for juggling as tool movements with human body movements to support the understanding of the characteristic tool operation of juggling and to practice these actions. First, the motion data of three clubs and a human body were simultaneously acquired by an optical motion capture system. Four kinds of motions were captured with 43 markers for a human body: 34 markers for the human body and three for each club. A human skeleton was made from the human body data and outputted in a BVH format. The positions of the three markers of each club were outputted, and the angels of the clubs were calculated from these markers. The motion data of the human body and the clubs were applied to a CG object for each, and a learning support content was produced to reproduce the motion data of the human body and the clubs simultaneously. To confirm the validity of the CG-reproduced juggling, an evaluation experiment was conducted with ten students. The results suggest that the content was easy to use and watching the movements was also easy. However, other opinions criticized the system's effectiveness and the utility of the learning support element.

Opportunities to consider choreography and the training regimen of dance are increasing because dance is a required subject in compulsory education. This study supports dance learning by developing a system that automatically generates the choreography of hip-hop dance for beginners by combining eight steps in a timeline from 37 steps that were acquired from a professional dancer and displays it with 3DCG animation. A preliminary experiment was conducted by two dance masters to clarify the problems of the algorithm in a previous study. The results of the preliminary experiment found a problem : the difficulties of inversion and repetition are changed by the kind of step. The difficulty increases by continuing rotary steps or the frequent appearance of steps with high physical value. I classified the steps into two kinds to improve the automatic generation algorithm. When a step is repeated, the next step is limited to the same step or a right or left inversion step, which is easy to make. In addition, by calculating the physical value of each step, the system can extract the degree of the physical value by digitizing it and confirm the frequency of its appearance. An experiment verified the effectiveness of the improved algorithm by comparing the proposed and previous algorithms. The result provided approximately 80% effectiveness.

The purpose of this research is to intuitively and easily search for necessary motion data from many motion data. In this research, I developed a system that retrieves motion data including similar human body postures to the user inputs using Kinect. The key poses extracted from the motion data were archived in a database and similar poses to the user input are retrieved by comparing these poses. The user operates this system only by human body movements, and the direction of the body is normalized to recognize human body poses regardless of the body direction. As feature values of human body postures, the distances for each axis between both wrists and both ankles are calculated. The distance of the top and bottom directions from the pelvis to the knees is also calculated. The positional relationship of the wrist and the center of the spine is checked. These feature quantities are described with two or three levels of parameters, and these patterns are used to retrieve poses. The key poses of the motions in accord with the inputted patterns are extracted from the database in real time, and the retrieval history is displayed. The motion, including the selected key pose from them, is played by animation. An experiment was conducted with ten students to evaluate the usefulness of the system. The results obtained by this system are useful for retrieving motion data, and I confirmed that this system easily and intuitively retrieves human body motion data.

There are many historical sliding doors with paintings in temples, and it is difficult to display all of them in museums. In this study, I developed a walk-through system in 3D space using Kinect to support exhibitions at museums. This study displays the whole room that is difficult to show in the actual exhibition of the museum and offers a virtual experience that is vivid. This study reproduces the arrangement of the sliding doors of a temple in a 3D space and expresses the perspective of a person who is virtually walking using Kinect in 3D space. This system uses the position data of the head, the feet, the hands, the shoulders, and the spine that are acquired by a Kinect and recognizes five poses and three motions. Poses are recognized by comparing the position of two joints. Motions are recognized by one initial pose and the difference between the current and previous poses. Depending on the recognition results, the user can change the perspective, open or close the sliding doors, roll up the bamboo blinds, and light or extinguish a candle in 3D space. Users can control the perspectives of the camera anytime and such interactions as opening or shutting the sliding doors when they enter specific areas. An experiment evaluated whether the temple rooms were reproduced by 3DCG; it also evaluated the system's operability and availability. The results showed that approximately 90% of the users answered that the system faithfully presented the rooms. In addition, for the operability of the walk-through, using this system was confirmed to be easy. This system offers a vivid and realistic virtual experience.

This paper describes a system that supports the creation of contemporary dance choreography using 3D motion data acquired by motion capture. In this study, we developed a system that automatically generates short choreographies by combining basic motions with multiple body-part motions. The generated choreographies are simulated in 3D animation. This system runs on a tablet computer, so user can select a basic motion and body-part categories by a touch operation. To increase the variety of generated choreographies, the whole body is segmented into six parts, and the synthesis timing is adjusted. The generation of unnatural motions is prevented by constraints, which include the interruption and determination of the motion synthesis as well as the adjustment of the synthetic timing with the ground state of the feet, the arms, and the torso. The system also has such functions as changing the viewpoint, changing the playback speed, and saving the created choreographies. To evaluate the effectiveness of the system, an experiment was conducted with eight subjects who are majored in dance at university. The subject created short dance pieces using the system and demonstrated the created choreographies. From questionnaire results gathered after the demonstration, we received many comments were received about the effectiveness to create new movements and to support dance training. Therefore, our system was confirmed to be effective for supporting dance creation.