Motion Laboratory

Etoki is an approach to explaining pictures drawn on curtains. I have archived Etoki with digital technologies and have developed a virtual Etoki system that allows the user to freely select a scene in 3D space. This system uses TVML and can simultaneously reproduce a scenario, pictures of Etoki, CG animations, explanations using sound, and subtitles. The system expands and displays the part of the picture corresponding to the selected scene. Furthermore, a CG character points to the picture with a virtual wand and explains the story. The digital content of "The Kumano kanshin jukkai mandara," Japanese artwork used for a demonstration at the Ryukoku Museum, was created with this system.

Recently, there has been a dramatic increase in Web contents created to display high-resolution imagery and 3DCG of cultural properties. Tablet devices are now widely used, and technologies for displaying them on a Web browser have attracted much attention. Such systems include, for example, Web3D and WebGL. This study uses WebGL, which is a standard specification for displaying interactive 3DCG on a Web browser without any plug-in. This study proposes an interface that is suitable for browsing digital archives of cultural properties. Users can browse a 3D object from various angles. Moreover, commentary on the picture in view is displayed on another page when parts of the 3D object are clicked. Content can be easily updated by simply replacing the 3D data and text data. I have created 3D content that is intended to animate images surrounding a casket using the proposed interface and have examined the capabilities of WebGL.

The purpose of this research is to clarify the choreographic algorithm applicable to classical ballet solo dances. The relationship between music and dance in classical ballet solo dances is analyzed by two methods. First, motion data and the corresponding music data of classical ballet are analyzed as time series data. Physical feature values are calculated from motion data by using translation information: the angle of the upper body and limbs, the acceleration of head top and limbs, and the distance of limbs. Musical feature values are calculated from MIDI data with pitch for each tone and measure. Motion and music are synchronized using the calculated time difference and analyzed with graphs. Expansion and contraction of the limbs related to pitch and the use of posture are discovered. Second, in order to define the music pattern, the chord progression of each beat is confirmed. Motion patterns are also defined using the graphs of the first analysis, and then motion and music patterns are compared. As a result, by comparing these data the difference between the progression of patterns in motion and that in music was found.

The purpose of this study is to give visitors a more interesting exhibit. I provide digitalized content for exhibitions with additional visual information. I expressed one of the dancers drawn on a casket by using 3DCG animation and stereoscopy. First, I researched the eastern dance style related to Buddhist culture based on the dancers drawn on the side of casket. The motion data of dance were captured from a performance based on movies of eastern dance. Then 3DCG animation was created by using these motion data. Moreover, a stereoscopy image that displays the casket with CG animation was created.

The purpose of this research is to clarify the elements that are necessary for acquiring the ballet skill. This study gives attention to the motion of arabesque, a balance motion made by standing on tiptoe, and analyzes the difference between success and failure in achieving this motion. The motion data and the floor reaction force data of the arabesque were acquired from one experienced female ballet dancer five times on the same day. First, joint positions were calculated by using the captured positions of two or more markers. Next, the angles of Torso, hip joint, and right ankle, which show the features of the arabesque, were calculated by using the calculated joint positions. These angles were compared with the corresponding floor reaction forces. As a result, the difference between successful arabesque and failed arabesque was discovered to originate in the Torso by the feature values and a generated graph.

Tablet PCs are becoming more popular all over the world with their multi-touch input system, light-weight design, and portability. They can be operated using the fingers, without any device such as a mouse. In this paper, I have developed a simulation system called "3DCG interior designer" for use with an iPad. The purpose of this system is to achieve efficient use of rooms for those who want an interior makeover. This system simulates a living space by arranging 3D furniture and human figures with motions in a virtual room. The user sets the layout of the virtual room and then arranges 3D objects of furniture in any location. The user can simulate the living space with CG character animations such as walking. In addition, furniture is moved by dragging-and-dropping the object, and the camera is operated with joysticks at both ends of the application's screen area. The 3D furniture objects can be automatically placed on the floor. I gave a questionnaire survey to 11 students asking them to evaluate the usefulness of the system. The results confirmed that this system is useful for people who want to rearrange rooms.

Recently, the incidence of obesity in young people and their rates of other lifestyle-related diseases are increasing due to lack of exercise. The goal of this research is to change the health consciousness of young people by introducing fun exercises. In this research, I propose an exercise-promotion game using multiple acceleration sensors and two-dimensional CG. Users attach the acceleration sensors to both ankles and play exercises such as walking. Users can control an avatar in two-dimensional space by the acquired acceleration data. As defined exercises, foot stepping, shaking, jumping, and kicking are available. Game music and game scoring were prepared to help users enjoy the exercises. I evaluated whether the user could actually enjoy the exercise and could comfortably operate the avatar. As a result, users reported opinions that they could enjoy the exercises and that they could comfortably control the operations.

Motion data have been widely used for recent digital contents, but most of such data intended for dance teaching only use human body motions. Therefore, in this study I reproduce 3DCG animation of an athletic hoop, that is, the equipment used in rhythmic gymnastics. Motion data and a physical model are used to reproduce CG animation of the hoop's motions. The basic motions of a hoop were captured with four markers by an optical motion capture system. I calculated the center position and rotation of the hoop using the formula of dot product and the outer product from the position coordinate of the markers. In addition, the throwing motions used with the hoop are calculated by assigning the initial velocity and the throwing angle based on the physical model. I tried to reproduce CG animation of the hoop with human body motion. The animation is described in VRML, and it can be seen from various angles in 3D space. I had ten students evaluate the simulation results. As a result, they expressed opinions such as "I could understand the movement of the hoop", and "I was interested in it."

This paper describes a motion synthesis system for supporting dance creation with a tablet computer and motion data. The system allows users to select each motion clip and preview it by 3DCG in real time. Body-part motions of contemporary dance by a professional dancer are captured and prepared as motion clips. The system synthesizes motions in two ways. One way is to blend the whole body motion to a base motion, and the other is to replace body parts of the base motion with different motion clips of the corresponding parts. Some motion clips can be selected by touching the tablet screen, and they can be blended to the base motion. The camera view in 3D space is rotated by dragging the screen, and it is scaled by pinching in and pinching out. Ten dancers who have experience in choreography creation evaluated the system with iPads. Dancers created choreographies with the system and then performed the created choreographies. As a result, it was confirmed that the system was useful for helping dancers to create new choreographies and to find unexpected motions.

• Asako Soga, Sakiko Matsumoto, A Motion Synthesis System for Dance using a Tablet, Proc. of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, p.178 (Orlando, USA), Mar. 2013

The purpose of this research is to develop a system using motion data that can be used for education and entertainment related to dance. I have developed a retrieval system for the basic steps of Hip-Hop dance. First, motion data with the position information of 25 points were segmented into units of basic motions. Next, the total translation of each limb (i.e., right leg, left leg, right arm, and left arm) was computed from the translational data. The system retrieves a Hip-Hop step from the step name or the difficulty level, which is inputted by the user through a GUI. For example, if a user searches for a step with a small translation of the leg movements, the system displays the selected results for steps in which the total translation of the legs is small. Results are displayed as a human-body animation by Java3D. I conducted an experiment using ten students who had no experience with Hip-Hop dance in order to evaluate the system's usefulness. As a result, they evaluated the operation of this system as simple, and thus it is useful for helping inexperienced persons to retrieve steps based on the feature values.

The purpose of this research is to support the study of dance by handling motion data. This paper describes research on CG characters that are suitable for learning Hip-Hop dance. The motion data of Hip-Hop dance are expressed by 18 rotational items and 1 translational item. Five CG models were created and outputted in VRML format: stick figure, skeleton figure, sketch figure, humanoid, and robot, which all have 57 DOF. I have developed a system that is able to display these CG characters and to play animation at the same time by using Wire Fusion. Thirty students evaluated CG models and decided which could be suitable for learning Hip-Hop dance and raising interest. As a result, the sketch figure model is the most suitable for learning Hip-Hop dance, and the humanoid model is suitable for raising interest.

When beginner dancers practice Hip-Hop dance, most of them follow the example of expert dancers. However, they cannot learn important factors of Hip-Hop dance such as the movement direction of each body part and the "sense" that is expressed as stylishness. In this research, I have developed a system to visualize learning support elements that cannot be conveyed with motions and words. Motion data from a professional dancer's Hip-Hop dance were captured by a motion capture system. The "sense" is displayed as onomatopoeia. The movement direction of each body part and the trajectory of motion are displayed as CG objects such as arrows, lines, and the sizes of objects. Displaying them around the image of a skeleton, users can see the visualized techniques of Hip-Hop dance. This system allows users to intuitively understand learning support elements of Hip-Hop dance. Moreover, the so-called count, the rhythm of Hip-Hop dance, is visualized so that users can easily understand each motion. The results of an evaluation by ten beginners indicate that the proposed system makes it easy to understand the learning support elements of Hip-Hop dance and to imagine the "sense."

With the development of software for creating 3DCG animation, 3DCG content using dance animations has become widely used. In addition, the behavioral simulation of virtual crowds and the representation of intangible culture has been performed. The goal of this research is to develop useful tools in dance education and creation, such as a self-study system for students and a creation-support system. I have developed a system that allows anyone to create 3D animation of dance groups. The system simulates choreography, formation, and transition of the dance groups using game devices. The system can edit the action of dance groups with a game pad and control them using a Kinect device. The simulation results are displayed by 3DCG animation. The edit system arranges dancers by drawing a line on a virtual stage and simulates dance animations by specifying the number of dancers, the formation pattern, and dancing motion. This system can simulate the created sequences many times by editing a timeline. The control system can change the formation of CG dancers by right-arm gestures and their motions by left-arm gestures in real time. Since the control system recognizes both hands' gestures at the same time, users can control two or more objects using a combination of gestures. I tried to represent a scene featuring dance groups by CG animation using the edit system. As a result, the system could represent a classical ballet piece. I confirmed that the gestures used for operation of the control system were intuitive.

• Asako Soga, Itsuo Yoshida, A Simulation System for Dance Groups using a Gamepad, Proc. of 7th International Conference on Computer Graphics Theory and Applications, pp.365-368 (Rome, Italy), Feb. 2012