Meta’s Reality Labs research team has published a study on avatar dress-up simulation in virtual space. This research is related to the technology “Codec Avatars” that automatically generates realistic avatars, and is one of the research results of the team.
Make your avatar dress up more realistic
Latest research “Dressing Avatars:Deep Photorealistic Appearance for Physically Simulated ClothingIntroduces a technique for generating realistic clothing animations on realistic 3D avatars. Virtual clothes can be worn by avatars of various body shapes, and clothes can be moved and resized between different avatars.
Until now, CG has been able to create avatars with clothes that are so realistic that they can be mistaken for the real thing in movies. However, it takes a long time to create such a realistic avatar. Especially in virtual space, it is difficult to realize the shape and movement of clothes that are comparable to reality in real time.
In this research, we simulate the movement of clothes and use machine learning to realize realistic clothes-wearing avatars. Therefore, it is possible to create an avatar dressed in clothes that looks more realistic in a shorter time than before.
However, there seems to be a long way to go before it can be achieved with Meta’s existing VR headsets and devices. This method uses NVIDIA’s “RTX 3090” and operates at regarding 13 frames / second, so it is difficult to say that it is “operating in real time”. He also said, “It is difficult to handle clothes with large loose folds such as kimono and clothes with multiple layers such as Junihitoe.”
“Codec Avatars” to generate photorealistic avatars
「Codec AvatarsIs a Meta research project to generate photorealistic avatars that operate in real time using input information from the Headmount Capture System (HMC). The HMC is equipped with a camera, accelerometer, gyroscope, magnetic sensor, and microphone.
The Codec Avatars research team has created a version that uses only eye tracking and microphone input to generate realistic avatars, a 2.0 version (high-quality version) that generates more realistic avatars, and an iPhone series equipped with “Face ID”. We have published various new methods such as a version generated only by.
News regarding Codec Avatars is also covered in the following articles:
(reference)arXiv、Meta、UploadVR
Mogura VR is a partner media for Upload VR.
