Avatars

Avatars are representations of a user’s presence in XR. User avatars represent each user’s position in the shared virtual space. Each user’s voice realistically emanates from their avatar’s position. The head and hand movements of users are tracked and perceived by other users as the gestures of the remote user’s avatar, enabling a level of non-verbal communication that is completely absent from audio calls, while requiring only a fraction of the network bandwidth of a live video call. 

Depending on the device you are using to engage in XR collaboration, your avatar may have additional details of presence. For example, if you are using an HMD that provides eye tracking technology, this can translate to your avatar’s eyes moving their gaze in sync with your own eyes. From another user’s point of view this can increase the sense of attention they are receiving, and it ultimately leads to a higher sense of social engagement. Note that in most systems, the user's facial expression is typically not faithfully replicated, but this will change as face-tracking functionality becomes available in VR HMDs.

Most XR collaboration platforms do not render full body avatars – you will usually see characters from the waist up. This is because lower body tracking isn’t a typical feature of the HMD devices. If you do see a full body avatar, it is not based on your actual body but instead simply an arbitrary model positioned based on where your head and hands are. This will change in the future and some platforms even support the use of full body tracking suits or accessories to increase the accuracy of your avatar’s representation. For example, with such tracking enabled if you were to crouch in the real world, your avatar would also change to crouching position. 

Avatar Realism

Avatar realism is one of the main factors that affects interpersonal interactions and co-presence in XR. This can be considered by both the faithfulness of the avatar’s representation of the actual person, and behavioral realism (realistic and natural physical behavior, e.g. blinks). 


Generally, users react positively to being able to see their virtual bodies, but many platforms do not offer highly-realistic avatars. Realism can range from cartoon style to semi-realistic to photorealistic:

Cartoon Style

Semi-Realistic

Photorealistic

Cartoon style avatars might not appeal to business users. It’s become more common in many platforms to allow the user to create avatars that are highly personal and realistic - even using your actual face photo or full body image. In a few tools, avatars are extremely sophisticated. In fact, there is a vibrant marketplace for avatar designs. Getting the best results requires some learning to build expertise in designing your avatar. 

Avatar Creation

There are typically three ways that you can create an avatar with different levels of realism.

Template-Based Avatar Creation

Example of Bitmoji (part of SnapChat)

Many XR collaboration platforms use pre-created templates for enabling the creation of a personal avatar. Avatars created using these approaches can range widely in terms of their realism and design - from highly cartoonish and overly simply designs to super realistic characters. In many you can even choose between a human and fantasy design such as robots, monsters, etc. The range of customization here typically focuses on color, style, and size, but also often supports facial details, accessories, and eyewear. These are typically not full body avatars.


Selfie- or Photo-Based Avatar

A fast-growing approach to avatar creation is using a mobile phone to capture a selfie picture and then applying computer vision and machine learning to process that image into an avatar. Some applications that do this will attempt to directly cut away and map your facial image onto a 3D face model so it maintains a level of high realism. Others will apply a more cartoon style rendering by attempting to compute a 3D model of a face which highly resembles the photo but is not directly using the photo as a texture.

XR collaboration tools which support this type of avatar creation usually come with a separate mobile application for that purpose. Such applications also often double up as tools for managing your account, getting support, and access to tutorials or help.

Once you have created your initial avatar, you can customize the avatar details and characteristics for further personalization. Take advantage of the always neat haircut and facial hair styles!

Full-Body Avatar

Example of DOOB - from doob group AG

A highly realistic full-body avatar can be created by performing a full-body scan via smartphone or external cameras (usually this will be a professional solution). Once such a scan is processed, you will have an animatable 3D model of yourself. Such avatar 3D models contain volumetric 3D mesh, skeletal rigging for motion, and real textures based on your image. Full body avatars can be put into a wide range of XR experiences, but only a few XR collaboration tools currently support them. One major reason is performance – such avatars are far more complex to render and require better network performance to create a consistent experience when there are a lot of users. The current limits of scanning technology, network performance, and rendering requirements mean that there is often a trade-off between lag and detail/resolution levels. 

Full-body realism is still nascent, but as the technology quickly develops, it presents an opportunity. Emotional expression conveyed through body language and facial expressions is actually a more important factor than photorealism in producing fully immersive interactions. 


“As the technology gets more sophisticated, VR business calls will usher in more nuanced, nonverbal communication that is lost in today’s video conferences. Research shows that approximately 65%-75% of all communication is nonverbal in nature.”   
- Jeremy Bailenson, Director, Virtual Human Interaction Lab, Stanford University

Real-Time Photorealistic Face Tracking

Example image courtesy of Facebook Reality Labs

Researchers are examining how to use a combination of eye tracking and face tracking (through inward facing sensors / cameras) to create real-time photorealistic avatars. This can enable accurate recreation of a user’s facial movement and appearance, which – along with full-body tracking – can be used to transfer emotion and body language to others in a collaboration space. Please read the “Importance of Presence” section for more details.

To learn more about avatar creation tools and what advances are occurring to create higher avatar realism, please read this report.