LiveHumanVR: Reinventing Presence

By Patrice Bonfy on February 10, 2017

Media & EntertainmentVirtual RealityLiveHumanVRMission


Virtual Reality is the future! But when is the future?

Virtual Reality is Booming, in Slow Motion

A Very Promising Media

The best experiment to convince yourself of the potential of Virtual Reality is to have other people, preferably not too tech-savvy, try a headset. They look around as if they had just regained sight. Their body language adapt to the experience they visualize. They are “inside.” This immersive power makes of Virtual Reality a robust and versatile storytelling media.

2016 was expected to be a cornerstone for consumer Virtual Reality adoption. Indeed, 5 million Samsung Gear VR were shipped, a few million premium headset (Oculus Rift, PS VR, and HTC Vive) were sold, and millions or dozens of millions of cheap Google Cardboard and alike headsets found a home. A conservative estimate gives at least 10 million VR equipped users.

The Chicken and Egg Problem

It seems however that these equipped users spend too little time using their headset. Not because of physical or social discomfort, but because of the chicken and egg problem described by Mark Zuckerberg for “his” Oculus platform. There are too few quality contents available to drive adoption. The low adoption makes investing in creating costly VR contents a risky bet, which leads back to too few quality contents available.

Leaders of the VR industry activate various strategies to come out of this vicious circle and allow the emergence of a “killer app.” The first one is investment. Facebook, HTC, and Google finance VR contents creation through internal studios and dedicated venture funds. Priming the pump may not be enough to develop a healthy ecosystem. The second one is the quest for an in-house killer app, which has lead to the introduction of several VR tools such as Tilt Brush, Oculus Rooms, user generated 360 photos and videos. Those are great innovations, but neither combine what we think a VR killer app needs: accessibility without costly equipment, usability without creative or technical talent, and a real demonstration of the immersive power of Virtual Reality.

Set The Creation and Experience of Immersive VR Contents Free

A digital expression form gets massively adopted when anyone with something to tell can use it, without strong technical skills nor expensive equipment. Reading articles on the internet was popularized by blog platforms, not by the traditional press. Online video consumption owes more to Youtube users than to TV and movie producers.

The success of Snapchat — and Instagram — “Stories” as well as of Facebook Live Videos demonstrates the tremendous public interest in contents created and published on the go, as they generate unmatched proximity with their creator.

We believe that it is possible right now to not only attract more regular users of Virtual Reality but also to create Virtual Reality experiences that are deeply human. All we have to do is let anyone, anonymous or star, artist or editorial author, become “present” for an audience in immersive virtual experiences, even with the cheapest headsets. This is the double experience of LiveHumanVR.

A good Metaverse first needs to be filled with real people.

LiveHumanVR Experiences

I am a LiveSpectator

It’s almost 20:30. I put my VR headset on because I have a meeting with Lady Gaga. 20:34 — she’s often a bit late — the app informs me that I’m about to enter her LiveSpace. It’s downloading. The environment is quite simple this week. I’m sitting in an old jazz club.

Some kind of glitter floats in the room and adds charm to the scene. A shiny particle of dust virtually tickles my nose. I’d almost forget that Lady Gaga is sitting in front of me at a piano. She looks at me. “Hey, LiveSpectator! Tonight I’d like to start with a whole new song. Then, as usual, we’ll decide together which songs to sing along.” By summoning the navigation menu, I can see that there are 32 455 other LiveSpectators, but it really feels as if Lady Gaga was performing exclusively for me.

20:56, I’m singing Poker Face along with Lady Gaga and the 5 478 other LiveSpectators who opened their mike. I feel a pat on my shoulder. My roommate is home, he politely suggests that I move the massacre to my bedroom. Without the chorus effect, I can’t deny that my performance isn’t as inspiring as it felt. The meeting was almost over anyway, let’s have dinner. Before going to bed, I discover the LiveShow of a historian who tells anecdotes of the antiquity in the 3D reconstitution of a Roman temple. Interesting, but I’m exhausted. I add him to my favorites to catch his replays and next meetings. Tomorrow, I’m meeting with Mark Zuckerberg. He hosts LiveSpectators every month to answer questions from the community. One of the issues I submitted is in the top 10 and will probably be on the agenda.

I am a LiveShowrunner

My presentation is ready. I upload it in the application for tonight LiveShow. I selected my LiveSpace among default options. It’s a conference room with very pure design, positively Scandinavian, with very high white walls merging in the “sky.” My very own cathedral for design. I’m a design teacher and the LiveShowrunner of Designow. My students talked me into using this application to broadcast my witty comments on design latest trends. At first, it was just another way to connect with them, but my Christmas show was a hit in replay. Since then, I have a few dozens of LiveSpectators at my weekly shows and several new collaboration requests.

Almost time. I install my foldable green screen. I put my webcam in position. I switch on the spotlight that I recently installed to improve the quality of my projection. I stand in front of my webcam, check the framing n my smartphone. All set, 23 persons are already waiting for me. I click on “Start LiveShow.”

The show went well. Some interesting questions that allowed me to further dig into the subjects. I’m going to go through the slides again to check the public reactions and answer questions that I may have missed. I’ve been invited tomorrow for a meeting “in real life” with other LiveShowrunners from London. There will be other teachers and speakers, but also comedians, singers, and sports commentators. They told us to expect a surprise. Word is Daft Punk would be planning a giant LiveShow, and wish to discuss it with experts…

Daft Punk in my VR Headset? Now we’re talking.

Let’s do it? What to prototype?

Isn’t everyone already working on this?

It is possible of course that a studio somewhere is working on developing something like that. But this angle on VR is clearly not the most common. Most public experiences, published or in production, focus whether on creating full virtual experiences or on capturing VR record of real events or places.

The mix of virtuality and reality that we envision for LiveHumanVR could be confused with Augmented Reality. It’s actually the opposite. With LiveHumanVR, the real element (the human) is integrated into a virtual environment whereas with Augmented Reality virtual elements are incorporated into real environments.

Oculus Rooms from Facebook is a great experience of presence in Virtual Reality. But it aims to tackle reciprocal presence where LiveHumanVR focuses on unilateral presence. In the current state of technology, reciprocity requires digital reconstruction of protagonists. The artistic choice of cartoon-like avatars demonstrates that realistic reconstruction is not possible in a scalable way.

So you say that technology is not ready?

That is probably the conviction of VR and AR industry leaders. We think that they have a perfectionist approach of human projection in Virtual Reality. They work on capturing and projecting people in 3D already. Which is indeed not a viable short-term plan for LiveHumanVR, for three reasons. First, it is a complex task to produce a 3D projection while conserving the natural “defaults,” which leads to more of a “video game character meeting” than “human presence” effect. Moreover, a full 3D reconstruction would at least require multiples cameras, meaning that LiveShowrunner would need to acquire costly equipment. Finally, transmitting and receiving live 3D projections would necessitate a high amount of bandwidth, thus adding a technical barrier for showrunners and spectators.

Which brings us to the keystone of LiveHumanVR concept: a relatively low-tech unilateral projection that makes the experience plausible without any technical breakthrough, just by combining existing technologies.

The cut-out human in 2D video

In LiveHumanVR, The LiveSpace is a 3D environment, selected or created by the LiveShowrunner and downloaded once by the LiveSpectator at the beginning of the LiveShow. LiveSpectators can look around and perceive distance to the elements composing the set, but they can not move around the LiveSpace. This prevents them from realizing that the LiveShowrunner that they believe is standing in front of them is actually a well positioned flat video of the cut-out performer.

Convincingly realistic human projections become “easy” to capture, cut out, and broadcast live, along with activation triggers of animations pre-loaded in the LiveSpace.

Think of cardboard cut-outs and the way they can trick you from the right angle. Now imagine them moving and talking.

Prototyping the “presence” feeling

Before going further, we need to validate that an “animated human cardboard cut-out” effectively tricks the spectators’ brains into feeling that the performer is here with them, significantly more than a traditional video. This can be tested by incorporating a transparent animated gif of a cut-out human in WebVR 3D environment. Not too hard.

You can contribute to the project if…

  • You work at Oculus, or another developer of Virtual Reality Platform and/or hardware (Sketchfab?) and have comments on our market analysis, or ideas to help us move forward.
  • You develop virtual environment from dawn to dusk and wish to help to build the prototype or more.
  • You are well versed in live video and have comments or advice on broadcasting and embedding a live cut-out video.
  • You project yourself as a LiveSpectator, or LiveShowrunner and have feedbacks for us.

Leave a comment and contact us!

TLDR*: Virtual Reality needs its “Killer App.” With LiveHumanVR, anyone can start his/her own live VR show and be “met” by his/her audience inside an immersive virtual world. All this with relatively standard technological equipment, for performers as well as for spectators.

*Too Long, Didn’t Read