Dolby.io and 3D Hologram Calling
Growing up in the late 90’s/early 2000’s, I experienced what it was like to be an immigrant in a new country and live far away from family. I remember the nightly routine of land-line phone calls. And for this reason, exploring new and better ways to communicate has become one of my passions. In this experiment I use the Looking Glass Portrait, a holographic light-field display, and Dolby.io to create a unique calling experiment.
Being self taught, I have benefitted greatly from inclusive community hackathons that welcome diverse creators. Overall this experience really pushed me to build my programming skills and as a result, I received an award! 🎉
The following are four use cases for hologram video calls.
Telemedicine
While two-way holographic communication can help bring people closer together, it can also help create better outcomes for people who receive specialized care across the internet.
With telemedicine now widely embraced, we can strive to have the most immersive, engaging, and informative visit possible. By showing 3D models of health related visual aides, doctors can make remote visits much more compelling to create better health outcomes. Web3 makes this even more compelling with the ability to safely verify your identity and easily provide health data on a browser.
Education
Engagement with a 3D avatar or communicating with a professor while being shown a 3D object in realtime can help motivate and improve comprehension of subjects that requires visualization of 3D structures.
3D Presentations
A presentation where 3D depth adds to the experience can be done, adding to the level of comprehension and retention by the message receiver. High quality audio and 3D video is the key.
Portals for Fans
Access to exclusive interactions with artists and celebrities could be a utility of an NFT token. By authenticating someone’s online identity through a browser, we could see special hologram presentations given to small groups of fans. One of the problems with many XR experiences is less than optimal sound quality. This presents another reason this experiment interested me.
Now with a few use cases, let’s take a look at the prototype that was built.
How it works
The first option allows users to video chat in 3D across a browser using a webcam.
In a later experiment I would like to try using a depth sensing or stereoscopic camera.
Using a version of holoplay.js and dolby.io, we were able to map the live video stream and audio to 3D video textures in the Looking Glass Portrait for a very unique opportunity to chat with someone in real-time in 3D without a headset.
The second desktop app includes a video player and simple UI for starting chat, stopping chat, playing and pausing video. The app was easily deployed with netlify.
Technologies Used
- (2) Looking Glass Portraits + Computer with webcam
- HoloPlay Service and HoloPlay.js
- Dolby.io
- Visual Studio Code
- Netlify
Challenges
Being self-taught without a CS background, programming has always been a great challenge for me. Thankfully the dolby.io team provided a helpful starter templates and tutorials in order to get my app up and running with Netflify.
What I learned
Be patient and go for small wins to lead into bigger feature implementations.
What’s next for Looking Glass Dolby MVP
For the desktop app as well as the hologram app, it would be great explore unique capabilities of dolby.io for these different use cases. This was merely an experiment to get it running but I do believe high quality audio should be a standard for holographic phone calls. It will also be exciting to continue exploring avatar conversations. With so much excitement around NFTs, it might also be fun to use profile pic avatars as masks.
