The Goal: capture 3D data in real time, transmit this data and reproduce it in a holographic imaging device.
Watch the behind the scenes video here
Holograms Are Cool
Working in 3D is already fun, but having a display where you can address pixels in 3D space is even more fun. The Looking Glass display prototype provides stereoscopic view of a 3D object or scene with novel vantage points - although it doesn't capture well on camera (i.e. from single vantage point) - trust me, it's cool!
Designing a System to Transmit 3D Data Over the Web
Several folks have already connected depth sensing cameras to holographic displays - but in today's climate of remote work, remote meetings, etc. I wanted to create a system where you could capture in one place and reproduce it in another.
For this reason, the compression and transmission of 3D data became a large focus of this experiment. The unique broadcast protocol over UDP was then easily picked up and interpretted by a custom Unity application to be reproduced as a hologram
We're Going To Need a Bigger Computer
A bottle neck was encountered which prevented higher resolution (and color!) transmission of 3D data. This was both on the transmit side (old PC) and the receiving side (Unity application needs to be optimized and multithreaded!).
As a result - if I return to this experiment - I think my focus would be on improving the fidelity of the holographic image while maintaining a smooth transmission rate.
The Goal: create a cool animated and interactive 3D element for the web site in Unity. (located here)
Epic Lighting. Without The Effort!
First off: Unity allows you to bring some pretty awesome visual effects that would otherwise be difficult to program such as lighting, shadows and material effects.
Yes, I remember the days of having to generate and draw polygonal geometry that looked like a "shadow" in openGL. and flash. ouch, just dated myself.
The Animation Engine Experiment
I created an animation engine which provides all real time motion.
the cool parts:
- it is based upon Penner's equations and associated math to allow for a wide range of motion profiles
- it adjusts animation timing and frame duration in real time based upon local performance (frames per second). i.e. If you've got a super grahpics workstation, you see the same stuff, just smoother and spread over more frames.
I was surprised to find that the application quickly crashed my mobile device (Pixel 2). My hope was that content generated in Unity and published for web would inherently be compatible with any/all devices :(
okay warning label posted.