The recent surge of new VR and AR displays has rekindled interest in 3D displays. However, most 3D displays require wearing special glasses and do not accurately reproduce depth cues as seen in the real world. It has long been a dream to build a 3D display where the viewer sees objects in 3D with correct depth cues and without wearing special glasses. The ultimate manifestation of this vision might be a holographic display where the 3D objects float in mid air, where a viewer can touch and manipulate them.

At the SIGGRAPH 2017 Emerging Technologies venue, we demoed a display that generates the sensation of seeing and touching a 3D virtual object that looks like a hologram. The viewer does not wear any glasses or special head gear. The display provides accurate depth cues, so it feels very natural and comfortable to view. The following two videos show the system in action and the experience of touching the virtual objects.

Here are some images of the crowds at SIGGRAPH 2017 that viewed our display demo:

The viewer sees a 3D virtual object that floats in mid air, in front of the display glass. This is different than Pepper’s ghost displays where the content is seen behind the display glass. The object looks like a hologram, although it is not (see next section for an explanation of how the display works). Within a restricted viewing volume, the viewer can move her head left and right, up and down, and forward and back, and she still sees the virtual object at the same location in space. This is a true 3D image where all depth cues are presented correctly, so our display does not cause eyestrain from mismatched depth cues, unlike stereo displays.

The viewer can extend her hand so that her fingers enter inside the virtual 3D object. As she does this, the rendering adjusts to provide a cutaway view of the 3D object so that her fingers correctly occlude the remaining portion of the virtual object. She also feels tactile feedback on her fingers. This generates the sensation of “touching” the virtual object, although of course this is not the same as feeling the hard surface of a real physical object.

The words, pictures and videos on this webpage cannot accurately convey the experience of seeing this display with your own eyes. If you didn’t see this at SIGGRAPH 2017, you’ll have to take my word that this is a compelling display that generates a different perception and sensation than typical stereo 3D displays.

We solicited feedback from attendees, and one of the most common responses was an expression of not liking head-worn displays. They preferred to see 3D content directly, without wearing near-eye displays, which this display system made possible.

This display provides the experience of viewing a 3D object that looks like a hologram, but this is not a holographic display. Instead, it is an optically reimaged volumetric display. The diagram below illustrates how the system works. Several angled, flat planes are rotated around at several hundred revolutions per minute. A fast projector is synchronized with the revolving planes. At each instant, the projector illuminates the parts of the plane that intersect the virtual 3D object. This generates a true 3D virtual object at the location of the red sphere. Since the points of light are emitted from the 3D locations where those points are supposed to be in space, all depth cues are correct.

That explains how this display provides true 3D with accurate depth cues, but how does this object appear in front of the display? We achieve that by combining the volumetric display with an optical reimaging glass made of large numbers of tiny mirror elements. These create a real optical image in front of the glass, where the distance of the object behind the glass matches the apparent location of the object in front of the glass.

We track the locations of the viewer's hands and fingers, and use an array of ultrasonic emitters to generate tactile sensations along the fingers as they enter the virtual 3D object.

For safety reasons, the rapidly spinning volumetric display is kept inside a large metal enclosure. The next image shows the size of the display and the location of the virtual 3D object.

This is not a practical display for a mass market product, due to its bulk and reliance on moving parts and expensive components such as the high speed projector. Our intent was to use this to test user experiences prior to the arrival of more practical display technologies that accomplish the same effect.

We built a custom swept-volume volumetric display, with a custom high speed projector synchronized to the rotating planes. The optical reimaging used an imaging glass made by Asukanet. We used a Leap motion sensor to track the viewer's hands, and an ultrasonic array from Ultrahaptics provided tactile feedback.

Seth Hunter built this system, with the help of Misty West and Hakki Refai. My role was champion and director.

There is a paper describing the system:

Hunter, Seth, Ron Azuma, Jonathan Moisant-Thompson, Dave MacLeod, Derek Disanjh. Mid-Air Interaction with a 3D Aerial Display. Proceedings of SIGGRAPH 2017 Emerging Technology Installation (Los Angeles, 30 July - 3 August 2017), 2 pages. PDF of the paper