From Star Trek’s Holodeck to Iron Man’s helmet, science fiction has set the bar high for real-time 3D holographic images. But while we are still pretty far from that, we are getting closer.
To appreciate the potential of 3D, think about how we get information about the world around us. The visual is very important: about 90 per cent of human input comes through our vision. But we don’t just look at information. We also want to interact with our surroundings. Yet often this need for interaction is limiting.
Take computers. Before touchscreens, you needed to use a keyboard and a mouse, but to use a keyboard, you need training. Smartphones took off because of their touchscreens – a truly intuitive technology.
We interact with the real world in 3D, so why should we not be able to interact with a 3D holographic image in the same way? The goal is to produce a 3D holographic image that you can touch, hold and interact with. An image you can see in front of glass, rather than behind glass. One that would react to you and respond in real time. How much more would that augment your experience?
That’s what we are working towards. One big problem is finding a way to make it more comfortable to view 3D images. Our eyes perceive depth images in two ways. They focus, using the eye muscle to see as clearly as possible, and they converge – they rotate towards each other. Focus and convergence happen at the same distance. But the current 3D display makes us focus on the screen while the 3D effect forces us to converge on the 3D object in a different place. Our brains can’t cope with focusing and converging at different distances, which causes headaches and dizziness.
We need a holographic display to solve such a visual conflict for comfortable 3D viewing experience. But we don’t yet have the computational power to provide the incredibly fast calculations required to generate large holograms. A high-definition TV, for example, needs three gigabits of computational power per second. For 3D images, we need around three terabits per second – or, to put it another way, 3,072 gigabits per second.
To overcome this challenge, we have created a method of ‘slicing’ images, which solves both these problems. When you watch a film, the pictures are moving but they are not continuous movement – they are frame by frame. Our technique works like that. We take a 3D object, slice it into layers, and stack the layers together. This method reduces the data amount needed by a hundredfold. The images we create can be viewed from any angle, in space, with depth. You don’t see the layers. It’s an illusion, just like a film: you see it as a continuous image. And it can be manipulated in real time, using a mouse. We don’t have a better way right now, but we hope that will come.
The next issue – which we haven’t yet solved – is to add the experience of touch to the holographic image. That will be a long journey but it will be a lot of fun.
So much for the Holodeck. One day, we also hope to catch up with Iron Man and his helmet viewer, which gives the wearer constant real-time information about their surroundings. Google Glass tried it but that was doomed to failure, too: it projected an image through the sides of the glass which was then presented in front of you. You had to focus on the image to see it clearly, then relax and return to normal. The disruption clearly wasn’t practical.
We have now created an augmented reality head mounted display where the image is always clear, no matter how you are looking at it. Again, it uses a different approach to get round the problem of dizziness and disorientation. Our headset sends a single pixel beam to the eye and scans it to form an image on the retina. So you only have to focus on a single beam rather than an image itself.
But while it is wonderful to be inspired by science fiction, our work has many real-life applications: in medical imaging, operating theatres, construction, outdoor sport, training and data manipulation, to name a few. We may not be at the Holodeck yet – but we are getting there.
One day, we also hope to catch up with 'Iron Man' and his helmet viewer, which gives the wearer constant real-time information about their surroundings.
Professor Daping Chu
This article first appeared in CAM - the Cambridge Alumni Magazine, issue 92.