AI-Generated 3D Holograms in Real-Time

Looking back to the beginning of this century, we’ve come a long way in terms of our technological development. Who would have thought, back in 1999, that you could access all the information and entertainment you could need from a relatively small mobile phone?

But there is still so much more to look forward to. Holographic technology would seem to be the stuff of science fiction, but recent developments at MIT suggest that it could be much closer to reality than you might have guessed.

On an everyday level, IT workloads are what we often lack in our business operations. When you have on-demand assistance from Computers In The City, an IT support provider in London, you will find your processes to be much more efficient and scalable.

3D holograms

Although consumer markets have been waiting for computer-generated 3D imagery for years, the processing power required for this has been too great, and the resulting imagery too low in quality.

Virtual reality headsets have not been as successful as they could have been due to a variety of adverse physical effects, and leading manufacturers recommend taking regular breaks. Although an illusion of 3D imagery is created in VR, users are still viewing a 2D display in a fixed plane, which falls short of the promise of 3D holograms.

But now, researchers at MIT have discovered a new deep-learning-based process for producing efficiently, and instantly. A team at the Department of Electrical Engineering and Computer Science have developed the process they call “tensor holography”, and have described their findings in a study published in Nature.

Breakthroughs in holography

Where photos reproduce light and colour in a 2D image, holograms also capture the phase of light waves, which enables a depiction of parallax and depth. They were first recorded in the mid-twentieth century when a laser beam was split, partly to illuminate the subject and partly for the phase of the light waves to use as a reference. Through this process, holograms were given a depth not possible in photographs, but they were still static and could not be reproduced.

Computer generated real-time holography

In the new process of computer-generated holography, a simulation technique is used to address these limitations, but this takes an extremely high level of complexity for a supercomputer. Liang Shi, the lead author of the study, explains that “Because each point in the scene has a different depth, you can’t apply the same operations for all of them, that increases the complexity significantly.”

But with an application of deep learning, the computer can generate real-time holography. This involves the use of a convolutional neural network that uses trainable tensors that process visual data in the same way as humans.

Thousands of pairs of computer-generated images were collected in a database, all of which represented a holographic image with information on the depth and color of every pixel. The team combined scenes that have a complexity of varied colors and shapes and an evenly distributed pixel range from the background to foreground.

The tensor network took the data learned from each image pair to improve its own ability to create holograms, which became faster than any calculations based on physics. The result is that the new process of tensor holography can use images and additional depth data to produce holograms in milliseconds. This has overcome the challenge that has held back the advancement of 3D holography for decades.

The extra depth information that is needed can be captured using a LiDAR sensor or a multi-camera arrangement, which are both offered by recently released smartphones. Amazingly, the tensor network can be compacted to just 1 MB of space, which paves the way for this potential technology.

Potential future applications and use

There is an added incentive in the new breakthrough as an optical engineer at Microsoft observed that holographic displays could be adjusted to any visual impairments. They also suggested that the technology could potentially be an improvement on glasses or contact lenses, which cannot correct every issue.

As a significant breakthrough in technology, the idea of 3D holograms is enough to get anyone excited. We are all able to imagine the wide range of applications such a new tool could have because we have all seen it before in the movies. These could include virtual reality, property viewing, the depiction of medical information, or 3D printing – holographic imagery would be much in demand in just about any industry.

This could be the next technology to disrupt and transform the world since the internet – and it’s finally here.