Experience Real-Time 3D Holograms on Your Smartphone!

Table of Contents

1. Introduction
2. Computer-Generated Holography: An Overview
   • 2.1 Applications of Computer-Generated Holography
   • 2.2 Challenges in Computer-Generated Holography
3. Conventional Methods for Generating Holograms
   • 3.1 Point-Based Method
   • 3.2 Light Field-Based Method
   • 3.3 Multi-Layer Image-Based Method
   • 3.4 Low Polygon Mesh-Based Method
4. Limitations of Existing Spatial Light Modulators
   • 4.1 Phase-Only Modulators
5. Direct Methods for Hologram Generation
   • 5.1 Double Phase Encoding
   • 5.2 Speckle Noise and Occlusion Boundaries
6. Introducing Deep Learning for Real-Time Holography
   • 6.1 Occlusion Modeling with Surface Mesh Reconstruction
   • 6.2 Convolutional Neural Network for Physical Simulation
   • 6.3 Collimated Projection Geometry for Computational Efficiency
7. Training the Convolutional Neural Network
   • 7.1 Creating a Custom Dataset for True 3D Holograms
   • 7.2 Network Architecture and Training Losses
8. Real-Time Photorealistic 3D Holography
   • 8.1 Faster Computation and Mobile Device Compatibility
   • 8.2 Generalization to Real-World Scenes
9. Experimental Results and Validation
   • 9.1 Simulated Focal Sweep of Real-World Scenes
   • 9.2 Artifact-Free Phase-Only Encoding
   • 9.3 Benchtop Holographic Projector
   • 9.4 Capturing and Projecting Real-Time 3D Holograms
10. Conclusion

Computer-Generated Holography: Advancing Real-Time 3D Graphics

Computer-generated holography has found important applications in various domains, such as biosensing, 3D visualization, and security. This technology enables the projection of true 3D holograms, offering advantages in virtual and augmented reality displays. However, generating a true 3D hologram for real-time applications is computationally expensive and slow. Existing methods suffer from challenges like occlusion modeling, computational limitations, and image quality trade-offs.

In this article, we explore the advancements in computer-generated holography that combine deep learning techniques to enable real-time, photorealistic 3D holography. We address the challenge of occlusion modeling by reconstructing a surface mesh and detecting foreground occlusion during sub-hologram calculation, ensuring a clean foreground reconstruction and preventing background leakage. To accelerate computation, we utilize a convolutional neural network (CNN) as a proxy for physical simulation, learning spatially invariant convolution kernels that build sub-holograms faster than traditional methods.

Our trained CNN provides accurate predictions of target holograms and significantly reduces computational costs, making it suitable for different devices, including mobile phones. Through extensive experimentation, we demonstrate the generalization of our approach to real-world scenes, showcasing The Simulation of focal sweeps and artifact-free phase-only encoding. We also validate our method using a benchtop holographic projector and present results of live capture and projection of real-time 3D holograms.

In conclusion, our proposed system represents a significant advancement in computer-generated holography, combining physics-based approaches with deep learning to generate real-time, high-resolution, speckle-free, full-color holograms. The availability of our dataset and code on GitHub enables further research and development in this field.