Transforming AR Experience: User-Aware Rendering

Table of Contents

1. Introduction
2. Device-Perspective Rendering
3. User-Perspective Rendering
4. User-Aware Rendering
5. Testing User-Aware Rendering 5.1. Study 1: Depth Perception 5.2. Study 2: Scene Exploration
6. Results of the User Study
7. Conclusion

User-Aware Rendering: Merging the Strengths of Device- and User-Perspective Rendering in Handheld AR

Augmented Reality (AR) experiences on handheld devices have traditionally used device-perspective rendering, where the virtual content is rendered inside the camera feed from the back-facing camera. While this approach allows for fitting a lot of content on the screen, it often leads to inconsistencies in size and scene depth estimations. To address this issue, researchers have developed user-perspective rendering, which aims to Align the virtual content with the real world from the user's perspective. However, user-perspective rendering has limitations, such as a narrower field of view.

To tackle these challenges, a new rendering technique called user-aware rendering has been proposed. This technique combines the strengths of both device-perspective and user-perspective rendering to provide users with an enhanced AR experience. User-aware rendering calculates an adaptive frustum Based on the user's head position and extends the device size virtually, resulting in a larger field of view. This implementation ensures that the virtual content remains aligned with what the user sees from the real world, regardless of the device's orientation.

To evaluate the effectiveness of user-aware rendering, a user study was conducted. The study consisted of two tasks: depth Perception and scene exploration. In the depth perception task, participants were asked to identify the depth order of constellations consisting of three cubes. The results showed that user-perspective rendering and user-aware rendering achieved higher accuracy rates compared to device-perspective rendering.

In the scene exploration task, participants had to search for and select targets that appeared above a table. User-perspective rendering was found to be slower due to its narrower field of view, while user-aware rendering with 1.5x device magnification and device-perspective rendering performed similarly in terms of speed.

Overall, user-aware rendering with 1.5x device magnification demonstrated superior depth perception and scene exploration capabilities. It provided a usable field of view of around 35°, aligned the center of the screen with the real world, and offered motion parallax based on head input. These findings suggest that user-aware rendering has the potential to enhance human perception in AR applications.