Pre-positioning virtual objects at their intended locations in the real world environment
Enhancing Public Engagement in Urban Planning with AR
Pre-positioning virtual objects in augmented reality (AR) is crucial for applications like urban planning, where stakeholders need to visualize proposed structures within existing environments. Traditional methods, such as signage and 2D renders, often fail to engage the public effectively. AR offers an immersive experience, allowing municipalities to showcase final designs in real-world settings, thereby enhancing public understanding and feedback. However, positioning virtual objects accurately in dynamic environments, like construction sites, presents challenges. Instead of relying on pre-generated 3D maps, using visual fiducial markers, such as AprilTags, provides a more adaptable solution, though considerations like high pedestrian and vehicle traffic must be addressed to ensure precise placement.
AR Pipeline for Pre-Positioning Virtual Objects
I have developed an AR pipeline that accurately pre-positions virtual objects at their intended locations in the real world. In this approach, users can view the virtual objects but cannot move them. The pipeline consists of four main modules: data capture, setting the origin, localization, and rendering. Each module plays a crucial role in ensuring precise alignment of virtual assets with the physical environment. This process is illustrated in the figure below.
Data Capturing
In this module, input data—including monocular camera frames and Inertial Measurement Unit (IMU) measurements—are captured. The camera operates at 30Hz, while the IMU records data at 400Hz. To ensure reliable localization, both data sources must maintain stable frequencies, as missing camera frames can cause the algorithm to fail.
Setting the Origin
Aligning the origins of the real and virtual worlds is essential. In AR applications, the device's initial pose is typically set as the origin, with all subsequent poses calculated relative to it. In this pipeline, an AprilTag serves as the common reference point for both worlds. At the start of the application, users scan the AprilTag, allowing the AR system to align its origin with the tag's position, ensuring accurate placement of virtual objects.
Localization
In this pipeline I used five state-of-the-art mono-inertial localization algorithms including: VINS-Mono, ORBSLAM3, Open-VINS, Kimera, and DM-VIO.
Rendering
A virtual cube is rendered on the screen using OpenCV. Initially, the algorithm estimates whether the virtual object is within the camera's field of view and then proceeds with rendering.
Results
The pipeline has been tested in both indoor and outdoor environments.
Indoor Environments
The virtual cube was placed at three different locations on a grid: (10, 2), (16, 14), and (60, 26). The following images demonstrate these placements:
Outdoor Environments
The virtual cube was placed at coordinates (0, 0) and (0, 5). The following videos demonstrate these placements:
Using this AR pipeline, we can reliably pre-position virtual objects in various environments, enhancing applications like urban planning by providing stakeholders with a tangible sense of proposed developments.