XRiddle

Overview

XRiddle is a standalone release of the AR architecture built for The Labyrinth of Echoes, a team immersive installation project at Hochschule Darmstadt inspired by Pink Floyd's work. I served as lead creative technologist on that project and built the full AR prototype. After the semester ended I stripped the copyright-sensitive content, refined the lifecycle isolation, cleaned and commented the code, and released it publicly.

The core technical problem is the same one the original prototype had to solve. AR Foundation and Vuforia both compete for the same device camera pipeline, and breaks or struggles if running in parallel.

Tested on Android, both frameworks support iOS by design and the architecture is ready for it, an Apple developer license just was not available at the time to verify.

The Problem

AR Foundation and Vuforia do different things well. AR Foundation handles plane detection and world anchoring. Vuforia handles image recognition and target-based tracking. There are real scenarios where you want both in the same experience.

The problem is they conflict at the hardware level. Both try to own the camera, the gyroscope, and the XR subsystem initialization. Running them naively together causes instability.

The Solution

The key insight was that the ARSession must remain active at all times. Disabling it causes a black screen because both AR Foundation and Vuforia share the underlying camera pipeline. Toggling the session itself is not the right level of control.

Instead, SwitchARMode toggles VuforiaBehaviour and ARPlaneManager independently while leaving ARSession running. Switching to image tracking mode enables VuforiaBehaviour and the ObserverBehaviour components on each image target, then disables ARPlaneManager. Switching to placement mode does the reverse. The two frameworks never actively conflict because only one set of components is processing camera input at a time, while the shared session underneath stays stable throughout.

UI elements are toggled through CanvasGroup alpha, interactable, and blocksRaycasts wherever possible, which avoids layout recalculation overhead and keeps the visual state consistent without destroying and rebuilding the canvas hierarchy. SetActive is reserved for the cases that need it. Which are the 3D tracked objects like the planes, the placed model, and the canvases managed by Vuforia's observer handler, which toggles Canvas components directly regardless of CanvasGroup state.

Canvas Reparenting

One non-obvious problem with Vuforia image tracking is that Vuforia's DefaultObserverEventHandler disables Renderer, Collider, and Canvas components on the target's children when a target is lost. If a UI canvas is a child of the image target, its Canvas component gets disabled when tracking is lost, so the canvas goes dark even if you manage its visibility through CanvasGroup alpha.

This was solved by reparenting the canvas to a persistent transform in the scene when a target is first tracked. The canvas still appears attached to the target visually, but it is no longer a child of the target hierarchy. When tracking is lost, Vuforia disables the components under the image target as usual, but the canvas is no longer among its children. Only one canvas is visible at a time, tracking a new target hides the previous one, so content swaps instead of stacking over each other.

What the Experience Does

In image tracking mode, pointing the camera at a target image triggers a puzzle or interactive element anchored to it. Multiple target images are supported, each with its own content.

In placement mode, scanning a real-world surface positions the 3D model at the detected plane location. A scale slider adjusts the XR Origin's inverse scale, making the placed model appear larger or smaller relative to the environment. Mode switching happens within the same session, which is the point.

What I Learned

The biggest lesson was to define system boundaries before writing a single line of code. The conflict between AR Foundation and Vuforia is not obvious until you try to run them together. Thinking through which system owns which resource, and when, before touching the editor would have saved significant debugging time. The specific lesson about ARSession staying active was the kind of thing that only becomes clear after rigorous testing. That habit of mapping ownership and boundaries upfront as best as possible now comes before any implementation decision on projects involving multiple systems or frameworks.