For many organizations, biometric-enabled access represents a significant convenience and security upgrade over physical badges, which can be easily lost or shared. Knowing this, we challenged ourselves to fit a biometric reader alongside a door controller, a badge reader, a camera, and a touchscreen in a single, weather-sealed enclosure.
But door readers aren't phones. You can't move them out of direct sunlight, can't wait for better lighting, and can't assume the person in front of the device is holding it steady a few inches from their face. These constraints shaped almost every decision behind the Face Unlock feature on our AF64 Access Station Pro, which provides seamless entry to secured spaces thanks to sub-second authentication and high-confidence liveness detection.
Unlike consumer-grade face unlock features, Verkada’s access control hardware must function amidst variable real-world conditions, from rain to high-noon solar glare and everything in between. To ensure the reliability of our Face Unlock feature, our team had to reconcile two distinct data streams operating at vastly different frame rates: high-fidelity RGB imaging and Indirect Time-of-Flight (iToF) depth and infrared light sensing.
RGB identification and the requirement for material alassification
We started by combining a standard Verkada badge reader with a Verkada camera so the device can see people as they approach a door. This camera captures standard, high-fidelity RGB (red, green, blue) color streams, the exact type of visual data that standard facial recognition algorithms are trained on. By storing face credentials (encrypted numerical vectors) of people who have access to the door, we can then use these facial recognition algorithms to match images from the camera against these stored vectors. If there's a match, the controller unlocks the door. However, with this functionality alone, an unauthorized individual could present a photo of someone else to “spoof” our device into unlocking the door. Therefore, we needed to add additional safeguards to prevent common spoofing attacks.
One effective technique to distinguish photos from people is called stereo vision. In this setup, two cameras are slightly offset similar to human eyes, allowing a system to capture depth information in addition to an RGB image. With a full 3D map of subjects and objects in the frame, algorithms can easily distinguish between a real person standing in front of the reader and a photo of that person. Unfortunately, this technique doubles the number of pixels entering the system and requires additional overhead to compute depth information. The SoC we wanted to use (Ambarella CV75) would likely not be able to process this data as fast as we would like for a near-instantaneous unlock experience.
Stereo Vision Test
Furthermore, what if someone wanted to trick the device with a face replica?
This is where infrared (IR) technology comes in. While 2D/3D spoofing can get past standard RGB-D (D for depth) systems, the difference in IR reflectivity (albedo) between human skin and synthetic materials provides a definitive liveness signal. To reliably grant access to individuals and prevent spoofing attacks, we needed to combine color detection from a RGB sensor with the depth and reflectance data from an IR sensor.
IR sensor selection: evaluating iToF vs. structured light for outdoor reliability
Having settled on the addition of an IR sensor for liveness detection , the next question was which type of sensor to use. One type of IR sensor, called structured light, projects a pattern of IR dots and measures their deformation. This is the most common type of sensor on the market today; if you’ve ever used Apple’s Face ID, you’ve used structured light.
Structured Light Test
The second option was iToF, Indirect Time-of-Flight, which measures the phase shift of modulated continuous-wave IR light to generate high-resolution, per-pixel depth maps.
We tested both types of IR sensors with Verkada hardware and started collecting data. Quickly, the winner became clear. Structured light worked well inside, but in direct sunlight, it was a disaster; ambient infrared overwhelmed the projected pattern and the depth information was unusable. Lesson learned: Door readers aren’t phones. They need to work at greater distances with a wider field of view, and you can’t move them around to improve lighting or avoid bad weather.
Because iToF relies on timing in addition to active IR illumination, we found it to be far more robust in these “real world” conditions. iToF significantly suppresses the DC component of ambient sunlight through synchronous demodulation and differential sampling. Combined with an RGB sensor for color, it gave us the optimal combination of security and reliability.
iToF Test
The multi-rate challenge
The Ambarella CV75 and the Verkada Command platform are optimized for a standard frame rate. While we initially considered higher frame rates for the camera on the AF64, we standardized the RGB streams to maximize exposure time. We found that higher frame rates significantly reduced face recognition recall in challenging backlit conditions, while staying at the same FPS (frames per second) as our platform and SoC ensures the sensor can gather enough light for reliable identification. Meanwhile, our iToF sensor captures depth and IR data at a lower FPS, as each frame requires significant post-processing before it can be used for inference.
Synchronizing these different frame rates without introducing UI lag or dropping biometric data became the primary architectural challenge we needed to solve.
To reconcile these streams, we set up two independent data pipelines: one handles the high-frequency RGB data for the display and video streams, while the other captures low-frequency iToF depth/IR data. When a face is detected in the RGB stream, we pull the closest IR/depth frame by timestamp and feed both into our liveness model. This model is a proprietary neural network that analyzes depth and IR signatures to determine with high confidence whether the camera is observing a real person.
However, sending these two different streams to our CPU would result in bus contention, essentially a data bottleneck for the chip that could degrade the responsiveness of the video feed. To mitigate this, we leveraged the CV75’s hardware scaler to downsample the depth packets directly at the MIPI CSI-2 ingress point (the hardware interface where the camera sensors physically feed data into the SoC) to a resolution optimized for our neural network, all without losing the spatial resolution required for inference. This required directly tuning the SoC registers to ensure the timing of the scaler was perfectly aligned with our dual-pipeline ingestion. Once implemented, we could run our proprietary liveness model with low latency while still satisfying the bandwidth and performance requirements for cloud streaming, remote management via Verkada Command, and other concurrent services.
Beta-testing: memory exhaustion during a live demo
Despite these architectural wins, the “real world” had one more test for us. One of our first tests was during a demo at an external event where a crowd of people wanted to test our new biometric reader in quick succession. The high traffic overloaded the system, exposing a critical memory bottleneck encountered when attempting to process too many faces in a short period of time. We quickly reproduced this problem in our lab using a dataset of high-traffic footage and identified memory exhaustion within the heaps reserved for the DSP/ML processors responsible for high-speed signal processing. Tuning the ION memory heap and adjusting the neural network’s priority in the scheduler resolved the conflict, stabilizing the inference engine for high-traffic environments.
Expanding biometric access
Face Unlock is now live across the AF64 product line. The sensor fusion framework and the iToF driver we built are already finding new applications across our broader hardware ecosystem, where these same liveness guarantees are required even as physical form factors vary.
The deeper payoff from this project was the systems knowledge: writing low-level sensor drivers, tuning SoC registers directly, and building dual-pipeline inference architectures. That expertise is now a foundational part of how we approach building new hardware.
Interested in working on problems like this? We’re hiring.
