Published In
2023 IEEE/CVF International Conference on Computer Vision (ICCV)
Document Type
Article
Publication Date
2023
Abstract
Single-photon 3D cameras can record the time-of-arrival of billions of photons per second with picosecond accuracy. One common approach to summarize the photon data stream is to build a per-pixel timestamp histogram, resulting in a 3D histogram tensor that encodes distances along the time axis. As the spatio-temporal resolution of the histogram tensor increases, the in-pixel memory requirements and output data rates can quickly become impractical. To overcome this limitation, we propose a family of linear compressive representations of histogram tensors that can be computed efficiently, in an online fashion, as a matrix operation. We design practical lightweight compressive representations that are amenable to an in-pixel implementation and consider the spatio-temporal information of each timestamp. Furthermore, we implement our proposed framework as the first layer of a neural network, which enables the joint end-to-end optimization of the compressive representations and a downstream SPAD data processing model. We find that a well-designed compressive representation can reduce in-sensor memory and data rates up to 2 orders of magnitude without significantly reducing 3D imaging quality. Finally, we analyze the power consumption implications through an on-chip implementation.
DOI
10.1109/ICCV51070.2023.00987
Persistent Identifier
https://archives.pdx.edu/ds/psu/41280
Publisher
IEEE
Citation Details
Gutierrez-Barragan, F., Mu, F., Ardelean, A., Ingle, A., Bruschini, C., Charbon, E., Li, Y., Gupta, M., & Velten, A. (2023, October 1). Learned Compressive Representations for Single-Photon 3D Imaging. 2023 IEEE/CVF International Conference on Computer Vision (ICCV). https://doi.org/10.1109/iccv51070.2023.00987