Compressive Spectral Imaging （高光谱计算成像与分析）

Introduction of Hyperspectral Images

Compared with the Red-Green-Blue (RGB) imaging system, hyperspectral imaging technique is an advanced technique with the ability to capture spatial information across a broader range of wavelengths, as shown in Fig. 1. Hyperspectral imaging technology is significant for vast applications, such as remote sensing, medical diagnosis, and industrial inspection, as illusrated in Fig. 2.

Fig1: Comparison of imaging techniques and difference between RGB images and hyperspectral images.

Fig2: Applications of hyperspectral images.

Introduction of Hyperspectral Imaging Techniques

For getting the spectral images, typical hyperspectral imaging systems are usually based on 1-dimensional (1D) or 2-dimensional (2D) scanning mechanism, e.g., whiskbroom, pushbroom, and bandpass filters scanning spectrometer. Recently, snapshot hyperspectral imaging techniques, especially, Compressive Spectral Imaging (CSI) or Spectral Compressive Imaging (SCI), has gained considerable attention for its ability to simultaneously capture spatial and spectral information in a single or few exposures. The principle of these four imaging techniques are listed in Fig. 3.

Fig3: Comparison of different hyperspectral imaging techniques.

Introduction of Computational Spectral Imaging Technique

Computational spectral imaging systems usually follows the `encode-decode' philosophy, which first encode the 3D spectral image into a compressive measurement and then decode (reconstruction) it. With the advances in high-performance computational power, the computational imaging technique has been attracting an increasing interest due to higher spatial and spectral resolutions. With the merit of low cost, the Coded Aperture Snapshot Spectral Imaging (CASSI) spectrometer is one of the most representative systems among the computational spectral imaging systems. In general, the typical CASSI systems include the Single-Disperser Coded Aperture Snapshot Spectral Imaging (SD-CASSI) and Dual-Disperser Coded Aperture Snapshot Spectral Imaging (DD-CASSI). The forward observation model can be seen in Fig. 4 (a)-(b). The computational spectral imaging can be formulated as an inverse imaging problem. Existing methods rely on selecting hand-crafted pirors, such as total variation, low-rank and then construct a constrained optimization problem to reconstruct hyperspectral images.

Fig4: (a) The forward observation model of SD-CASSI system; (b) the forward observation model of DD-CASSI. system (`spe.' denotes `spectral' and `spa.' represents `spatial' in the figure).

There are some interesting and specific refereneces you can read.
[1] M. E. Gehm, R. John, D. J. Brady, et al., “Single-shot compressive spectral imaging with a dual-disperser architecture,” Optics Express, vol. 15, no. 21, pp. 14 013–14 027, 2007.
[2] G. R. Arce, D. J. Brady, L. Carin, et al., “Compressive coded aperture spectral imaging: An introduction,” IEEE Signal Processing Magazine, vol. 31, no. 1, pp. 105–115, 2013
[3] L. Huang, R. Luo, X. Liu, et al., “Spectral imaging with deep learning,” Light: Science & Applications, vol. 11, no. 1, pp. 1–19, 2022.

Our Research

[1] Yurong Chen, Yaonan Wang, Hui Zhang, “Prior Image Guided Snapshot Spectral Compressive Imaging,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023 (doi: 10.1109/TPAMI.2023.3265749). [Code:] https://github.com/YurongChen1998/Prior-Image-Guided-Snapshot-Spectral-Compressive-Imaging

[Abstract]: Spectral images with rich spatial and spectral information have wide usage, however, traditional spectral imaging techniques undeniably take a long time to capture scenes. We consider the computational imaging problem of the snapshot spectral spectrometer, i.e., the Coded Aperture Snapshot Spectral Imaging (CASSI) system. For the sake of a fast and generalized reconstruction algorithm, we propose a prior image guidance-based snapshot compressive imaging method. Typically, the prior image denotes the RGB measurement captured by the additional uncoded panchromatic camera of the dual-camera CASSI system. We argue that the RGB image as a prior image can provide valuable semantic information. More importantly, we design the Prior Image Semantic Similarity (PIDS) regularization term to enhance the reconstructed spectral image fidelity. In particular, the PIDS is formulated as the difference between the total variation of the prior image and the recovered spectral image. Then, we solve the PIDS regularized reconstruction problem by the Alternating Direction Method of Multipliers (ADMM) optimization algorithm. Comprehensive experiments on various datasets demonstrate the superior performance of our method.

Fig5: Prior image guided snapshot spectral compressive imaging.

Fig6: The theoretical analysis of the proposed PIDS and the upper bound of the PIDS.

[Experiment results]: Results on the KAIST dataset.

Fig6: The theoretical analysis of the proposed PIDS and the upper bound of the PIDS.

[2] A. Yin et al. “SSAPN: Spectral-Spatial Anomaly Perception Network for Unsupervised Vaccine Detection”

” IEEE TII “A. Yin et al. “SSAPN: Spectral-Spatial Anomaly Perception Network for Unsupervised Vaccine Detection” 2022.

[Abstract]: Due to the high degree of maintenance of industrial medicine production and the difficulty of collecting fewer abnormal samples, we build a laboratory level hyperspectral medicine detection platform, construct a series of medical hyperspectral detection data sets, and carry out algorithm research

Fig6: Established medical hyperspectral data set.

Fig7: Foreign object detection algorithm based on spatial spectral mapping.

[Experiment results]: Results on the dataset.

Fig7: Hyperspectral foreign body test results.

Acknowledgement （依托项目）

高端医药光谱成像及智能检测分析仪器研制.（自科重大仪器项目）
精密医药光谱图像感知与智能检测技术.（企业项目）

Published Work

Yurong Chen, Yaonan Wang, Hui Zhang*, “Prior Image Guided Snapshot Spectral Compressive Imaging,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023 (doi: 10.1109/TPAMI.2023.3265749). [Code]

项目组成员

陈煜嵘（在读）

博士二年级

刘立柱（在读）

博士二年级

Future research direction

In the future, the project team will further carry out work in hyperspectral imaging, including but not limited to hyperspectral imaging system design, hyperspectral cell segmentation, living tumor segmentation, and origin tracing of biological reagents.

For hyperspectral pharmaceutical industry, medical treatment

Hyperspectral medical lesion cell segmentation

Hyperspectral in vivo tumor segmentation

Trace the origin of Chinese medicinal materials

cooperation between school and enterprise

楚天科技

圣湘生物

卓立汉光

中达瑞和

如果您对此领域感兴趣并想加入我们的研究, 请联系我(chenyurong1998@outlook.com)或者张辉老师(zhanghuihby@126.com).谢谢!