User:Yimingzzz/sandbox
PhyCV is the first computer vision library which utilizes algorithms directly derived from the equations of physics governing physical phenomena. The algorithms appearing in the first release emulates the propagation of light through a physical medium with natural and engineered diffractive properties followed by coherent detection. Unlike traditional algorithms that are a sequence of hand-crafted empirical rules, physics-inspired algorithms leverage physical laws of nature as blueprints for inventing algorithms. In addition, these algorithms can, in principle, be implemented in real physical devices for fast and efficient computation in the form of analog computing. Currently PhyCV has two algorithms, Phase-Stretch Transform (PST) and Phase-Stretch Adaptive Gradient-Field Extractor (PAGE). Both algorithms have CPU and GPU versions. PhyCV is now available on GitHub and can be installed from pip.
History[edit]
Phase-Stretch Transform (PST) and Phase-Stretch Adaptive Gradient-field Extractor (PAGE) were inspired by the physics of the photonic time stretch [1][2] (a hardware technique for ultrafast and single-shot data acquisition). The edge detection algorithm PST was open-sourced in 2016 and as of mid 2022 has 800+ stars and 200+ forks on GitHub. The directional edge detection algorithm PAGE was open-sourced in Feburary, 2022. PhyCV was originally developed and open-sourced by Jalali-Lab @ UCLA in May 2022. In PhyCV, the original open-sourced code of PST and PAGE is significantly refactored and improved to be modular, more efficient, GPU-accelerated and object-oriented.
Background[edit]
Phase-Stretch Transform (PST)[edit]
Phase-Stretch Transform (PST) is a computationally efficient edge and texture detection algorithm with exceptional performance in visually impaired images [3][4][5]. The algorithm transforms the image by emulating propagation of light through a device with engineered diffractive property followed by coherent detection. It has been applied in improving the resolution of MRI image [6], extracting blood vessels in retina images [7], dolphin identification [8], and waste water treatment [9], single molecule biological imaging [10], and classification of UAV using micro Doppler imaging [11].
Phase-Stretch Adaptive Gradient-Field Extractor (PAGE)[edit]
Phase-Stretch Adaptive Gradient-field Extractor (PAGE) is a physics-inspired algorithm for detecting edges and their orientations in digital images at various scales [12][13]. The algorithm is based on the diffraction equations of optics. Metaphorically speaking, PAGE emulates the physics of birefringent (orientation-dependent) diffractive propagation through a physical device with a specific diffractive structure. The propagation converts a real-valued image into a complex function. Related information is contained in the real and imaginary components of the output. The output represents the phase of the complex function.
Highlighted Features[edit]
Modular Code Architecture[edit]
The code in PhyCV has a modular design which faithfully follows the physical process from which the algorithm was originated. Both PST and PAGE modules in the PhyCV library emulate the propagation of the input signal (original digital image) through a device with engineered diffractive property followed by coherent (phase) detection. The dispersive propagation applies a phase kernel to the frequency domain of the original image. This process has three steps in general, loading the image, initializing the kernel and applying the kernel. In the implementation of PhyCV, each algorithm is represented as a class in Python and each class has methods that simulate the steps described above. The modular code architecture follows the physics behind the algorithm. Please refer to the source code on GitHub for more details.
GPU Acceleration[edit]
PhyCV supports GPU acceleration. The GPU versions of PST and PAGE are built on PyTorch accelerated by the CUDA toolkit. The acceleration is beneficial for applying the algorithms in real-time image video processing and other deep learning tasks. The running time PhyCV algorithms on CPU and GPU for videos at different resolutions are shown below.
| CPU (Intel i9-9900K) | GPU (NVIDIA TITAN RTX) | |
|---|---|---|
| 1080p (1920x1080) | 550 ms | 6.5 ms |
| 2K (2560 x 1440) | 1000 ms | 12 ms |
| 4K (3840 x 2160) | 2290 ms | 28.5 ms |
| CPU (Intel i9-9900K) | GPU (NVIDIA TITAN RTX) | |
|---|---|---|
| 1080p (1920x1080) | 2800 ms | 55 ms |
| 2K (2560 x 1440) | 5000 ms | 95 ms |
| 4K (3840 x 2160) | 11660 ms | 219 ms |
Installation and Examples[edit]
Please refer to the GitHub README file for a detailed technical documentation.
References[edit]
- ^ Time-stretched analogue-to-digital conversion. Bhushan et al. Eletronic Letters techniques, 1998
- ^ Time stretch and its applications. Mahjoubfar et al. Nature Photonics, 2017
- ^ Physics-inspired image edge detection. Asghari et al. IEEE Global Signal and Information Processing Symposium, 2014
- ^ Edge detection in digital images using dispersive phase stretch. Asghari et al. International Journal of Biomedical Imaging, 2015
- ^ Feature Enhancement in Visually Impaired Images. Suthar et al. IEEE Access, 2018
- ^ Fast Super-Resolution in MRI Images Using Phase Stretch Transform, Anchored Point Regression and Zero-Data Learning. He et al. IEEE International Conference on Image Processing, 2019
- ^ A local flow phase stretch transform for robust retinal vessel detection. Challoob et al. In International Conference on Advanced Concepts for Intelligent Vision Systems, 2020
- ^ Dolphin Identification Method Based on Improved PST. Wang et al. In 2021 IEEE/ACIS 6th International Conference on Big Data, Cloud Computing, and Data Science (BCD), 2021
- ^ Image segmentation of activated sludge phase contrast images using phase stretch transform. Ang et al. Microscopy, 2019
- ^ Phase stretch transform for super-resolution localization microscopy. Ilovitsh et al. In Biomedical Optics Express, 2016
- ^ Classification of Drones Using Edge-Enhanced Micro-Doppler Image Based on CNN. Singh et al. Traitement du Signal, 2021
- ^ Phase-stretch adaptive gradient-field extractor (page). Suthar et al. Coding Theory, 2020
- ^ Phase-Stretch Adaptive Gradient-Field Extractor (PAGE). MacPhee et al. arXiv preprint arXiv:2202.03570, 2022