A New Twist Two-step Iterative Shrinkage Thresholding Algorithms For Image Restoration
Tammara Freimark
Fiber bundle endomicroscopy has potential for facilitating high-resolution (HR) in vivo imaging. One of the main challenges of this technique is the improvement of image restoration for better visualization. In this paper, we propose to reconstruct a HR image without a fixed honeycomb pattern from a noisy observation image, which can be formulated as an inverse problem. We use the obtained fixed honeycomb pattern as a prior image and use a two-step iterative shrinkage thresholding algorithm with a total variation regularization to solve this problem. In addition to the fixed honeycomb pattern removal, our method can also improve spatial resolution. The feasibility of our method is demonstrated by the images obtained from the USAF target and spider silks. In each ease, our method recovers more details than that recovered by the conventional method. The proposed theoretical framework for the removal of the honeycomb pattern in this paper shows promising and wide applications for fiber bundle imaging.
In this paper, we present a theoretical framework to remove the fixed honeycomb patterns. We formulate this problem as a deconvolution and restoration problem, i.e., estimating a HR image from the observed blurred and noisy LR image. We use a two-step iterative shrinkage thresholding (TwIST) algorithm26 with the total-variation (TV) regularization to address this problem. Our method enables the removal of the honeycomb patterns and improves the spatial resolution, which is demonstrated by the images obtained from the USAF target and spider silks.
A schematic of the custom-built fiber endomicroscopy system for fluorescence imaging is shown in Fig. 1. A 475 nm light-emitting-diode (LED) source is used to illuminate the sample, and the emitted light is separated by a dichroic mirror toward a CCD camera (Micro-shot, MC25, 6.45 m pixel size) through the same fiber bundle (GastroFlex UHD). To excite fluorescence at a certain wavelength, an excitation filter is positioned after the LED source. An emission filter is placed after the dichroic mirror to select fluorescence at a specific wavelength while rejecting the excitation light. A lens with 150 mm focal length is used to relay light to the camera. The employed fiber bundle has a diameter of 1.4 mm and contains approximately 30 000 cores. A miniature objective lens with a 2.5-fold magnification is designed at the distal end of the fiber bundle to achieve a high magnification imaging close to the sample surface. The proximal end of the fiber bundle is coupled to an objective lens (Olympus, NA = 0.3). The magnification between the proximal end of the bundle and the camera is approximately 8.3.
Figure 2 illustrates the acquisition process of a raw image in the fiber bundle when receiving fluorescent light from the sample being imaged. The vectors x0, x, x1, and y represent the obtained fluorescent intensities at the sample being imaged, at the distal end of the fiber bundle, at the proximal end of the fiber bundle, at the imaging plane, respectively. First, the fluorescence in x0 passing through the miniature objective lens is collected in x, and the spatial blurring effect can be characterized by the convolution with the point spread function (PSF) h0 of the miniature objective lens. Then, the obtained light intensity in x1 is characterized by pointwise multiplying of x with the matrix M, where M represents the optical transmittance of the fiber bundle considering the irregular shape and size of the fiber cores in the fiber bundle. Finally, a LR image y in the imaging plane is obtained by the convolution with PSF h of the imaging system. Note that the miniature objective lens is designed to achieve high magnification imaging and usually has a high-NA. Therefore, we do not consider this blurring effect and aim to recover the intensity in x0 with x0 = x.
Estimating an original HR x from a known LR image y using Eq. (1) is a typical ill-posed linear inverse problem. Consequently, this problem requires additional regularization to reduce uncertainties and improve computational performance. We use the state-of-the-art algorithm based on TwIST26 to address this problem by minimizing the following objective function:
where λ is the regularization parameter, which is used to control the relative weight between the l2-norm of the residual and total-variation (TV) norm of the estimation. Δihx and Δivx denote the horizontal and vertical first-order local difference operators,26 respectively. Usually, the more the honeycomb patterns, the larger the TV norm.
The flow chart of the algorithm is shown in Fig. 3. The two-step iteration starts after x0 is initialized and x1 is obtained by the denoising function, i.e., each iteration depends on the two previous iterations. Note that α and β are iteration parameters. For each iteration, the residual R is calculated and an IST step with α = β = 1 is used to guarantee monotonically decreasing residuals. The image x will be updated iteratively until some stopping criterions are satisfied, such as the relative change in residual estimation is less than a threshold or the maximal loop number is finished. This algorithm is easy to implement and computationally inexpensive, which can realize a HR image reconstruction.
The smallest line pairs on the USAF target that can be resolved by applying Gaussian smoothing with a pre-histogram equalization are G7E5, as shown in Fig. 5(b). This corresponds to a spatial frequency of 203 lp/mm and has a contrast value of 3.1%, as shown in Fig. 5(f). The Gaussian smoothing can remove honeycomb patterns. However, there is no significant improvement in spatial resolution observed in Figs. 5(a) and 5(b). For our proposed method, the smallest line pairs are G8E3, as shown in Figs. 5(c) and 5(d). This corresponds to a spatial frequency of 323 lp/mm, which results in an approximately 1.59-fold resolution improvement compared with the reconstructed image by Gaussian smoothing. Meanwhile, Fig. 5(f) clearly shows the resolution improvement obtained using our proposed method. Note that the additional histogram equalization can improve the contrast of the reconstructed image by our proposed method, as shown in Figs. 5(d) and 5(e).
The experimental results demonstrate that our proposed method can remove honeycomb patterns and correct for the problem of core heterogeneity and auto-fluorescence, which further leads to achieve an improvement in the spatial resolution for fiber bundle imaging. Combined with our imaging system, our method can resolve features separated by 1.55 m on the USAF target. Compared with other methods, our method has the following advantages. First, we use a deconvolution operation to obtain the optical transmittance of the fiber bundle, which does not rely on the assumption about the spatial structure of the fiber bundle. Second, our proposed method can be applied to a wide range of fiber bundle imaging systems. It is possible that significant improvements in spatial resolution could be obtained by using multiple LR images. Third, we formulate this problem of honeycomb pattern removal as a deconvolution and restoration problem. This problem is addressed in a framework by using the TwIST algorithm with TV regularization. It may be possible to use other algorithms28 with different regularizations to address this problem and improve estimation performances.
One problem associated with our method is the background noise. In wide-field fiber bundle imaging, the background fluorescence of the fiber bundle is significant, which provides a significant offset to all fluorescent measurements from the samples and limits the image resolution. In this paper, we use a commercially available Cellvizio Gastroflex UHD Probe, which consists of 30 000 cores. The core spacing is approximately 3 m, and the miniature objective lens has a 2.5-fold magnification at the distal end of the fiber bundle. Based on Nyquist sampling, we would, therefore, expect a resolution of around 1.2 m, which corresponds to G08E05 on the USAF. It is necessary to further reduce the influence of background noise on the quality of our final images in the future.
In summary, we have developed a theoretical framework to remove the fixed honeycomb patterns for fiber bundle images based on the TwIST algorithm with a TV constraint. The experimental results confirmed our theoretical predictions. In addition to the honeycomb pattern removal, our method can reduce the effects of the noise on the images and further recover more details, which results in almost a 1.59-fold improvement in resolution. Our approach can be used for other fiber-bundle-based imaging systems and further developed to realize real-time high-resolution visualization in clinical endomicroscopy imaging.
Compressed sensing (CS) has been successfully demonstrated to reconstruct ultrafast dynamic scenes in ultrafast imaging techniques with large sequence depth. Since compressed ultrafast imaging used a two-step iterative shrinkage/thresholding (TwIST) algorithm in previous image reconstruction, some details of the object will not be recovered when the amount of data compression is large. Here we applied a more efficient Total Variation (TV) minimization scheme based on augmented Lagrangian and alternating direction algorithms (TVAL3) to reconstruct the ultrafast process. In order to verify the effectiveness of the TVAL3 algorithm, we experimentally compare the reconstruction quality of TVAL3 algorithm and TwIST algorithm in an ultrafast imaging system based on compressed-sensing and spectral-temporal coupling active detection with highest frame rate of 4.37 trillion Hz. Both dynamic and static experimental results show that, TVAL3 algorithm can not only reconstruct a rapidly moving light pulse with a more precise profile and more fitted trajectory, but also improve the quality of static objects and the speed of reconstruction. This work will advance the ultrafast imaging techniques based on compressed sensing in terms of image reconstruction quality and reconstruction speed, which finally helps promoting the application of these techniques in areas where high spatial precision is required, such as phase transitions and laser filamentation in nonlinear solids, etc.
c80f0f1006