Deep Cracks

[Jason]

Adhesivebandage or tape: Watertight medical tape can seal cracks. It also keeps the wound moist, which helps it heal. For small cuts, you can also use the sticky part of a bandage. Put them across the wound. They can act as a bridge to hold the skin together.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

There are several studies on segmentation and recognition of tongue cracks, and early studies almost concentrated on the methods of threshold segmentation and line detection. The threshold method uses the grayscale difference or color difference between the tongue crack and other parts of the tongue surface to establish a threshold for segmentation, while the line detection method calculates the speed of change in the color or grayscale brightness of the edge of the tongue crack, so as to obtain the contour of the crack. Rhee5 used U-Net and an adaptive threshold technique to extract tongue cracks within post-processing. Liu et al.6 attempted to extract tongue cracks by using a method based on the wide line detector which extracts the whole of the line by employing an isotropic nonlinear filter. Li et al.7 proposed a new method using statistic feature extracted by wide line, such as Max-distance, to train a binary SVM as a classifier for cracked tongue. And now more deep learning methods are used for crack defection based on classification, pixel segmentation and object detection8,9,10. Weng et al.11 proposed to train the tooth-mark and crack detection model by using tongue images annotated bounding-box. It was a weakly supervised method that added several classification branches to recognize the tooth-marked tongue and cracked tongue according to the YOLO object detection model. Peng et al.12 proposed a P-type neural network architecture based on a lightweight encoder-decoder structure which could get the detailed extraction result at pixel level. Xue et al.13 proposed to use cracked and non-cracked regions to train Alexnet to extract deep features of cracked regions. This method focuses on localized cracked regions and trains a multi-instance support vector machine (SVM) to make the final decision. However, the effectiveness and generalization of these methods still need to be improved.

Therefore, this paper applies a deep learning network SBDL to extract and identify tongue cracks. The rest of this article consists of five parts. Section "Tongue crack extraction based on SBDL" described the tongue crack extraction based on SBDL. Section "Results" described the details of the experimental and reported the results. The results of the experiment were discussed in Section "Discussion". Finally, the conclusion was drawn in Section "Conclusion".

The flow chart of tongue crack extraction based on SBDL is shown in Fig. 2, which mainly includes five steps: tongue crack labeling, model training, model testing, optimization of tongue crack extraction results and result evaluation. Firstly, the data of cracked tongue and non-cracked tongue obtained by the tongue image segmentation model is allocated to training samples and test samples; secondly, the annotation tool Crack-Tongue is used to label the cracks in tongue image with the size of 400 400 to get the ground truth of tongue crack, and then put them into the corresponding training set and test set; thirdly, the tongue crack extraction model is trained by using the training set and SBDL network; fourthly, the test results with the size of 50 50 are optimized by means of the erosion and refinement operations to obtain the final results; finally, the results are evaluated quantitatively.

The tongue images used in this paper were provided by the Four Diagnostic Information Comprehensive Laboratory of Shanghai University of Traditional Chinese Medicine, including 176 images with tongue crack and 140 images without tongue crack. These tongue images are assigned to training samples and test samples in a ratio of 8:2, which means that there are 141 images with tongue crack and 112 images without tongue crack in training samples, and there are 35 images with tongue crack and 28 images without tongue crack in test samples. Since the tongue image dataset is too small, overfitting is easy to occur during training, which makes the training results more specific to a certain type of images and lacks sufficient generalization ability. Therefore, it is necessary to expand the number of tongue crack samples.

Adding salt and pepper noise. Salt and pepper noise, also known as impulse noise, is a common noise. After the image is subjected to this kind of noise, the gray value of the noise point is very different from the surrounding pixels. Usually, even if the image suffers very little salt and pepper noise, its details will be greatly destroyed15. Visually speaking, images subjected to salt and pepper noise will randomly produce small white or black dots.

Changing the contrast. There are many ways to change the contrast, which can be mainly divided into three types: image sharpening, smooth denoising and grayscale adjustment according to the processing purpose. This paper mainly uses grayscale adjustment to change the contrast.

After the above three image processing methods, the crack tongue data in training samples has been expanded from the original 141 to the existing 564. Therefore, there are 564 positive and 112 negative samples used as training samples; and there are 35 positive samples and 28 negative samples are used as test samples.

The self-developed Crack-Tongue labeling tool is used to label tongue crack. The tool uses the shortest path between the labeling points to label based on the magnetic lock sleeve, and obtains a polygon that can fit the contour of the tongue crack. The labeling result is shown in Fig. 4.

Deep learning networks have two important factors that determine the speed of computation: the amount of training data and the computation amount of network parameters and floating-point numbers. Usually, in order to improve the accuracy of the training model, deep learning needs to use a large number of data samples to perform deeper feature learning under the adjustment of multiple network parameters. In order to improve the training speed and reduce the amount of calculation, the SBDL network divides the network into two stages: semantic segmentation stage and decision stage14. It uses the public crack detection to train and test, which achieves very good results. It uses a small data set to train a crack detection network model with small amount of calculation and high precision. Considering the surface crack detection problem as a binary image segmentation problem, pixel-level semantic segmentation can firstly be achieved through a semantic segmentation network, and then the segmentation results can be used as the input of the next decision stage.

The tongue crack extraction model was trained by using the SBDL network in tensorflow 1.13 with CUDA10.0 and cuDNN7.5.0. The whole training process adopted stochastic gradient descent (SGD) to achieve a faster convergence speed. For the binary classification problem, the segmentation network only classified a single pixel in the image, used Mean Square Error (MSE) loss regression to perform bounding box regression, and used the cross-entropy loss function as a single pixel binary classification training strategy. The two networks were trained separately. Firstly, the segmentation network was trained, and then the decision network was trained while the weights of the segmentation network remained unchanged. The decision network only fine-tuned the weights during training, which largely avoid the risk of overfitting. Both stages were trained using SGD with a learning rate and cross-entropy loss rate of 0.005 and 0.1. Due to the image size and GPU memory constrained, the batch size was set to 2. Although the selection of training samples was random, in order to balance the number of cracked and non-cracked tongue images, the network detected cracked tongue image when the iteration number was even and non-cracked tongue image when the iteration number was odd. This approach ensured that the network detected tongue images at a constant rate. Otherwise, the imbalance of feature learning would bring poor generalization ability, overfitting and low learning speed to the model. The experiments are conducted on a 64-bit windows server with one Intel Core i7-10700 CPU, a 16G memory and one Nvidia RTX 3060 GPU.

The extraction of tongue cracks belongs to semantic segmentation. Semantic segmentation is pixel-level classification. The indicators used to quantitatively evaluate the effect of semantic segmentation are Mean Pixel Accuracy (MPA), Mean Intersection over Union (MIoU) and Frequency Weighted Intersection over Union (FWIoU). And Class Pixel Accuracy (CPA) and Intersection over Union (IoU) are specially added as the evaluate indicators of tongue crack segmentation. The formulas are as follows:

In the formula, TP means the positive samples are correctly predicted, and TN means the negative samples are correctly predicted; FP and FN mean the samples are mispredicted, the former means the positive samples are wrongly predicted as negative samples, and the latter means the negative samples are wrongly predicted as positive sample. SE refers to the number of positive samples correctly identified, and specificity (SP) refers to the number of negative samples correctly identified. Accuracy (ACC) refers to the total number of samples correctly identified.

At the same time, considering the application of tongue crack recognition in actual situations, this paper also adopts the professional evaluation for tongue cracks in traditional Chinese medicine (TCM). The experts of TCM usually use three terms: single crack, double cracks and multiple cracks to describe the number of tongue cracks, and use shallow cracks and deep cracks to describe the depth of tongue cracks. Therefore, in the objective study of tongue diagnosis in modern Chinese medicine, the visible index is commonly used to indicate the number of cracks, and the depth index indicates the depth of the crack. The specific formulas of the visible index and the depth index are as follows:

3a8082e126