__HOT__ Download Apple Fruit Images
Sunta Sharp
Accurate fruit recognition is one of the crucial steps necessary for the commercialization of robotic harvesting. Many studies have been conducted to recognize the fruit of horticultural crops automatically, such as sweet peppers, cucumbers, citrus, mangos, tomatoes, and apples [3,4,5,6,7,8]. However, challenges still remaining in the implementation of robotic harvesting in regard to fruit recognition. This is due to various factors, especially variable light, and proximal color background. Therefore, more attention has been focused on using different image acquisition devices and means to obtain useful images and favorable characteristic evidence to support the determination of fruit. Some researchers have utilized active imaging systems for fruit recognition: Liu et al. [9] developed a two-dimensional vision sensor system using two kinds of laser beams, to detect matured fruit based on the difference of laser reflectance on a plant, obtaining a recognition accuracy of 67%. Feng et al. [10] used a Time-of-Flight (ToF) camera to acquire multi-source images for the recognition of overlapped fruit, obtaining a fruit recognition rate of 83.67% to 94.22%. Nguyen et al. [11] used a Kinect sensor to acquire depth and color information for the detection and localization of red and bicolored apples, the processing time was below 1 s for a simultaneous detection of 20 apples. Other researchers have sought to create light-stable environments by adding light sources and shields: Song et al. [12] used a Xenon flashlight (with a light pulse to illuminate) to reduce the influence of ambient light on the images and a light shield to block direct sunlight. Gongal et al. [13] used a tunnel structure with a number of LED lights being installed inside to create a controlled, uniform lighting environment and also added capability for nighttime data collection, which achieved an identification accuracy of 79.8%. These past studies have mainly focused on the improvement of the hardware sensors for image acquisition, which could increase the overall cost, even though the recognition could be improved to some degree. These efforts, from a sensor system improvement perspective, are mainly aimed to eliminate the effect of sunlight variation. However, changes in sunlight can also help us to distinguish objects that have different responses in terms of thermal radiation.
With the development of visual sensor technology, an increasing number of object characteristics can be recognized using sensors. Besides using color, shape, and texture, the surface temperature variation of different objects under different conditions has also become a very important feature for object recognition. Thermal radiation is defined as the phenomenon of radiant electromagnetic waves due to the temperature of an object. All objects with a temperature above absolute zero can emit thermal radiation, and the higher the surface temperature, the more radiant the energy. Thermal imaging utilizes electromagnetic radiation emitted from an object and produces a pseudo image of the thermal distribution of the object [14]. In the pseudo images, different objects show in different colors, which could be potentially used for object identification in agricultural production. Recently, a number of applications utilizing thermal imaging have been reported. Raza et al. [15] combined thermal and visible light image data with depth information to remotely detect plants infected with the tomato powdery mildew fungus. Zhu et al. [16] used infrared thermal imaging technology to detect the temperature information of tomato and wheat during the incubation period following the introduction of inoculum. Satone et al. [17] assessed the surface of apples and subsurface defects with thermal images. García-Tejero et al. [18] and Wiriya-Alongkorn et al. [19] utilized thermal imaging to detect plant water stress, which could be an interesting tool for improving irrigation scheduling [18,19]. The studies above analyzed the difference of the plants with stress issues, rather than healthy (well-managed) plants. Some other studies have reported on the identification of different objects in an image using surface temperatures. For example, a few studies have worked on the recognition of citrus fruit from the tree canopy, including the appropriate time for acquiring fruit images, and integration of color image with image registration [20,21,22]. The image registration is an indispensable process, and the accuracy of image registration is one of the factors affecting recognition results. An innovative MSX technology, unlike traditional thermal superposition techniques, integrates infrared images with visible light images and highlights the texture characteristics of objects in infrared images without additional registration. The imaging results using this technology not only increase the evidence to support the fruit region, but also save the time in regard to image processing.
Illustration of images acquired by FLIR C3 camera (FLIR Systems Inc., Wilsonville, USA) in two different scenes: (a) RGB (Red, Green, Blue) image from part of a Crimson Crisp apple tree; (b) Thermal image for the targeted part of a Crimson Crisp apple tree; (c) MSX image for this part of a Crimson Crisp apple tree; (d) RGB image from part of a Gold Rush apple tree; (e) Thermal image for this part of a Gold Rush apple tree; (f) multi-spectral dynamic imaging (MSX) image for this part of a Gold Rush apple tree.
More specifically, the value of each pixel in an MSX image is not directly reflected to that of the appeared color. Instead, these pixel values are used as the entry address of a table item in a color look-up table, to find the intensity values of R, G, and B used to display an image. Compared with a black-and-white grayscale image, the pseudo-color image enhances the image effect and enriches the image information. Considering that most of fruit are distributed in the orange region of an MSX image, which is close to the red component (one component in an RGB color space), the brightness value of a single red component is extracted by the component method, and the grayscale processing of MSX image is carried out.
Typically, during robotic harvesting, fruit images are acquired and processed prior to the robotic arm executing [11]. In this study, the algorithm took about 740 ms to process an image, which is about 120 ms for recognizing one fruit region with the average number of six pieces of fruit in an image. The movement of the robotic arm to reach a fruit in a harvesting cycle normally takes much longer than 120 ms, and the speed we achieved for fruit detection could be sufficient for the harvesting process. As we can see, the pre-processing step takes up a large proportion of the total time in our algorithm, and it could be potentially improved by using a multi-thread acceleration method in the future.
An effective algorithm was developed to detect fruit in the MSX images. The red component of input images was chosen as it highlighted the characteristic of the target. Morphological theory and small area removal strategy effectively removed non-target regions in binary images; the texture characteristics were extracted to enhance the support judgment of the target regions; and the final results were obtained by the linear separable SVM. During the processing period, most fruit regions can be detected, and the feature vectors used in the post-processing are few, so it is faster to use a simple classification model.
When you think of the tech company Apple, do you think of that iconic logo - the silhouette of an apple with a bite taken out? Well, Apple owns the trademark for that logo, meaning they have the exclusive right to use it. In Switzerland, the company is now trying to take that protection further. Apple wants to trademark the apple, as in the fruit. Here to tell us more about Apple's trademark battle is reporter Gabriela Galindo. Hey there.
GALINDO: Yeah. So what Apple has done in Switzerland is that they applied for a trademark of an apple - so, an image of an apple. And specifically, they're trying to gain rights over a photographic - a really true-to-life depiction of an apple variety that's called the Granny Smith. Now, this is the generic green apple that you can find in the supermarket anywhere. And so they want to own the rights to that particular image in Switzerland, and they applied for that protection to the IP rights office in Switzerland. It's called the IPI. And last fall, the IPI said you can have it, but only for some of the goods that you want it for. So then now what's currently happening is that Apple is appealing that. And so what Apple wants is full protection and not just the limited number of goods that the IPI institute agreed to give it for.
KELLY: I'm just trying to wrap my head around this. The image of an apple - a Granny Smith apple - seems - it seems so common, so generic. Can a company, even one called Apple, really make the claim to own it?
GALINDO: Well, that's the big question, really. We will not know that until the court reaches its decision and that can take months. That could take years. But what's interesting here is that, beyond Switzerland, there are countries that have granted it protection for this particular apple. Some examples of countries who have given it is Israel, Japan, the European Union, the African Union have given it.
KELLY: So I want to bring in one other group that has a stake in this, and this is Swiss apple farmers. They, for more than a century, have had a logo that depicts an apple, as you would expect, with a white Swiss cross. This is, like, a play on the Swiss flag, I guess. Tell me where they fit in here.
GALINDO: Yeah. So this could potentially be a big concern for apple growers in Switzerland, and there's an association of growers in particular that is the oldest - the largest, and their fears go as far out as, you know - are we going to be able to continue advertising with our logo?
8d45195817