Google Vision API vs GoogleLens

[Alfred Lam]

I am hoping to use Google Vision API to help identify bird down to species level in jpeg photos. Instead of jumping directly into code, I tested a few photos by drag and drop into the URL https://cloud.google.com/vision/?utm_source=google&utm_medium=cpc&utm_campaign=japac-HK-all-en-dr-bkws-all-all-trial-b-dr-1003987&utm_content=text-ad-none-none-DEV_c-CRE_252357841563-ADGP_Hybrid+%7C+AW+SEM+%7C+BKWS+~+T1+%7C+BMM+%7C+ML+%7C+M:1+%7C+HK+%7C+en+%7C+Vision+%7C+General-KWID_43700030642373205-kwd-84320397668&userloc_9061598&utm_term=KW_%2Bgoogle%20%2Bvision&gclid=CjwKCAjwtO7qBRBQEiwAl5WC2zT9hMJBawDLCCXXHj0bJiWwdlobry9hx1P9ksgcw-bZuLjh3K-o0hoC1jQQAvD_BwE

I get only very broad identification saying the photo is a bird, checking the json feedback details also failed to see correct species identification. Yes, for the same photo that I upload into Google Photos and then trigger Google Lens from my Samsung phone, I get extremely accurate result. I tested a few photos of other bird species and get the same result.

I intended to batch process thousands of photos using the Vision API if the results were good. While Google Lens gives very good results, I do not know of any easy way to batch process thousands of files through it. Why Visions API's output cannot goes into species level with accuracy like that of Google Lens? I would suppose they can and should leverage the same backend ML logic. Or will there be additional options or instructions possible if I call the Vision API from actual code instead of the above drag-and-drop test page?

I attached 2 sample files

[Ali (Cloud Platform Support)]

Hi,

If you send a request for label detection, the response will include the labels with the highest confidence detected from the image. Testing this with some sample images of birds, if the confidence is high enough, it will also display a species if one is detected. 

That being said, if you are looking to detect specific bird species, I would suggest having a look at Cloud AutoML Vision. This would allow you to train custom models to detect the labels of your choice and could be tailored to detect the targeted bird species. 

[Alfred Lam]

Thank you for answer, will give it a try

[Alfred Lam]

Tried the suggestion by Ali, uploading images into my Google Cloud Storage (GCS) and then invoke the 'label detection'  API test page against my GCS stored image https://cloud.google.com/vision/docs/labels?apix_params=%7B%22resource%22%3A%7B%22requests%22%3A%5B%7B%22features%22%3A%5B%7B%22maxResults%22%3A5%2C%22type%22%3A%22LABEL_DETECTION%22%7D%5D%2C%22image%22%3A%7B%22source%22%3A%7B%22imageUri%22%3A%22gs%3A%2F%2Fwestern-watch-197913.appspot.com%2F3K5A3244.JPG%22%7D%7D%7D%5D%7D%7D

The result I got is same as whatever I got testing those images against the test site  https://cloud.google.com/vision/?utm_source=google&utm_medium=cpc&utm_campaign=japac-HK-all-en-dr-bkws-all-all-trial-b-dr-1003987&utm_content=text-ad-none-none-DEV_c-CRE_252357841563-ADGP_Hybrid+%7C+AW+SEM+%7C+BKWS+~+T1+%7C+BMM+%7C+ML+%7C+M:1+%7C+HK+%7C+en+%7C+Vision+%7C+General-KWID_43700030642373205-kwd-84320397668&userloc_9061598&utm_term=KW_%2Bgoogle%20%2Bvision&gclid=CjwKCAjwtO7qBRBQEiwAl5WC2zT9hMJBawDLCCXXHj0bJiWwdlobry9hx1P9ksgcw-bZuLjh3K-o0hoC1jQQAvD_BwE

The result is not able to pinpoint the Bird specie, only gave answer 'Bird'. And yet again, the result from Google Lens is much better, giving correct specie name. I am back to where I started, how do I tap into the Google Lens backend to identify my bird-photos in batch mode? Does Google Lens use the Google Vision API platform at all?

[Jun (Cloud Platform Support)]

Hi Alfred, 

Per the information I found "How to integrate Google lens in my app" and "Is Google Lens available as an API service?", it seems that the Google Lens backend is not the same as Cloud Vision API. Thus you'll probably need to contact Google Lens support for information on how can you tap into the Google Lens backend to identify your bird-photos in batch mode.