Weird prediction in Edge mode

[Andrea Campolonghi]

I have a model trained in AutoML Vision. I use the edge mode to be able load and deploy my model into a docker container using gcr.io/cloud-devrel-public-resources/gcloud-container-1.14.0:latest. The results I get are pretty different from what I expected after reviewing the evaluate section. I tried deploying the model on GCloud AutoML infra and I confirmed that I get completely different predictions from querying the cloud deployment vs my running local container. Any way to debug this?

[tielve]

Hi,

As you can see in our documentation:

• Edge models are optimized for inference on an Edge device. Consequently, Edge model accuracy will differ from Cloud model accuracy.

The models are usually too large so they must be optimized to be used on mobile. There are multiple techniques to optimize a model  (freeze the graph, quantization, memory mapping, etc) and I believe that most of them will have an effect on the model's accuracy.

When you train your edge model, you can choose between 3 different package sizes. If you didn't already, I suggest that you choose the biggest package which is the one that will have the highest accuracy.

I hope this helps

[Andrea Campolonghi]

Hi,

I already trained the model with the highest accuracy.  Consider we are talking about a pretty small model of less than 1k images.

The edge device I export for is docker and I gave plenty of resources to the running container.

The issue is that predictions do not slightly differ but are very very far. The same image on Cloud report 0.9 confidence for a defined label and 0.19 from the local container.

I will try optimizing the exported model.

Andrea

--
© 2018 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Cloud Vision Discussion Google Group (cloud-visi...@googlegroups.com) to participate in discussions with other members of the Google Cloud Vision community and the Google Cloud Vision Team.
---
You received this message because you are subscribed to a topic in the Google Groups "cloud-vision-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cloud-vision-discuss/smRIhpc5GzU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cloud-vision-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-vision-discuss/ddf28743-cfbd-43f0-a28d-ded74aee804en%40googlegroups.com.

--

Andrea Campolonghi

Cell : +39 347 2298435

acampo...@gmail.com

[Andrea Campolonghi]

Did you ever saw a so big difference between cloud and edge predictions?

Thanks

Andrea

[tielve]

I didn't ever see such a difference between cloud and edge predictions.

I can see now that you may be experiencing a product issue so, as you have already created an issue on issuetracker, I suggest that we continue there because Google Groups are reserved for general product discussion whereas Issue Tracker for product issues (unexpected behaviors) and feature requests .

Thx