load openvino model issue

[Mohammed Elhmadany]

Hi team,

I have a problem with when starting to call predict function after loading the pre-trained OpenVINO model or loading multiple times by using InferenceModel object using AnalyticsZoo. while I did the same steps with the Caffe model and then called predict function works fine my python version is 3.6. also, I did the same steps using the BigDl InferenceModel object and python version 3.7. the same issue exists. and illustrated as the following:-

**Zoo inference object with using prediction function:- 

** with BigDl  inference object when trying load openvino model more time:

how could  I solve this issue

[Qiyuan Gong]

Hi, 

    Seems our openvino native code in inference model (2020.3.2 LTS) failed to load your OpenVINO model. 

    Can you tell me your local OpenVINO model version (latest model is not supported by inference model)?

    

    If you are using latest OpenVINO, we can load it with Orca (https://github.com/intel-analytics/BigDL/tree/main/python/orca/example/learn/openvino) .

Have a nice day!

Qiyuan

[Qiyuan Gong]

Hi,

If you are using Orca OpenVINO inference, please ensure you are using latest OpenVINO (conda install openvino). You can check details in https://pypi.org/project/openvino/

BTW, yolo model is a little special in OpenVINO’s model zoo. 
Can you tell me how do you get this yolo.xml (covert from TensorFlow or other format)?

Have a nice day!

Qiyuan Gong

On Mar 1, 2022, 18:26 +0800, Mohammed Elhmadany <elhm...@gmail.com>, wrote:

Hi, Thank you for your reply,

I follow OpenVINO Estimator Inference example you mentioned (https://github.com/intel-analytics/BigDL/tree/main/python/orca/example/learn/openvino) applied on VM ubuntu 18.04 and install OpenVINO (2020.3.2 LTS) by steps provided here (https://docs.openvino.ai/latest/openvino_docs_install_guides_installing_openvino_linux.html) also my Conda python is 3.6. but I got this error (AttributeError: 'openvino.inference_engine.ie_api.InferRequest' object has no attribute 'input_blobs'). is this required to upgrade the OpenVINO version or something is missing?

<yolo1.PNG>

<yolo2.PNG>

--
You received this message because you are subscribed to the Google Groups "User Group for BigDL and Analytics Zoo" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bigdl-user-gro...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bigdl-user-group/6a7ac6c5-a6af-4c2b-9f45-786a2c2f794bn%40googlegroups.com.

[Mohammed Elhmadany]

Hi   Qiyuan
good day.

I have followed the example provided in   ( https://github.com/intel-analytics/BigDL/tree/main/python/orca/example/learn/openvino ) and it's okay working with me as images data. but my data is video frames and the solution I did as I see is not optimized in that regarding memory consumption because  I load all frames in the whole video into a list and then apply prediction by a group of frames. as I understand it is the input data to be predicted. XShards, numpy array,s, and a list of NumPy arrays are supported. as here ( https://github.com/intel-analytics/BigDL/blob/main/python/orca/src/bigdl/orca/learn/openvino/estimator.py)  so if there is an optimized method to deal with videos data in the prediction method using  Openvino  Estimator that will be fine. also, I tried to use image_set from videoframesRDD as (zoo.feature.image.imageset.DistributedImageSet) type  but I faced  an error in the input data should be  NumPy arrays  [ValueError: Only XShards, Spark DataFrame, a numpy array, and a list of numpy arrays are supported as input data, but get RDD]

[Qiyuan Gong]

Hi,

   Thank you for feedback! :)

   Current BigDl doesn’t provide video APIs. As you mentioned, we can read video into images (list or numpy) then feed it to Orca OpenVINO. But, this solution is not memory efficient, especially for working process who is reading that video. A better solution is split that video into small videos by frame num or length, then read it by BigDL & Spark in parallel with several workers. That requires careful coding with opencv and BigDL. To help our customers better handle these datatype, we build cluster serving (only support openvino 2020.3.2 model) feature for parallel model serving. With Cluster Serving, we can use a simple python client to read frames from video one by one or by batch, then feed them into message queue. Cluster serving workers will fetch these frames in message queue, then predict them in parallel.

   Image_set is built with Spark RDD. You can use toDF to covert it into dataframe then pass it to Orca OpenVINO.

Have a nice day!

Qiyuan

[Qiyuan Gong]

Hi,

Thank you for feedback! :)

Current BigDl doesn’t provide video APIs. As you mentioned, we can read video into images (list or numpy) then feed it to Orca OpenVINO. But, this solution is not memory efficient, especially for working process who is reading that video. A better solution is split that video into small videos by frame num or length, then read it by BigDL & Spark in parallel with several workers. That requires careful coding with opencv and BigDL. To help our customers better handle these datatype, we build cluster serving (only support openvino 2020.3.2 model) feature for parallel model serving. With Cluster Serving, we can use a simple python client to read frames from video one by one or by batch, then feed them into message queue. Cluster serving workers will fetch these frames in message queue, then predict them in parallel. 

Image_set is built with Spark RDD. You can use toDF to covert it into dataframe then pass it to Orca OpenVINO.

Have a nice day!

Qiyuan Gong

You received this message because you are subscribed to a topic in the Google Groups "User Group for BigDL and Analytics Zoo" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/bigdl-user-group/J6fZ9B5OMfE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to bigdl-user-gro...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bigdl-user-group/4d0539b1-454d-439b-8bac-e742c2061887n%40googlegroups.com.