Detect Multiple Features on One API Call

Charles Caspar

unread,

Jun 8, 2019, 2:35:41 PM6/8/19

to cloud-videointel-discuss

Hi,

Is it possible to detect multiple features on one video at the same time? For example, I want to run a video through the API and perform label detection, text detection, shot change detection, and speech transcription simultaneously, without processing the same video 4 times. Previously, I had been able to pass these 3 arguments as features, and run one API call per video. (Sample code below)

features = [videointelligence.enums.Feature.LABEL_DETECTION,

videointelligence.enums.Feature.SHOT_CHANGE_DETECTION,

videointelligence.enums.Feature.TEXT_DETECTION]

operation = video_client.annotate_video('filepath', features=features)

However, with the current documentation, I see that each feature type has a distinct 'video_context' and 'config', thus requiring a separate 'annotate_video' method for each feature. Is it possible to collate these into one method or do I have to run each feature as a distinct API call?

-------------------------------------------------------

essenceglobal.com
Facebook • Twitter • YouTube • Instagram

Lei Yang

unread,

Jun 8, 2019, 5:18:31 PM6/8/19

to Charles Caspar, cloud-videointel-discuss

Hi Charles,

You can run multiple features in one request. The needed features's config can be put into one video_context (configs are not exclusive).

e.g.

speech_config = videointelligence.types.SpeechTranscriptionConfig(
    language_code='en-US',
    enable_automatic_punctuation=True)

label_config = videointelligence.types.LabelDetectionConfig(stationary_camera=True)

...

video_context = videointelligence.types.VideoContext(
    speech_transcription_config=speech_config, label_detection_config=label_config,...)

Thanks!

--
© 2018 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043

Email preferences: You received this email because you signed up for the Google Cloud Video Intelligence API Discussion Google Group (cloud-videoi...@googlegroups.com) to participate in discussions with other members of the Google Cloud Video Intelligence API community and the Google Cloud Video Intelligence API Team.
---
You received this message because you are subscribed to the Google Groups "cloud-videointel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-videointel-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-videointel-discuss/54283e4a-7189-456b-b105-1183e26cf055%40googlegroups.com.

--

Lei Yang (杨蕾) |

Software Engineer, Play |

yanga...@google.com |

650-214-4097

houman azemati

unread,

Sep 11, 2019, 5:46:16 PM9/11/19

to cloud-videointel-discuss

How do I use the context in the case for video streaming? (StreamingAnnotateVideoRequest does not take a context as an argument).

Lei Yang

unread,

Sep 11, 2019, 6:53:27 PM9/11/19

to houman azemati, cloud-videointel-discuss

Hi Houman,

There is StreamingVideoConfig in StreamingAnnotateVideoRequest.

And each stream in video Streaming API only allow ONE feature type.

Thanks & Regards,

--

© 2018 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043

Email preferences: You received this email because you signed up for the Google Cloud Video Intelligence API Discussion Google Group (cloud-videoi...@googlegroups.com) to participate in discussions with other members of the Google Cloud Video Intelligence API community and the Google Cloud Video Intelligence API Team.
---
You received this message because you are subscribed to the Google Groups "cloud-videointel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-videointel-d...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-videointel-discuss/aeff2420-9710-4361-beed-4ab7b46c2646%40googlegroups.com.

Charles Caspar

unread,

Nov 4, 2019, 3:34:54 PM11/4/19

to cloud-videointel-discuss

Hi Lei,

I have found that Speech Transcription is selectively not working when I try to process numerous features at once. In the first block of code, I did not process Text Detection. As a result the Speech Transcription was successful. In the second block of code, my only change was to add Text Detection. After this change, the Speech Transcription returns an empty result. Do you know what is causing this?

Code block 1:

# initialize videointelligence client
video_client = videointelligence.VideoIntelligenceServiceClient(credentials=credentialsFromVault)


# declare features to process
features = [videointelligence.enums.Feature.LABEL_DETECTION, videointelligence.enums.Feature.SHOT_CHANGE_DETECTION,
            videointelligence.enums.Feature.SPEECH_TRANSCRIPTION]


# setup configs
label_config = videointelligence.types.LabelDetectionConfig()
shot_config = videointelligence.types.ShotChangeDetectionConfig()
speech_config = videointelligence.types.SpeechTranscriptionConfig(language_code='en-US', enable_automatic_punctuation=True)


# run file through videointelligence client
video_context = videointelligence.types.VideoContext(label_detection_config=label_config,
                                                     shot_change_detection_config=shot_config,
                                                     speech_transcription_config=speech_config)


print('\nProcessing video for labels, shot changes, text, and speech.')


operation = video_client.annotate_video('gs://video_path/'+x, features=features, video_context=video_context)
result = operation.result(timeout=1500)
print('Finished processing '+ x)

Code block 2:

# initialize videointelligence client
video_client = videointelligence.VideoIntelligenceServiceClient(credentials=credentialsFromVault)


# declare features to process
features = [videointelligence.enums.Feature.LABEL_DETECTION, videointelligence.enums.Feature.SHOT_CHANGE_DETECTION,
            videointelligence.enums.Feature.SPEECH_TRANSCRIPTION, videointelligence.enums.Feature.TEXT_DETECTION]


# setup configs
label_config = videointelligence.types.LabelDetectionConfig()
shot_config = videointelligence.types.ShotChangeDetectionConfig()
text_config = videointelligence.types.TextDetectionConfig()
speech_config = videointelligence.types.SpeechTranscriptionConfig(language_code='en-US', enable_automatic_punctuation=True)


# run file through videointelligence client
video_context = videointelligence.types.VideoContext(label_detection_config=label_config,
                                                     shot_change_detection_config=shot_config,
                                                     speech_transcription_config=speech_config,
                                                     text_detection_config=text_config)




print('\nProcessing video for labels, shot changes, text, and speech.')


operation = video_client.annotate_video('gs://video_path/'+x, features=features, video_context=video_context)
result = operation.result(timeout=1500)
print('Finished processing '+ x)

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-videointel-discuss+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-videointel-discuss/54283e4a-7189-456b-b105-1183e26cf055%40googlegroups.com.

Lei Yang

unread,

Nov 6, 2019, 2:51:29 PM11/6/19

to Charles Caspar, cloud-videointel-discuss

Resolved and discussed with Charles off this thread.

Also reply to the group about merged results across features:

Due to each feature, especially detected video and audio length could be different, the results of different features may not be grouped into one annotation_results,

i.e. you could have multiple annotation_results in one response.

The order of items in annotation_results is not and does not need to be deterministic but related with processing speed of features.

Thanks & Regards,

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-videointel-d...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-videointel-discuss/54283e4a-7189-456b-b105-1183e26cf055%40googlegroups.com.

--

Lei Yang (杨蕾) | Software Engineer, Play | yanga...@google.com | 650-214-4097

-------------------------------------------------------
essenceglobal.com
Facebook • Twitter • YouTube • Instagram

--
© 2018 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043

Email preferences: You received this email because you signed up for the Google Cloud Video Intelligence API Discussion Google Group (cloud-videoi...@googlegroups.com) to participate in discussions with other members of the Google Cloud Video Intelligence API community and the Google Cloud Video Intelligence API Team.
---
You received this message because you are subscribed to the Google Groups "cloud-videointel-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-videointel-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-videointel-discuss/c83b3d4f-b358-40ab-85fc-568533bd5549%40googlegroups.com.

Reply all

Reply to author

Forward