Bounding Box with Text Annotation

[Yong Tang]

Hi,

In tensorflow-io we recently merged a PR (https://github.com/tensorflow/io/pull/254) which adds an option to provide a text annotation when you draw bounding boxes. This might be useful for object tracking and computer vision.

The API tensorflow_io.image.draw_bounding_boxes is compatible with tensorflow main repo's tf.image.draw_bounding_boxes, except you could provide a Text string for annotation.

Note 

1) In order to keep a consistent view we didn't use system Font. Instead we use Google's Open Sans font (Apache License 2.0).

2) This API is currently only available in tensorflow-io's nightly build. It will be in tensorflow-io's next release v0.7.0.

The implementation uses open source FreeType library (BSD style FreeType License). I am still trying to get familiar in FreeType's text rendering mechanisms so there might be some edge cases that have not been covered.

After sometime if the implementation is stable enough we could open a PR to tensorflow's main repo as well.

Below is an example of the image with tensorflow_io.image.draw_bounding_boxes:

https://user-images.githubusercontent.com/6932348/59042692-c4e9ae00-882f-11e9-9d46-05184400985a.png

The following is the code snippet.

import tensorflow as tf

import tensorflow_io.image as image_io

with open("Sagrada_Familia.jpg", "rb") as f:

  jpeg_contents = f.read()

p = tf.image.decode_jpeg(jpeg_contents) 

p = tf.expand_dims(p, 0)

p = tf.image.convert_image_dtype(p, tf.float32)

bb = [[[0.12, 0.10, 0.85, 0.9]]]

#p = tf.image.draw_bounding_boxes(p, bb) # No text with bf's draw_bounding_boxes

p = image_io.draw_bounding_boxes(p, bb, ["Sagrada Familia"]) # With text in tensorflow-io.

p = tf.squeeze(p, [0])

p = tf.image.convert_image_dtype(p, tf.uint8)

p = tf.image.encode_png(p)

png_contents = tf.Session().run(p)

with open("p.png", "wb") as f:

  f.write(png_contents)

Thanks

Yong