RuntimeError: bob.ip.base.LBPTop - cannot process LBPTop: C++ exception caught:

[simux]

Got this error while trying to extract lbptop features from video

RuntimeError: bob.ip.base.LBPTop - cannot process LBPTop: C++ exception caught: 'time parameter in direction XY (1) has to be 118'

Any help?

[Tiago Freitas Pereira]

Hi,

Could you please paste a small example on how to reproduce the issue?

Thanks

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+unsubscribe@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/
---
You received this message because you are subscribed to the Google Groups "bob-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Tiago

[simux]

import bob.ip.base
import bob.io.video
import numpy

input = bob.io.video.reader('./video.mp4')

vin = input.load()

nFrames = vin.shape[0]

grayFrames = numpy.zeros(shape=(nFrames,vin.shape[2],vin.shape[3]))

for i in range(nFrames):

    grayFrames[i] = bob.ip.color.rgb_to_gray(vin[i,:,:,:])

    

lbp_xy = bob.ip.base.LBP(8,1)

lbp_xt = bob.ip.base.LBP(8,1)

lbp_yt = bob.ip.base.LBP(8,1)

lbptop = bob.ip.base.LBPTop(lbp_xy, lbp_xt, lbp_yt)

t = int(max(lbp_xt.radius, lbp_yt.radius))

w = int(grayFrames.shape[1] - lbp_xy.radii[0]*2)

h = int(grayFrames.shape[2] - lbp_xy.radii[1]*2)

output_xy = numpy.zeros((t,w,h),dtype='uint16')

output_xt = numpy.zeros((t,w,h),dtype='uint16')

output_yt = numpy.zeros((t,w,h),dtype='uint16')

lbptop(grayFrames,output_xy, output_xt, output_yt)

print(output_xy)

print(output_xt)

print(output_yt)

[Tiago Freitas Pereira]

Hi Simux, 

Thanks to make the things clearer by posting an example.

I just patched it to make it work.

I replaced your video by a random numpy array.

I the issue was the allocation of the output array.

In the example we have a video of 100x320x480 (gray scaled).

With a LBP of radius 1 intersecting 3 planes, we need an output of 98x318x478.

Follow bellow a working example

Cheers

```

import bob.ip.base
import bob.io.video
import bob.ip.color
import numpy

#input = bob.io.video.reader('./video.mp4')
#vin = input.load()
vin = numpy.random.rand(100, 3, 320, 480)

nFrames = vin.shape[0]
grayFrames = numpy.zeros(shape=(nFrames, vin.shape[2], vin.shape[3]))

for i in range(nFrames):
    grayFrames[i] = bob.ip.color.rgb_to_gray(vin[i, :, :, :])


lbp_xy = bob.ip.base.LBP(8, 1)
lbp_xt = bob.ip.base.LBP(8, 1)
lbp_yt = bob.ip.base.LBP(8, 1)
lbptop = bob.ip.base.LBPTop(lbp_xy, lbp_xt, lbp_yt)

t = nFrames - 2*int(max(lbp_xt.radius, lbp_yt.radius))

w = int(grayFrames.shape[1] - lbp_xy.radii[0] * 2)
h = int(grayFrames.shape[2] - lbp_xy.radii[1] * 2)
output_xy = numpy.zeros((t, w, h), dtype='uint16')
output_xt = numpy.zeros((t, w, h), dtype='uint16')
output_yt = numpy.zeros((t, w, h), dtype='uint16')

lbptop(grayFrames, output_xy, output_xt, output_yt)
print(output_xy)
print(output_xt)
print(output_yt)

```

--

-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+unsubscribe@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/
---
You received this message because you are subscribed to the Google Groups "bob-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Tiago

[salma kasraoui]

Hi Tango,

Thank you for your help, but I still have problem.

When I run the code above it does work. However when I add my image sequence, I am having this error:

TypeError: method only accepts 3-dimensional arrays with shape (3, height, width), not (720, 1280, 3)

Can you please tell me how can I convert my data such that it matches the required shape.

Thank you.

below my code:

import cv2

import glob

import os

import bob.ip.base

import bob.io.video 

import bob.io.matlab 

import numpy

ImageFormat="*.jpg"

mypath='./Sequence/'

directory=os.path.join(mypath, ImageFormat)

ImageFiles=glob.glob(directory)

volData = []

for el in ImageFiles:

        frame = cv2.imread(el, cv2.IMREAD_ANYCOLOR)

        volData.append(frame)

print volData

vin = numpy.asarray(volData)

nFrames = vin.shape[0]

grayFrames = numpy.zeros(shape=(nFrames, vin.shape[2], vin.shape[3]))

for i in range(nFrames):

    grayFrames[i] = bob.ip.color.rgb_to_gray(vin[i, :, :, :])

lbp_xy = bob.ip.base.LBP(8, 1)

lbp_xt = bob.ip.base.LBP(8, 1)

lbp_yt = bob.ip.base.LBP(8, 1)

lbptop = bob.ip.base.LBPTop(lbp_xy, lbp_xt, lbp_yt)

t = nFrames - 2*int(max(lbp_xt.radius, lbp_yt.radius))

w = int(grayFrames.shape[1] - lbp_xy.radii[0] * 2)

h = int(grayFrames.shape[2] - lbp_xy.radii[1] * 2)

output_xy = numpy.zeros((t, w, h), dtype='uint16')

output_xt = numpy.zeros((t, w, h), dtype='uint16')

output_yt = numpy.zeros((t, w, h), dtype='uint16')

lbptop(grayFrames, output_xy, output_xt, output_yt)

print(output_xy)

print(output_xt)

print(output_yt)

ᐧ

You received this message because you are subscribed to a topic in the Google Groups "bob-devel" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/bob-devel/P8RpbRKAySg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to bob-devel+unsubscribe@googlegroups.com.

[Tiago Freitas Pereira]

Hi Salma,

Would be nicer to have the whole trace to analyse the issue. Could you please post it next time  ?

I think your issue is probably here:

`grayFrames[i] = bob.ip.color.rgb_to_gray(vin[i, :, :, :])`

I've seen that you are reading the video with OpenCV.
OpenCV and Bob loads color images differently.
Bob does [CHANNELS, HEIGHT, WIDTH ] and OpenCV [HEIGHT WIDTH CHANNELS].

If you want to use bob.ip.color to convert from rgb to gray, please make sure that your input is proper set ([CHANNELS, HEIGHT, WIDTH ])

 

Cheers

Tiago

[Manuel Günther]

In general, I would discourage to use OpenCV for image IO in python. I have seen many weird issues with OpenCV's image loading, for example that it happily returns None for a non-existing image without providing an error message or raising an exception. Also, for some images, cv2.imread just hangs in an endless loop.

Since you are using other bob functionality, I would encourage to use the bob.io.image.load instead. In this way, the images are read in exactly the format that other bob functions will use them later on.

Manuel

[salma kasraoui]

Hi,

Thank you! It is working now. However I don't see lbptop code returned by the class lbptop. 

ᐧ

--

[Tiago Freitas Pereira]

Hi Salma,

Could you please provide more information (code, error message, etc..)?

Thanks

You received this message because you are subscribed to the Google Groups "bob-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Tiago

[salma kasraoui]

Hi Tiago,

I meant the class lbptop computes lbptop on all the image sequence? So to have the lbptop, I only concatenate the histograms of output_xy, output_xt and output_yt respectively?

Thank you

ᐧ

[salma kasraoui]

Hi Tiago,

I want to perform lbptop on blocks in an image sequence but I have problem in subdivising the sequence into 4x4 blocks.

I tried this:

bloc_1 = numpy.zeros(shape=(nFrames, vin.shape[2]/2, vin.shape[3]/2))

for ii in range(nFrames):

    bloc_1[ii] = bob.ip.color.rgb_to_gray(vin[ii, :, 0:vin.shape[2]/2, 0:vin.shape[3]/2])

    

    

bloc_2 = numpy.zeros(shape=(nFrames, vin.shape[2]/2, vin.shape[3]/2))

for jj in range(nFrames):

    bloc_2[jj] = bob.ip.color.rgb_to_gray(vin[jj, :, vin.shape[2]/2:vin.shape[2], 0:vin.shape[3]/2])

bloc_3 = numpy.zeros(shape=(nFrames, vin.shape[2]/2, vin.shape[3]/2))

for kk in range(nFrames):

    bloc_3[kk] = bob.ip.color.rgb_to_gray(vin[kk, :, 0:vin.shape[2]/2, vin.shape[3]/2:vin.shape[3]])

bloc_4 = numpy.zeros(shape=(nFrames, vin.shape[2]/2, vin.shape[3]/2))

for ll in range(nFrames):

    bloc_4[ll] = bloc_1[ii] = bob.ip.color.rgb_to_gray(vin[ii, :, vin.shape[2]/2:vin.shape[2], vin.shape[3]/2:vin.shape[3]])

but I got this error:

bloc_2[jj] = bob.ip.color.rgb_to_gray(vin[jj, :, vin.shape[2]/2:vin.shape[2], 0:vin.shape[3]/2])

ValueError: could not broadcast input array from shape (2,100) into shape (3,200)

Can you please help me?

Thank you!

ᐧ

[Tiago Freitas Pereira]

Hi Salma,

Could you please provide an executable example?

It's not necessary to attach the video; vin can be initialized like this:

vin = numpy.random.rand(n_frames, 3, width, height)

Thanks

[salma kasraoui]

Hi Tiago,

Please find attached an executable example on how to divide the sequence.

Thank you!

ᐧ

[Tiago Freitas Pereira]

Hey,

This is not executable. 
What are the values for (nFrames, width, height)?

Try to set to the same values that you have in your video.

This also may help you debug the thing yourself.

[salma kasraoui]

Hi Tiago,

I have images of size 200x200

nFrames = 25

[Tiago Freitas Pereira]

Hey,

There is an issue with the code you sent.

you are doing

bloc_2 = numpy.zeros(shape=(nFrames, vin.shape[2], vin.shape[3]))

  instead of:

 bloc_2 = numpy.zeros(shape=(nFrames, vin.shape[2]/2, vin.shape[3]/2))

[salma kasraoui]

Hi Tiago,

I tried bloc_2 = numpy.zeros(shape=(nFrames, vin.shape[2]/2, vin.shape[3]/2)) first and it doesn't work.

Then I tried bloc_2 = numpy.zeros(shape=(nFrames, vin.shape[2], vin.shape[3])) and I got the same error.

For a single image I can process the division. But with a sequence I am blocked!

Regards.

[Tiago Freitas Pereira]

Hey,

Have a look in this test code.

[salma kasraoui]

Hi Tiago,

I am having the same error! i think there is a problem in loading data. Can you please see the attached file and tell me if reading data is written in a correct form.

Thank you!