Can Numba handle int4, float4, etc.?

[Chris Uchytil]

I am trying to pass in a numpy int4 array using pyopencl, np.zeros([W*H], dtype=cl_array.vec.int4). When I pass this as an argument to the Kernal Numba spits this back out at me "ValueError: too many values to unpack". Can Numba handle int2, int4, etc or does it not allow this?

[Stanley Seibert]

We haven't ever tested with these custom dtypes, although I would have expected a structured dtype to work.  Can you show a little more detail about what you are trying to do?  (Which part is inside the Numba jit function, and which part is outside, etc.)

On Sat, Jul 30, 2016 at 1:07 PM, Chris Uchytil <chrisuch...@gmail.com> wrote:

I am trying to pass in a numpy int4 array using pyopencl, np.zeros([W*H], dtype=cl_array.vec.int4). When I pass this as an argument to the Kernal Numba spits this back out at me "ValueError: too many values to unpack". Can Numba handle int2, int4, etc or does it not allow this?

--
You received this message because you are subscribed to the Google Groups "Numba Public Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to numba-users...@continuum.io.
To post to this group, send email to numba...@continuum.io.
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/numba-users/2a19df95-982b-4a98-ae0d-e1a878d82fee%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.

[Chris Uchytil]

Thanks for the response. I'll give you a little background first. I am translating some CUDA C++ code over to Python. The code I am going to post is step 1 of the 2 step process. Step 2, which I haven't started yet, is taking the created array and sending it to PyOpenGL to create a display for the array of intensity values. The array values being calculated in the Kernel represent intensity of light at points certain distances away from the starting position. In the C code each value in the array is a uchar4, although I think int4 or float4 will do, and each of the values represent the intensity in blue, red, green, and alpha. What I have done to make it work is make a 2D array where the second dimension is a 4 length array taking the place of the 4 uchar values.

Here is the code that uses the 2D array. It works. I am hoping to translate the d_out[i][0] = intensity, d_out[i][1] = intensity, d_out[i][2] = 0, and d_out[i][3] = 255 into something like d_out[i].x = intensity, d_out[i].y = intensity, etc. and pass in a 1D array like this

out = np.zeros([W*H], dtype=cl_array.vec.int4) instead of np.zeros([W*H, 4]). When testing my code and passing in the int4 array I tried commenting out basically the entirety of the cuda.jit funtion just to see if I could pass it the int4 array. This is what was giving me the "ValueError: too many values to unpack" error.

import numpy as np

import math

from numba import jit, cuda, int32, float32

import pyopencl as cl

import pyopencl.array as cl_array

W = 500

H = 500

TX = 32

TY = 32

@cuda.jit(device = True)

def clip(n):

if n > 255:

n = 255

elif n < 0:

n = 0

return n

@cuda.jit

def distanceKernel(d_out, w, h, pos):

c = cuda.blockIdx.x*cuda.blockDim.x + cuda.threadIdx.x

r = cuda.blockIdx.y*cuda.blockDim.y + cuda.threadIdx.y

i = r*w + c

if c >= w or r >= h:

return

d = math.sqrt((c - pos[0]) * (c - pos[0]) + (r - pos[1]) * (r - pos[1]))

intensity = clip(255 - int(d))

d_out[i][0] = intensity

d_out[i][1] = intensity

d_out[i][2] = 0

d_out[i][3] = 255

def main():

pos = np.array([0.0,0.0])

blockSize = (TX, TY)

bx = (W + TX - 1)/TX

by = (W + TY - 1)/TY

gridSize = (bx, by)

out = np.zeros([W*H, 4])

d_out = cuda.to_device(out)

distanceKernel[blockSize, gridSize](d_out, W, H, pos)

out_final = d_out.copy_to_host()

print out_final

if __name__ == "__main__":

main()

[Siu Kwan Lam]

It's due to a small problem in the numpy dtype -> numba type logic.  I opened an issue here: https://github.com/numba/numba/issues/2020

--

You received this message because you are subscribed to the Google Groups "Numba Public Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to numba-users...@continuum.io.
To post to this group, send email to numba...@continuum.io.

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/numba-users/f6b2c501-19ee-4d9a-bf48-eb3e5997caef%40continuum.io.

For more options, visit https://groups.google.com/a/continuum.io/d/optout.

--

Siu Kwan Lam

Software Engineer

Continuum Analytics

[Chris Uchytil]

Thanks so much Siu!

[Chris Uchytil]

Hey Siu,

How often do you push updates live? I saw that you fixed the issue and I am just wondering if there is an "unstable" release with all the most recent updates or if it won't take long for a new update to be pushed. Thanks

[Stanley Seibert]

Our CI system automatically builds new conda packages whenever Github master is updated.  (This can take an hour or two since we need to build and test numba for ~40 different combinations of OS, Python and NumPy.)   They are uploaded to the "numba" channel, and can be installed with:

conda install -c numba numba

The PR fixing your bug is here:

https://github.com/numba/numba/pull/2021

Once that PR is merged, you should see new dev packages within a few hours.

On Mon, Aug 8, 2016 at 9:06 PM, Chris Uchytil <chrisuch...@gmail.com> wrote:

Hey Siu,

How often do you push updates live? I saw that you fixed the issue and I am just wondering if there is an "unstable" release with all the most recent updates or if it won't take long for a new update to be pushed. Thanks

--

You received this message because you are subscribed to the Google Groups "Numba Public Discussion - Public" group.

To unsubscribe from this group and stop receiving emails from it, send an email to numba-users+unsubscribe@continuum.io.

To post to this group, send email to numba...@continuum.io.

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/numba-users/ef6b3a03-198b-4b71-9a54-042921ccda56%40continuum.io.

[Chris Uchytil]

This is in response to the update to 0.28.1. I can pass in dtypes like cl_array.vec.uchar4 with no problem but as soon as I try to change any values within the array I am met with this error. I can post the code in as well if needed. I thought it might be a type error because if I do something like this "d_out[0][0] = 0" I get the LoweringError: Cannot cast int64 to Record error. I tried this "d_out[i][0] = np.uint8(0)" as well but I get this at the end instead Cannot resolve setitem: Record.

numba.errors.LoweringError: Caused By:

Traceback (most recent call last):

  File "/home/uchytilc/anaconda2/lib/python2.7/site-packages/numba/compiler.py", line 249, in run

    stage()

  File "/home/uchytilc/anaconda2/lib/python2.7/site-packages/numba/compiler.py", line 617, in stage_nopython_backend

    self._backend(lowerfn, objectmode=False)

  File "/home/uchytilc/anaconda2/lib/python2.7/site-packages/numba/compiler.py", line 572, in _backend

    lowered = lowerfn()

  File "/home/uchytilc/anaconda2/lib/python2.7/site-packages/numba/compiler.py", line 559, in backend_nopython_mode

    self.flags)

  File "/home/uchytilc/anaconda2/lib/python2.7/site-packages/numba/compiler.py", line 845, in native_lowering_stage

    lower.lower()

  File "/home/uchytilc/anaconda2/lib/python2.7/site-packages/numba/lowering.py", line 125, in lower

    self.lower_normal_function(self.fndesc)

  File "/home/uchytilc/anaconda2/lib/python2.7/site-packages/numba/lowering.py", line 160, in lower_normal_function

    entry_block_tail = self.lower_function_body()

  File "/home/uchytilc/anaconda2/lib/python2.7/site-packages/numba/lowering.py", line 185, in lower_function_body

    self.lower_block(block)

  File "/home/uchytilc/anaconda2/lib/python2.7/site-packages/numba/lowering.py", line 200, in lower_block

    self.lower_inst(inst)

  File "/home/uchytilc/anaconda2/lib/python2.7/contextlib.py", line 35, in __exit__

    self.gen.throw(type, value, traceback)

  File "/home/uchytilc/anaconda2/lib/python2.7/site-packages/numba/errors.py", line 249, in new_error_context

    six.reraise(type(newerr), newerr, sys.exc_info()[2])

  File "/home/uchytilc/anaconda2/lib/python2.7/site-packages/numba/errors.py", line 243, in new_error_context

    yield

  File "/home/uchytilc/anaconda2/lib/python2.7/site-packages/numba/lowering.py", line 200, in lower_block

    self.lower_inst(inst)

  File "/home/uchytilc/anaconda2/lib/python2.7/site-packages/numba/lowering.py", line 292, in lower_inst

    return self.lower_setitem(inst.target, inst.index, inst.value, signature)

  File "/home/uchytilc/anaconda2/lib/python2.7/site-packages/numba/lowering.py", line 360, in lower_setitem

    return impl(self.builder, (target, index, value))

  File "/home/uchytilc/anaconda2/lib/python2.7/site-packages/numba/targets/base.py", line 959, in __call__

    return self._imp(self._context, builder, self._sig, args)

  File "/home/uchytilc/anaconda2/lib/python2.7/site-packages/numba/targets/arrayobj.py", line 481, in setitem_array

    val = context.cast(builder, val, valty, aryty.dtype)

  File "/home/uchytilc/anaconda2/lib/python2.7/site-packages/numba/targets/base.py", line 627, in cast

    "Cannot cast %s to %s: %s" % (fromty, toty, val))

LoweringError: Cannot cast int64 to Record([(('x', 's0'), '|u1'), (('y', 's1'), '|u1'), (('z', 's2'), '|u1'), (('w', 's3'), '|u1')]): %".261" = load i64, i64* %"$const106.1"

File "dist.py", line 34

[1] During: lowering "d_out[i] = $const106.1" at dist.py (34)

Failed at nopython (nopython mode backend)

Cannot cast int64 to Record([(('x', 's0'), '|u1'), (('y', 's1'), '|u1'), (('z', 's2'), '|u1'), (('w', 's3'), '|u1')]): %".261" = load i64, i64* %"$const106.1"

File "dist.py", line 34

[1] During: lowering "d_out[i] = $const106.1" at dist.py (34)

[Stanley Seibert]

OK, thanks for the bug report.  Can you open an issue?

https://github.com/numba/numba/issues

--
You received this message because you are subscribed to the Google Groups "Numba Public Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to numba-users+unsubscribe@continuum.io.
To post to this group, send email to numba...@continuum.io.

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/numba-users/fb0a9f62-7d93-40c3-b5e4-68f59b84bd5a%40continuum.io.

[Chris Uchytil]

Will do.

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/numba-users/CADv3RKT25MLWesUvC3r-JunFTGArjykJnPSr2h3rAuQzwDVZUQ%40mail.gmail.com.

[Stanley Seibert]

Awesome, thanks!

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/numba-users/CAAQHW3kUQR_FpQanwXwkZG2ypO4DgUrTWCF2eF-VRoiE0TSyHQ%40mail.gmail.com.