Numpy advanced indexing with 1D and 2D index arrays in one function

[J Ello]

Hi everyone,

I'm currently trying to write a numba function for solving my array
equations. Everything works fine except that I at one point have to index a
1D numpy array either with a 1D or a 2D numpy index array. I tried checking
for this by using

to get the standard advanced indexing function for 1D index arrays and
otherwise using slicing to get a 1D view of the 2D index array to again be
able to do advanced indexing.
But when trying to pass a 2D index array to my function I always get the
error:

*numba.errors.InternalError: unsupported array index type array(int32, 2d,
C) in [array(int32, 2d, C)]*And when trying to pass a 1D index array to my
function I always get this error:

*numba.errors.InternalError: cannot index array(int32, 1d, C) with 2
indices: (slice<a:b>, int64)*My short example code snippet looks like this:

Unluckily this does not work, since the compiler seems to check if all
parts of the function work for the given pidx array, right?
This code snippet is part of a much larger function which is called in many
million iterations, but the arrays used for this function are fixed and
constant for all iterations and will be passed to the larger function with
each call.
How can I still use indexing for both kinds of arrays? Writing separate
functions is not preferred, but currently my way to make it work. Is there
any other way?
And is there a time schedule for the implementation of general 2D advanced
indexing?

Thanks in advance!

Sincerely,
Johannes

[J Ello]

I really don't know what happened here. Posting this question took about 30 minutes from clicking on "Post question" until it was shown and it completely cut out my code. And in this post I can't activate the code environment. Don't know what google did with the backend to their groups... Thus I ll have to post the code in normal formatting.

So here comes the short code snippet after "I tried checking for this by using":
if idx_array.ndim == 1:


And my larger code snippet after "My short example code snippet looks like this:"

import numpy as np
import numba

arr = np.arange(100, 201)  # result array, always 1D
vals = np.arange(101)  # source array to index with either 1D or 2D index array, always 1D
pid1d = np.array([5, 17, 23, 96])  # index array for result array in 1D case, always 1D
pid2d = np.array([5, 17, 23, 96, 18, 55])  # index array for result array in 2D case, ALWAYS 1D
pidx1d = np.array([0, 12, 50, 100])  # index array for source array in 1D case, always 1D
pidx2d = np.array([[0, 0], [0, 12], [0, 50], [0, 100], [0, 12], [0, 50]])  # index array for source array in 2D, ALWAYS 2D!


@numba.jit(nopython=True)
def upd_arr(arr, pid, vals, pidx):
    # if part has no double ports at cells:
    if pidx.ndim == 1:
        arr[pid] = vals[pidx]
    elif pidx.ndim == 2:
        # preferred indexing way by slicing:
        arr[pid] = vals[pidx[:, 1]]
        # alternative indexing way by for loop:
#        for i in range(pidx.shape[0]):
#            arr[pid[i]] = vals[pidx[i, 1]]

    return arr

# 1d test case:
res1d = upd_arr(arr, pid1d, vals, pidx1d)
# getting error: numba.errors.InternalError: cannot index array(int32, 1d, C) with 2 indices: (slice<a:b>, int64)# 2d test case
res2d = upd_arr(arr, pid2d, vals, pidx2d)
# getting error: numba.errors.InternalError: unsupported array index type array(int32, 2d, C) in [array(int32, 2d, C)]



Sorry for the inconvenience!

[Stanley Seibert]

Hi,

I'm not sure why Google is handling your post in a strange way, but to answer your question:

Numba treats the number of dimensions in the NumPy array as part of the type specialization, and unfortunately our current type inference system requires a consistent type throughout the function body to generate nopython mode code.  However, your use case is precisely why we added the generated_jit decorator (borrowed from Julia):

http://numba.pydata.org/numba-doc/latest/user/generated-jit.html

You can write a Python function that returns one implementation if the array is 1D, and a different implementation if the array is 2D.

--
You received this message because you are subscribed to the Google Groups "Numba Public Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to numba-users+unsubscribe@continuum.io.
To post to this group, send email to numba...@continuum.io.
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/numba-users/1aabebc3-8cf3-4530-95f4-fdd0457bb338%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.

[J Ello]

Hi,

thanks for your help! I'll look into generated_jit and try to solve my problem!

To unsubscribe from this group and stop receiving emails from it, send an email to numba-users...@continuum.io.