Mutable structs

[Jim Pivarski]

(Same project as before, another question.)

I've followed the "Extending Numba" documentation to make the example Interval type. I'd like to do the same for a mutable struct, ultimately to make something that looks like an iterator. Old versions of Numba had mutable structs as first-class citizens, but that was removed. I read on numba-nextgen-docs (which I assume are this-gen) that mutable structs on the stack are discouraged because their semantics don't follow Python's:

x = mutable() # stack allocate
y = x         # copy x into y
y.value = 1   # update y.value, which does not affect x.value

But actually, this behavior would be okay because I'm implementing something like a cursor over data. A user would want assignment to be a copy.

Each cursor immutably points to two arrays and carries two (potentially) mutable indexes. Here are two examples, xss (a list of lists of x) and xs (a list of x):

@numba.njit
def getxss(data, size, dataindex, sizeindex):
    sizeindex += 1
    for i in range(size[sizeindex - 1]):
        yield dataindex, sizeindex

        length = size[sizeindex]
        sizeindex += 1
        for j in range(length):
            dataindex += 1

@numba.njit
def getxs(data, size, dataindex, sizeindex):
    for i in range(size[sizeindex]):
        yield dataindex, sizeindex
        dataindex += 1

For now, they export their state by yielding tuples of dataindex and sizeindex and update as generators. It solves my problem, but scales poorly. Another version of this with inlined code is five times faster on my test example.

Furthermore, this little test:

@numba.njit
def iterator(numEntries):
    for n in range(numEntries):
        yield n

@numba.njit
def useiterator(numEntries):
    out = 0
    for n in iterator(numEntries):
        out += n
    return out

@numba.njit
def useloop(numEntries):
    out = 0
    for n in range(numEntries):
        out += n
    return out

shows that "useloop" is a million times faster than "useiterator". Clearly, iterators aren't being inlined, but it's much worse than that.

One way I could go would be to modify the AST of the user's function, instrumenting it with inline versions of my iterators before giving the function to Numba. I don't like that solution because it duplicates work that Numba does and users would have to use two decorators in the right order. Maybe I could integrate it as a Numba rewrite rule, but then I'd have to understand and depend on Numba's internal IR.

Any suggestions?

-- Jim

[Jim Pivarski]

Numba used to have pointer tools like addressof. Is there any way I could pass my struct to a function as a pointer and then use that to modify it in place?

I see that there's CPointer and the possibility of calling functions that have been compiled in C. I could use that as my escape hatch, but I'd rather do it all in Numba for more cross-platform portability.

Thanks,

-- Jim

[Jim Pivarski]

A third thought: can I write C code directly in Numba (i.e. does it provide access to Clang) or maybe write the equivalent in LLVM IR (which I'd have to learn, but I'm assuming it's similar to writing C)?

-- Jim

[Jim Pivarski]

I think I'm 99% of the way there. (Ignore the other avenues of escaping through C and CPointer.)

I found that the

numba.extending.make_attribute_wrapper(IntervalType, "hi", "hi")

line in the interval example can be expanded to

@numba.extending.infer_getattr
class StructAttribute(numba.typing.templates.AttributeTemplate):
    key = IntervalType
    def generic_resolve(self, typ, attr):
        if attr == "hi":
            return numba.types.float64

@numba.extending.lower_getattr(IntervalType, "hi")
def struct_getattr_impl(context, builder, typ, val):
    val = numba.cgutils.create_struct_proxy(typ)(context, builder, value=val)
    return numba.targets.imputils.impl_ret_borrowed(context, builder, numba.types.float64, val.hi)

and there's a "lower_setattr" decorator to go along with "lower_getattr". I couldn't find any examples of how to use this, but I think it's something like

@numba.extending.lower_setattr(IntervalType, "hi")
def struct_setattr_impl(context, builder, sig, args):
    assert isinstance(sig.args[0], IntervalType)
    assert isinstance(sig.args[1], numba.types.Float)
    interval, newvalue = args
    val = numba.cgutils.create_struct_proxy(sig.args[0])(context, builder, value=interval)
    ptr = val._get_ptr_by_name("hi")
    return builder.store(newvalue, ptr)

That is, I can check the setattr argument types through "sig" and have "args" as llvmlite instructions. The "builder.store" call creates an llvmlite instruction to update the value of "hi" in the interval object.

However, my new instruction is not inserted into the generated function. I can look at the generated code with

@numba.njit
def doit10():
    interval = Interval(4.4, 6.4)
    interval.hi = 99.999
    return interval.width

cres = doit10.overloads.values()[0]
print(cres.library.get_llvm_str())

and see that the first "define" function (the one with the converted code, as opposed to the one that boxes/unboxes Python) is exactly the same whether or not the assignment statement is included.

So clearly, I'm making the instruction, but not putting it into the stream of generated code. How do I do that last step?

Thanks,

-- Jim

[joler...@gmail.com]

I think "numba nextgen" was flypy which last I saw on this mailing list was effectively shelved.

I'm also interested in custom iterators but it seems like a bit difficult at the moment to achieve.

[Jim Pivarski]

I meant it when I said that I'm 99% of the way there. The setattr is clearly being called (witnessed by print statements) and I'm generating LLVM IR that type-checks (it complained when I put a float where a pointer belongs).

I just don't know where to put the result— somewhere in "context"? I had thought that using the "builder" to make the IR would automatically put it into the output code, but no. Returning it from the function didn't work, either. This would be very easy for someone who's familiar with IR generation.

If someone can help me past this blocker, I'll post the end-to-end solution as an illusive example for others.

Thanks!
Jim

[joler...@gmail.com]

The code isn't clear to me. Is ut just overloading set atr? Wouldn't it be easier if we could just access it as a generic function?

also have you confirmed interval is stack allocated?

[Jim Pivarski]

On Monday, May 29, 2017 at 7:41:42 PM UTC-5, joler...@gmail.com wrote:

The code isn't clear to me. Is ut just overloading set atr? Wouldn't it be  easier if we could just access it as a generic function?

I'm overloading __setattr__, but I'd be happy doing it as a generic function instead. However, I don't know how to do that. The only example I have of defining methods on extension types use the high-level API, and that won't let me modify data members of my new struct (the IntervalType that has a data model inheriting from StructModel).

My problem is that there's a make_attribute_wrapper for read-only attributes of a StructModel, but no equivalent for read-write attributes. Whether I write a function with a simple @numba.njit or with the high-level API, it won't let me assign to those attributes. If I had any way to assign to those attributes— even if it was a function with an ugly name like __myinterval_setattr_atpos(interval, 1, 3.14)— then I could write a high-level function around it to implement my iterator and hide that ugliness.

also have you confirmed interval is stack allocated?

I don't know how to check that. But if it really is a struct in the C sense, it should be. I don't see any incref/decref in the generated IR.

[joler...@gmail.com]

so can you now add methods and functions to the struct?

anyway way to make the attributes generic to whatever is passed in?

[Jim Pivarski]

I can add methods to the struct using the high-level API (numba.extending.overload_method), but this does not let me set struct attributes.

Here's a complete, working example, made from the code examples in the Numba documentation:

import numba

class Interval(object):
    def __init__(self, lo, hi):
        self.lo = lo
        self.hi = hi

    def __repr__(self):
        return "Interval({}, {})".format(self.lo, self.hi)

class IntervalType(numba.types.Type):
    def __init__(self):
        super(IntervalType, self).__init__(name="Interval")

intervaltype = IntervalType()

@numba.extending.typeof_impl.register(Interval)
def typeof_index(val, c):
    return intervaltype

@numba.extending.type_callable(Interval)
def type_interval(context):
    def typer(lo, hi):
        if isinstance(lo, numba.types.Float) and isinstance(hi, numba.types.Float):
            return intervaltype
    return typer

@numba.extending.register_model(IntervalType)
class IntervalModel(numba.extending.models.StructModel):
    def __init__(self, dmm, fe_type):
        members = [("lo", numba.types.float64), ("hi", numba.types.float64)]
        super(IntervalModel, self).__init__(dmm, fe_type, members)

numba.extending.make_attribute_wrapper(IntervalType, "lo", "lo")

numba.extending.make_attribute_wrapper(IntervalType, "hi", "hi")

@numba.extending.lower_builtin(Interval, numba.types.Float, numba.types.Float)
def impl_interval(context, builder, sig, args):
    typ = sig.return_type
    lo, hi = args
    interval = numba.cgutils.create_struct_proxy(typ)(context, builder)
    interval.lo = lo
    interval.hi = hi
    return interval._getvalue()

@numba.extending.unbox(IntervalType)
def unbox_interval(typ, obj, c):
    lo_obj = c.pyapi.object_getattr_string(obj, "lo")
    hi_obj = c.pyapi.object_getattr_string(obj, "hi")
    interval = numba.cgutils.create_struct_proxy(typ)(c.context, c.builder)
    interval.lo = c.pyapi.float_as_double(lo_obj)
    interval.hi = c.pyapi.float_as_double(hi_obj)
    c.pyapi.decref(lo_obj)
    c.pyapi.decref(hi_obj)
    is_error = numba.cgutils.is_not_null(c.builder, c.pyapi.err_occurred())
    return numba.extending.NativeValue(interval._getvalue(), is_error=is_error)

@numba.extending.box(IntervalType)
def box_interval(typ, val, c):
    interval = numba.cgutils.create_struct_proxy(typ)(c.context, c.builder, value=val)
    lo_obj = c.pyapi.float_from_double(interval.lo)
    hi_obj = c.pyapi.float_from_double(interval.hi)
    class_obj = c.pyapi.unserialize(c.pyapi.serialize_object(Interval))
    res = c.pyapi.call_function_objargs(class_obj, (lo_obj, hi_obj))
    c.pyapi.decref(lo_obj)
    c.pyapi.decref(hi_obj)
    c.pyapi.decref(class_obj)
    return res

@numba.njit
def test1():
    interval = Interval(4.4, 6.4)
    return interval.hi
print(test1())

@numba.njit
def test2():

    interval = Interval(4.4, 6.4)
    interval.hi = 99.999

    return interval.hi
print(test2())

@numba.extending.overload_method(IntervalType, "onlyget")
def interval_onlyget(interval, arg):
    if isinstance(arg, numba.types.Float):
        def onlyget_impl(interval, arg):
            return interval.hi + arg
        return onlyget_impl

@numba.njit
def test3():
    interval = Interval(4.4, 6.4)
    return interval.onlyget(3.14)
print(test3())

@numba.extending.overload_method(IntervalType, "alsoset")
def interval_alsoset(interval, arg):
    if isinstance(arg, numba.types.Float):
        def alsoset_impl(interval, arg):
            interval.hi = interval.hi + arg
            return interval.hi
        return alsoset_impl

@numba.njit
def test4():
    interval = Interval(4.4, 6.4)
    return interval.alsoset(3.14)
print(test4())

"test1" should pass because it simply gets the value of "hi". "test2" fails with

LoweringError: No definition for lowering Interval.hi = float64
File "test2.py", line 79
[1] During: lowering "(interval).hi = $const0.5" at test2.py (79)

Failed at nopython (nopython mode backend)
No definition for lowering Interval.hi = float64
File "test2.py", line 79
[1] During: lowering "(interval).hi = $const0.5" at test2.py (79)

(full stack trace to follow) because numba.extending.make_attribute_wrapper only defines getters for the struct attributes, not setters.

"test3" is also okay; the method "onlyget" does not attempt to modify "hi," but "test4" fails with

LoweringError: No definition for lowering Interval.hi = float64
File "test2.py", line 100
[1] During: lowering "(interval).hi = $0.4" at test2.py (100)

Failed at nopython (nopython mode backend)
No definition for lowering Interval.hi = float64
File "test2.py", line 100
[1] During: lowering "(interval).hi = $0.4" at test2.py (100)
[2] During: resolving callee type: BoundFunction((<class '__main__.IntervalType'>, 'alsoset') for Interval)
[3] During: typing of call at test2.py (107)

These aren't bugs: I simply haven't opened up any methods that can modify the struct attributes; my question is how to do that.

I know that I can replace

numba.extending.make_attribute_wrapper(IntervalType, "hi", "hi")

with

@numba.extending.infer_getattr
class StructAttribute(numba.typing.templates.AttributeTemplate):
    key = IntervalType
    def generic_resolve(self, typ, attr):
        if attr == "hi":
            return numba.types.float64

@numba.extending.lower_getattr(IntervalType, "hi")
def struct_getattr_impl(context, builder, typ, val):
    val = numba.cgutils.create_struct_proxy(typ)(context, builder, value=val)
    return numba.targets.imputils.impl_ret_borrowed(context, builder, numba.types.float64, val.hi)

and everything works as it did before— I just looked up the definition of "make_attribute_wrapper" and expanded it by hand.

By analogy, the setter would be something like

@numba.extending.lower_setattr(IntervalType, "hi")
def struct_setattr_impl(context, builder, sig, args):
    assert isinstance(sig.args[0], IntervalType)
    assert isinstance(sig.args[1], numba.types.Float)
    interval, newvalue = args
    val = numba.cgutils.create_struct_proxy(sig.args[0])(context, builder, value=interval)
    ptr = val._get_ptr_by_name("hi")
    return builder.store(newvalue, ptr)

Adding this setter, "test2" no longer raises an exception, though it returns the wrong result: 6.4 (the initial value of "hi") instead of 99.999 (the assigned value). Numba is using my setter definition, executing the type-check and LLVM IR construction I've defined, and then dropping it on the floor. With or without the assignment line,

@numba.njit
def test2():
    interval = Interval(4.4, 6.4)
    interval.hi = 99.999   # this line!
    return interval.hi

"test2" gets expanded to

define i32 @"__main__.test2$2."(double* noalias nocapture %retptr, { i8*, i32 }** noalias nocapture readnone %excinfo, i8* noalias nocapture readnone %env) #0 {
entry:
  store double 6.400000e+00, double* %retptr, align 8
  ret i32 0
}

The existence of the struct gets optimized away, leaving this as a method that just returns 6.4. The LLVM IR I generated to store 99.999 in "interval.hi" never makes it to the output function.

So I just need to know how to get this generated IR into the output function.

Thanks,

-- Jim

Full stack trace for "test2" and "test4" before the lowering errors described in the text above:

Traceback (most recent call last):
  File "test2.py", line 81, in <module>
    print(test2())
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/
dispatcher.py", line 286, in _compile_for_args
    return self.compile(tuple(argtypes))
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/
dispatcher.py", line 532, in compile
    cres = self._compiler.compile(args, return_type)
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/
dispatcher.py", line 81, in compile
    flags=flags, locals=self.locals)
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/
compiler.py", line 693, in compile_extra
    return pipeline.compile_extra(func)
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/
compiler.py", line 350, in compile_extra
    return self._compile_bytecode()
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/
compiler.py", line 658, in _compile_bytecode
    return self._compile_core()
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/
compiler.py", line 645, in _compile_core
    res = pm.run(self.status)
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/
compiler.py", line 236, in run
    raise patched_exception
numba.errors.LoweringError: Caused By:
Traceback (most recent call last):
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/compiler.py", line 228, in run
    stage()
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/compiler.py", line 583, in stage_nopython_backend
    self._backend(lowerfn, objectmode=False)
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/compiler.py", line 538, in _backend
    lowered = lowerfn()
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/compiler.py", line 525, in backend_nopython_mode
    self.flags)
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/compiler.py", line 811, in native_lowering_stage
    lower.lower()
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/lowering.py", line 122, in lower
    self.lower_normal_function(self.fndesc)
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/lowering.py", line 157, in lower_normal_function
    entry_block_tail = self.lower_function_body()
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/lowering.py", line 182, in lower_function_body
    self.lower_block(block)
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/lowering.py", line 197, in lower_block
    self.lower_inst(inst)
  File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/errors.py", line 249, in new_error_context
    six.reraise(type(newerr), newerr, sys.exc_info()[2])
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/errors.py", line 243, in new_error_context
    yield
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/lowering.py", line 197, in lower_block
    self.lower_inst(inst)
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/lowering.py", line 327, in lower_inst
    impl = self.context.get_setattr(inst.attr, signature)
  File "/home/pivarski/.local/lib/python2.7/site-packages/numba/targets/base.py", line 576, in get_setattr
    % (typ, attr, valty))
LoweringError: No definition for lowering Interval.hi = float64

[Jim Pivarski]

It looks like Antoine Pitrou drafted Numba Enhancement Proposal #2 (NBEP2), which is what I'm trying to use. Should I try to contact him? My use-case could end up improving examples; I'll share my solution when it works.