Treat ndarray from closure as part of the signature?

0 views
Skip to first unread message

Compl Yue

unread,
Apr 20, 2017, 8:23:53 AM4/20/17
to Numba Public Discussion - Public
Howdy,

Here we've been intensively using Numba for a while, it's really amazing and powerful.

Now I'm prototyping a framework to allow python scripting-alike programs to assembly data series and their computing dependencies altogether, then update all data with a single point trigger. 

Of course I want all the triggered computation to run within a single trip `nopython` call, but for the rest of those can't, I manage to call them in object mode:

       
if self._advancers is None:
           
# jit compile a fast (nopython mode) advancer function and a slow (object mode) function

            # numba is finding callees from outer scope, satisfy it by filling this function's scope
            locals().update({'f%d' % i: computer.stage_advanced for i, computer in enumerate(self.computers)})

           
# the fast and slow function code, use single space indent
            fast_func = '''
def fast_adv_stg(since_tp,before_tp):
 pass

'''
            slow_func = '''
def slow_adv_stg(since_tp,before_tp):
 pass

'''

            for i, computer in enumerate(self.computers):
                adv_func
= computer.stage_advanced
                is_fast
= False
                if hasattr(adv_func, 'nopython_signatures'):
                    jit_opt
= getattr(adv_func, 'targetoptions', {})
                    is_fast
= jit_opt.get('nopython', False)
               
if is_fast:
                    fast_func
+= ' f%d(since_tp,before_tp)\n' % (i,)
               
else:
                    slow_func
+= ' f%d(since_tp,before_tp)\n' % (i,)

           
# dynamically compile a function to call each computer function by literal function name
            fast_adv_stg = eval_in_context(fast_func, locals())
            fast_adv_stg
= nb.jit(nopython=True)(fast_adv_stg)
            slow_adv_stg
= eval_in_context(slow_func, locals())
            slow_adv_stg
= nb.jit(nopython=False)(slow_adv_stg)
           
self._advancers = fast_adv_stg, slow_adv_stg
       
else:
            fast_adv_stg
, slow_adv_stg = self._advancers

        fast_adv_stg
(since_tp, before_tp)
        slow_adv_stg
(since_tp, before_tp)


But many `computer.stage_advanced` functions need to update various different ndarrays, so they can NOT be in a single formed signature. 
And Numba currently treats global/nonlocal variable as compile constant, so no way to have them run in nopython mode while updating those ndarrays, that's a very pity.

I'd like to query you folks if ndarrays from closure shall be writable by nopython functions? 
I roughly think if the accessed ndarrays are treated as fixed part of the function's signature, the way is clear.

Thanks with best regards,
Compl

Reply all
Reply to author
Forward
0 new messages