Constructing Python tuple not allowed without gil

[Hao Wang]

Dear community:

I was trying to run the following grid search function using Cython :

def grid_search():
    for k in prange(7, 20, nogil=True):
        for z in range(4, 20, nogil=True):
            output_file = 'output_' + str(k) + '_' + str(z)
            compute_main(k, z, output_file)

And I got the following error when I run "python3 cc.py build_ext --inplace" :

churn_predict.pyx:643:24: Calling gil-requiring function not allowed without gil

Please let me know how I can fix the problem.

[da-woods]

1. str creates a Python string and so requires the GIL. As does the string concatenation.

2. The second "range" loop should not have a nogil=True argument (since that's only an argument for prange).

3. You may need to type "k" and "z" as C integer types (I'm not sure how good Cython's type inference is in these cases).

4. (I'm assuming that compute_main is a cdef function that doesn't require the GIL, although  you don't show it so it's hard to be sure). 

You either need to use a C or C++ string formatting operation instead of Python strings, or you need to accept that your operation requires the GIL and abandon prange.

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/0ba27e5f-92ad-4b50-8dbe-6da883971623n%40googlegroups.com.

[Stefan Behnel]

da-woods schrieb am 08.12.21 um 09:25:

> 1. str creates a Python string and so requires the GIL. As does the string
> concatenation.
> 2. The second "range" loop should not have a nogil=True argument (since
> that's only an argument for prange).

… and because once the GIL *is* released, there is no use in releasing it
again.

> 3. You may need to type "k" and "z" as C integer types (I'm not sure how
> good Cython's type inference is in these cases).
> 4. (I'm assuming that compute_main is a cdef function that doesn't require
> the GIL, although  you don't show it so it's hard to be sure).

I'll assume that the code below was just an example, but it's generally a
better approach to separate the computation from the output, run the
computation in parallel, and then do the file writing afterwards (also in
parallel or not).

> You either need to use a C or C++ string formatting operation instead of
> Python strings, or you need to accept that your operation requires the GIL
> and abandon prange.
>
> On 08/12/2021 03:47, Hao Wang wrote:
>> Dear community:
>>
>> I was trying to run the following grid search function using Cython :
>>
>> def grid_search():
>>     for k in prange(7, 20, nogil=True):
>>         for z in range(4, 20, nogil=True):
>>             output_file = 'output_' + str(k) + '_' + str(z)
>>             compute_main(k, z, output_file)
>>
>> And I got the following error when I run "python3 cc.py build_ext
>> --inplace" :
>>
>> churn_predict.pyx:643:24: Calling gil-requiring function not allowed
>> without gil
>>
>> Please let me know how I can fix the problem.

Hope that helps,
Stefan

[David Menéndez Hurtado]

On Wed, 8 Dec 2021, 9:27 am da-woods, <dw-...@d-woods.co.uk> wrote:

1. str creates a Python string and so requires the GIL. As does the string concatenation.

2. The second "range" loop should not have a nogil=True argument (since that's only an argument for prange).

3. You may need to type "k" and "z" as C integer types (I'm not sure how good Cython's type inference is in these cases).

4. (I'm assuming that compute_main is a cdef function that doesn't require the GIL, although  you don't show it so it's hard to be sure). 

You either need to use a C or C++ string formatting operation instead of Python strings, or you need to accept that your operation requires the GIL and abandon prange.

Can you not use with gil: to construct the string? That part will not be parallelised, but it should be fast enough.

On 08/12/2021 03:47, Hao Wang wrote:

Dear community:

I was trying to run the following grid search function using Cython :

def grid_search():
    for k in prange(7, 20, nogil=True):
        for z in range(4, 20, nogil=True):
            output_file = 'output_' + str(k) + '_' + str(z)
            compute_main(k, z, output_file)

And I got the following error when I run "python3 cc.py build_ext --inplace" :

churn_predict.pyx:643:24: Calling gil-requiring function not allowed without gil

Please let me know how I can fix the problem.

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/0ba27e5f-92ad-4b50-8dbe-6da883971623n%40googlegroups.com.

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cython-users/a0c4eb29-57c6-90b0-6afa-9de9a01be117%40d-woods.co.uk.

[da-woods]

>
> Can you not use with gil: to construct the string? That part will not
> be parallelised, but it should be fast enough.
>

Yes you could. You'd want to time it to check that the parallelization
remains worthwhile, but that's often a good approach for small bits of
work that need the GIL.