Re: [cython-users] Working with uint64 fails with Exception OverflowError: 'long int too large to convert to int'

[Robert Bradshaw]

On Thu, Nov 8, 2012 at 3:17 PM, Volker Jaenisch
<volker....@inqbus.de> wrote:
> Hello Cython users!
>
> Maybe this problem is already solved, so please send me a pointer to the
> solution,and please forget that I asked.
>
> Have a small cython routine that mainly splits a comma seperated csv line
> and generates a 64 bit unsigned int returned as an
> unsigned long long "can_msg".
>
> cdef decode_can_id_and_msg(char * linestrip, char * can_id, unsigned long
> long * can_msg):
> cdef char* token
> cdef int shift = 0
>
> can_msg[0] = 0
> token = strtok(linestrip, ',')
> strcpy(can_id, token)
>
> token = strtok(NULL, ',')
> while (token != NULL) & (shift < 64) :
> can_msg[0] += atoi(token) << shift
> shift +=8
> token = strtok(NULL, ',')
>
> The cython routine itselfs works flawlessly and returns the correct uint64
> value.
>
> But in the main routine where I do stuff like this:
>
> cdef unsigned long long can_msg
> ...
> decode_can_id_and_msg(linestrip, can_id, &can_msg)
> print(can_msg)
> ...
> writer.append_data(table_name, (0, can_msg) )
> -> this is a write to HDF5
>
> I got the error:
> Exception OverflowError: 'long int too large to convert to int' in ..
>
> This is due to the magic conversion of cython that tries to convert the
> unsigned long long back to an int
> [http://docs.cython.org/src/reference/language_basics.html#parameters]
> and fails.
>
> How can I cast the unsigned long long to e.g. an numpy.int64 to work on with
> that variable?
> Is there an other way to work transparently with a uint64 type in
> cython/python?
>
> Any help welcome, thanks in advance

What it writer? What's the signature of append_data? It could be that
writer.append_data is throwing the exception, not your Cython code.
(Have you tried it from Python?) You should be able to work with
uint64 just fine by declaring them as "unsigned long long" just fine.

- Robert