Best practice for pseudo random number generation in cython

2,675 views
Skip to first unread message

Olivier Grisel

unread,
Jul 15, 2013, 7:41:59 AM7/15/13
to cython-users
Hi all,

I would like to know what people recommend to generated uniform random
numbers (e.g a double in range [0, 1)) that is:

- cross platform (windows, mac, linux, bsd* shall ideally return the
same sample sequence for a fixed given seed state)
- thread safe (explicit passing of the mutated state of the prng or
using a thread-local state internally)
- without locking the GIL
- easily wrappable in cython

numpy has a cython wrapper for randomkit:

https://github.com/numpy/numpy/tree/master/numpy/random/mtrand

but this is not exposed for other cython code to build against: no
mdrand.pxd to cimport.

The libc.rand_r is apparently deprecated and not supported under all
versions of the Microsoft compiler. libc.random is not thread safe as
far as I can tell.

Any suggestion?

--
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel

Oscar Benjamin

unread,
Jul 15, 2013, 7:48:31 AM7/15/13
to cython...@googlegroups.com
On 15 July 2013 12:41, Olivier Grisel <olivier...@ensta.org> wrote:
> Hi all,
>
> I would like to know what people recommend to generated uniform random
> numbers (e.g a double in range [0, 1)) that is:
>
> - cross platform (windows, mac, linux, bsd* shall ideally return the
> same sample sequence for a fixed given seed state)
> - thread safe (explicit passing of the mutated state of the prng or
> using a thread-local state internally)
> - without locking the GIL
> - easily wrappable in cython
>
> numpy has a cython wrapper for randomkit:
>
> https://github.com/numpy/numpy/tree/master/numpy/random/mtrand
>
> but this is not exposed for other cython code to build against: no
> mdrand.pxd to cimport.

Have you looked at simplerandom:

https://pypi.python.org/pypi/simplerandom/0.8.0


Oscar

Lars Buitinck

unread,
Jul 15, 2013, 8:18:34 AM7/15/13
to cython...@googlegroups.com
2013/7/15 Olivier Grisel <olivier...@ensta.org>:
> I would like to know what people recommend to generated uniform random
> numbers (e.g a double in range [0, 1)) that is:
>
> - cross platform (windows, mac, linux, bsd* shall ideally return the
> same sample sequence for a fixed given seed state)
> - thread safe (explicit passing of the mutated state of the prng or
> using a thread-local state internally)
> - without locking the GIL
> - easily wrappable in cython
>
> numpy has a cython wrapper for randomkit:
>
> https://github.com/numpy/numpy/tree/master/numpy/random/mtrand
>
> but this is not exposed for other cython code to build against: no
> mdrand.pxd to cimport.
>
> The libc.rand_r is apparently deprecated and not supported under all
> versions of the Microsoft compiler. libc.random is not thread safe as
> far as I can tell.

random is POSIX, and POSIX doesn't specify the RNG used, so the
results are still not the same across platforms. (Maybe MSVC has it,
I'm not sure.)

> Any suggestion?

/* Copyright 1990 The Regents of the University of California */
#define RAND_R_MAX 0x7FFFFFFF

int my_rand_r(unsigned *seed)
{
*seed = *seed * 1103515245 + 12345;
return (*seed % ((unsigned)RAND_R_MAX + 1));
}

This is the rand_r from 4.4BSD, as it currently lives in OpenBSD's
libc [1], with RAND_MAX changed to a larger constant to prevent crappy
old MSVC from throwing away 16 random bits for nothing. Needless to
say, this is not a crypto-strong RNG.

(I just sent this as a pull request to Gilles as well.)

[1] http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libc/stdlib/rand.c?rev=1.9;content-type=text%2Fplain

--
Lars Buitinck
Scientific programmer, ILPS
University of Amsterdam

Olivier Grisel

unread,
Jul 15, 2013, 8:50:26 AM7/15/13
to cython-users
2013/7/15 Lars Buitinck <L.J.Bu...@uva.nl>:
Interesting, have you tested if it is uniform enough (e.g. just by
plotting an histogram using matplotlib or better tools such as
dieharder)?

Lars Buitinck

unread,
Jul 15, 2013, 9:03:30 AM7/15/13
to cython...@googlegroups.com
2013/7/15 Olivier Grisel <olivier...@ensta.org>:
> Interesting, have you tested if it is uniform enough (e.g. just by
> plotting an histogram using matplotlib or better tools such as
> dieharder)?

A million runs of this modulo 1024 produces a histogram that is almost
perfectly square:

>>> def rand_r(seed):
... seed = (seed * 1103515245) % 2**32 + 12345
... return seed, seed % (RAND_MAX + 1)
...
>>> def test(seed, n):
... r = []
... for _ in xrange(n):
... seed, x = rand_r(seed)
... r.append(x % 1024)
... plt.hist(r, bins=1024)
...
>>> def test(seed, n, mod):
... r = []
... for _ in xrange(n):
... seed, x = rand(seed)
... r.append(x % mod)
... plt.hist(r, bins=mod)
...
>>> test(27717389, 1000000, 1024)


(Larger moduli cause Matplotlib to thrash on my machine.)

Lars Buitinck

unread,
Jul 15, 2013, 9:04:34 AM7/15/13
to cython...@googlegroups.com
2013/7/15 Lars Buitinck <L.J.Bu...@uva.nl>:
>>>> def test(seed, n, mod):
> ... r = []
> ... for _ in xrange(n):
> ... seed, x = rand_r(seed)
> ... r.append(x % mod)
> ... plt.hist(r, bins=mod)
> ...

I meant to post this definition of test, sorry for the noise.

Oscar Benjamin

unread,
Jul 15, 2013, 9:07:20 AM7/15/13
to cython...@googlegroups.com
On 15 July 2013 13:50, Olivier Grisel <olivier...@ensta.org> wrote:
>>
>> int my_rand_r(unsigned *seed)
>> {
>> *seed = *seed * 1103515245 + 12345;
>> return (*seed % ((unsigned)RAND_R_MAX + 1));
>> }
>>
[snip]
>
> Interesting, have you tested if it is uniform enough (e.g. just by
> plotting an histogram using matplotlib or better tools such as
> dieharder)?

Ther algorithm above is a linear congruential generator:
http://en.wikipedia.org/wiki/Linear_congruential_generator#Advantages_and_disadvantages_of_LCGs
'''
LCGs should not be used for applications where high-quality randomness
is critical. For example, it is not suitable for a Monte Carlo
simulation because of the serial correlation (among other things).
They should also not be used for cryptographic applications
'''

The simplerandom module that I linked to contains lots of better PRNGs
than this. You can see the C source here:

https://bitbucket.org/cmcqueen1975/simplerandom/src/b27ff8bd8d11bf1daa208656444f3cbde516dbf5/c/marsaglia/marsaglia-rng.c?at=default

Those were all written by George Marsaglia who posted them here:
https://groups.google.com/forum/#!msg/sci.stat.math/5yb0jwf1stw/ApaXM3IRy-0J


Oscar

Nathaniel Smith

unread,
Jul 15, 2013, 10:23:31 AM7/15/13
to cython...@googlegroups.com
On Mon, Jul 15, 2013 at 12:41 PM, Olivier Grisel
<olivier...@ensta.org> wrote:
> Hi all,
>
> I would like to know what people recommend to generated uniform random
> numbers (e.g a double in range [0, 1)) that is:
>
> - cross platform (windows, mac, linux, bsd* shall ideally return the
> same sample sequence for a fixed given seed state)
> - thread safe (explicit passing of the mutated state of the prng or
> using a thread-local state internally)
> - without locking the GIL
> - easily wrappable in cython

I'd probably try the ones in GSL first:
https://github.com/twiecki/CythonGSL

-n

Mark Miller

unread,
Jul 19, 2013, 2:36:27 PM7/19/13
to cython...@googlegroups.com

The topic of random number generation seems to come up pretty frequently. The usual responses mainly seem to suggest using GSL or Numpy's Cython code.

But theses aren't necessary easy solutions for a lot of us (OK...me). I have searched, but I have not found any tutorials that really help me to get these things up and running.  So in case I have missed it, is there a good resource that can help give me some step-by-step instructions for bringing a high-quality random number generator into my Cython code?

Thanks,

-Mark

 
--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.



Sturla Molden

unread,
Jul 19, 2013, 4:06:18 PM7/19/13
to cython...@googlegroups.com


The topic of random number generation seems to come up pretty frequently. The usual responses mainly seem to suggest using GSL or Numpy's Cython code.

But theses aren't necessary easy solutions for a lot of us (OK...me). I have searched, but I have not found any tutorials that really help me to get these things up and running.  So in case I have missed it, is there a good resource that can help give me some step-by-step instructions for bringing a high-quality random number generator into my Cython code?

Using randomkit from Cython is stright forward. It is just one .c file to compile and one .h file to include. These are the files you need:


Then you e.g. have something like this in Cython (not debugged):

cdef extern from "randomkit.h":
    ctypedef struct rk_state: pass
    unsigned long rk_random(rk_state * state)
    void rk_seed(unsigned long seed, rk_state * state)

from libc cimport stdlib

import os

cdef class RandomKit:

    """ Simple randomkit wrapper """

    cdef rk_state *state

    cdef __cinit__(RandomKit self):
        self.state = <rk_state*> stdlib.malloc(sizeof(rk_state))
        if (self.state == NULL): raise MemoryError
        
    cdef __init__(RandomKit self):
        cdef unsigned long *seedptr
        cdef object seed = os.urandom(sizeof(unsigned long))
        seedptr = <unsigned long*>(<void*>(<char*> seed))
        rk_seed(seedptr[0], self.state)

    def __dealloc__(RandomKit self):
        if self.state:
            stdlib.free(self.state)

    cdef unsigned long random(RandomKit self):
        return rk_random(self.state)

Or just use rk_state locally in a function:

def foobar():
    cdef rk_state state
    cdef unsigned long *seedptr, random_value
    cdef object seed = os.urandom(sizeof(unsigned long))
    seedptr = <unsigned long*>(<void*>(<char*> seed))
    rk_seed(seedptr[0], &state)
    random_value = rk_random(&state)

(Look in randomkit.h if you need other distributions.)


Same thing when calling MKL or ACML random number generators from Cython. If you know how to call C from Cython, you just use them as you would in C. Just read the docs.

Note that randomkit, MKL and ACML uses the same numerical algorithm (Mersenne twister MT19937). Though the one in MKL and ACML might be more optimized (I haven't tested). This algorithm is very good for Monte Carlo simulations, but very bad for cryptography.



Sturla


Thomas Wiecki

unread,
Jul 19, 2013, 4:12:08 PM7/19/13
to cython...@googlegroups.com
Alternatively you can use the random number generation code of GSL, e.g. via CythonGSL:

Thomas


Sturla Molden

unread,
Jul 19, 2013, 4:36:53 PM7/19/13
to cython...@googlegroups.com

Den 15. juli 2013 kl. 13:41 skrev Olivier Grisel <olivier...@ensta.org>:

> Hi all,
>
> I would like to know what people recommend to generated uniform random
> numbers (e.g a double in range [0, 1)) that is:
>
> - cross platform (windows, mac, linux, bsd* shall ideally return the
> same sample sequence for a fixed given seed state)
> - thread safe (explicit passing of the mutated state of the prng or
> using a thread-local state internally)
> - without locking the GIL
> - easily wrappable in cython
>

I use a setup like this for parallel Monte Carlo simulations. The main point is that all Monte Carlo threads use the same random number generator for numerical correctness, which makes it quite complex to ensure deterministic behavior. If I have used multiple PRNGs (one per Monte Carlo thread), the streams of random bits might noe be independent (though the immense period of the MT19937 makes it highly unlikely that they are not).

The principle is that one master thread runs the MT19937 and generates random bits. These are written in chunks to fixed-size buffers. Then there are consumer-threads that use code from randomkit to transform the random bits from these buffers into variables with distributions I want. There are two queues per consumer thread. The MT19937 thread iterates over all the consumer threads; for each consumer thread it receives a buffer from one queue, fills it with bits, and writes it to another queue. It is very important that each consumer thread have their own two queues and don't use the same two queues; otherwise the OS' scheduler would prevent deterministic behavior with multiple runs from the same seed. I could post the code, but it is quite complex and mostly interesting for those with very special interest :-)

Sturla





Sturla Molden

unread,
Jul 19, 2013, 7:02:55 PM7/19/13
to cython...@googlegroups.com

Josh Ayers

unread,
Sep 20, 2013, 12:04:52 AM9/20/13
to cython...@googlegroups.com


On Monday, July 15, 2013 4:41:59 AM UTC-7, ogrisel wrote:
Hi all,

I would like to know what people recommend to generated uniform random
numbers (e.g a double in range [0, 1)) that is:

- cross platform (windows, mac, linux, bsd* shall ideally return the
same sample sequence for a fixed given seed state)
- thread safe (explicit passing of the mutated state of the prng or
using a thread-local state internally)
- without locking the GIL
- easily wrappable in cython


I reimplemented Python's Mersenne Twister algorithm in Cython recently, and I think it meets all of your requirements.  The generator's state is maintained in a Cython class, the important functions can all be called without the GIL, and it produces identical output to the functions in Python's random module (assuming the same initial seed).

The code is here: https://bitbucket.org/joshayers/cythonlib/src

You'll want cythonlib/cyrandom.pyx and src/_random.c.  There are also unit tests in cythonlib/tests/cyrandom_tests.py.

Hope that helps,
Josh

Mark Miller

unread,
Sep 24, 2013, 4:36:40 PM9/24/13
to cython...@googlegroups.com

On Thu, Sep 19, 2013 at 9:04 PM, Josh Ayers <josh....@gmail.com>


I reimplemented Python's Mersenne Twister algorithm in Cython recently, and I think it meets all of your requirements.  The generator's state is maintained in a Cython class, the important functions can all be called without the GIL, and it produces identical output to the functions in Python's random module (assuming the same initial seed).

The code is here: https://bitbucket.org/joshayers/cythonlib/src

You'll want cythonlib/cyrandom.pyx and src/_random.c.  There are also unit tests in cythonlib/tests/cyrandom_tests.py.

Hope that helps,
Josh


Thanks for posting this, Josh. I appreciate the input that others have provided on this topic, but your approach is the first one that I've been able to correctly interpret (and more importantly, implement).

It would be wonderful to have something like this build right into Cython.

I'm on a Windows machine, so I ran into some problems with the implementation's reliance on your cytime code. For some reason, the line

"cdef extern from "sys/time.h" nogil:"

in _time.pxd was causing problems. If I'm interpreting things correctly, cytime.time() was just used to set the random number generator seed when one was not explicitly provided. I got around the issue in my code by just defaulting to a hard-coded seed when none was provided. I was also able to get things to work by using Python's time.time() function to create a seed.

And just to check, I wrote a quick implementation to benchmark the performance of your code relative to using numpy's random number generator (see file cyrandom_benchmark.pyx). In the case of the latter, I applied the recommended (I think) approach of generating all of the random numbers needed at once as an array, and then indexing into the numpy array as a memoryview to get individual numbers when needed.

Your code appears to be faster, especially under the realistic case where large numbers of random numbers will be generated. When smaller numbers of random numbers are needed, numpy + memoryviews seemed to do slightly better. But I suspect that this had more to do with the overhead associated with creating the instance of the generator and seeding it more than anything else.

# from file 'test_script.py'
import timeit
import numpy
numbers = [1000000, 100000, 10000, 1000, 100]

for number in numbers:
    print "cython: %s numbers" % str(number)
    c = timeit.Timer('benchmark.cython_rnd(%s)' % str(number),
                            'import cyrandom_benchmark as benchmark')
    print numpy.mean(c.repeat(10, 100))
    print "numpy: %s numbers" % str(number)
    n = timeit.Timer('benchmark.numpy_rnd(%s)' % str(number),
                            'import cyrandom_benchmark as benchmark')
    print numpy.mean(n.repeat(10, 100))
    print ''

cython: 1000000 numbers
0.994764532009
numpy: 1000000 numbers
1.14871078844

cython: 100000 numbers
0.100348045003
numpy: 100000 numbers
0.101275137725

cython: 10000 numbers
0.0108441837721
numpy: 10000 numbers
0.0108986984952

cython: 1000 numbers
0.00176594144735
numpy: 1000 numbers
0.00175197033742

cython: 100 numbers
0.000871037067272
numpy: 100 numbers
0.000778992107692

So again, thank you Josh for posting this really informative code.

-Mark
cyrandom_benchmark.pyx
test_script.py

Josh Ayers

unread,
Sep 26, 2013, 12:05:50 AM9/26/13
to cython...@googlegroups.com


Thanks for posting this, Josh. I appreciate the input that others have provided on this topic, but your approach is the first one that I've been able to correctly interpret (and more importantly, implement).

Great, glad it's useful for you.
 
I'm on a Windows machine, so I ran into some problems with the implementation's reliance on your cytime code. For some reason, the line

"cdef extern from "sys/time.h" nogil:"

in _time.pxd was causing problems. If I'm interpreting things correctly, cytime.time() was just used to set the random number generator seed when one was not explicitly provided. I got around the issue in my code by just defaulting to a hard-coded seed when none was provided. I was also able to get things to work by using Python's time.time() function to create a seed.

Yeah, I never actually ran the unit tests under Windows.  I just made an update that uses Python's time.time() function as an initial seed on Windows, and it seems to have fixed the problem.  Let me know if it still doesn't work for you.

Thanks,
Josh
 

Mark Miller

unread,
Sep 26, 2013, 11:53:42 AM9/26/13
to cython...@googlegroups.com

Hi Josh:  I just pulled your updates and checked them. I'm getting an error from the setup.py file associated with the line

"    with open(path, 'wt', encoding='utf-8') as fobj:" in the 'write_availability_pxi()' function.

TypeError: 'encoding' is an invalid keyword argument for this function

I'm using Python 2.7.5, and am guessing that it is the problem?

-Mark
 
Thanks,
Josh
 

Mark Miller

unread,
Sep 26, 2013, 12:10:48 PM9/26/13
to cython...@googlegroups.com
Hi Josh:

I took out the encoding parameter of that function call and got things to compile without apparent issues. But I am seeing a number of errors when testing the code. If you are actively working on this, please let me know if you'd like any additional testing or input.

-Mark

EE......EE.FFE......FFEE..E.E...
======================================================================
ERROR: test_three_threads (__main__.MultipleThreadsCompareTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\mpmiller\Marks\Python_code\cythonlib\cyrandom_tests.py", line 278, in test_three_threads
    self.run_n_threads(num_threads, num_iter)
  File "C:\Users\mpmiller\Marks\Python_code\cythonlib\cyrandom_tests.py", line 262, in run_n_threads
    threads.append(ThreadTarget(rand, num_iter))
  File "C:\Users\mpmiller\Marks\Python_code\cythonlib\cyrandom_tests.py", line 242, in __init__
    super().__init__()
TypeError: super() takes at least 1 argument (0 given)

======================================================================
ERROR: test_two_threads (__main__.MultipleThreadsCompareTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\mpmiller\Marks\Python_code\cythonlib\cyrandom_tests.py", line 273, in test_two_threads
    self.run_n_threads(num_threads, num_iter)
  File "C:\Users\mpmiller\Marks\Python_code\cythonlib\cyrandom_tests.py", line 262, in run_n_threads
    threads.append(ThreadTarget(rand, num_iter))
  File "C:\Users\mpmiller\Marks\Python_code\cythonlib\cyrandom_tests.py", line 242, in __init__
    super().__init__()
TypeError: super() takes at least 1 argument (0 given)

======================================================================
ERROR: test_getrandbits_negative_bits (__main__.RandomCompareTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\mpmiller\Marks\Python_code\cythonlib\cyrandom_tests.py", line 163, in test_getrandbits_negative_bits
    self.assertRaisesRegex(ValueError, r'number of bits less than zero',
AttributeError: 'RandomCompareTests' object has no attribute 'assertRaisesRegex'

======================================================================
ERROR: test_getrandbits_ouput_too_small (__main__.RandomCompareTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\mpmiller\Marks\Python_code\cythonlib\cyrandom_tests.py", line 168, in test_getrandbits_ouput_too_small
    self.assertRaisesRegex(ValueError, r'output.*too small',
AttributeError: 'RandomCompareTests' object has no attribute 'assertRaisesRegex'

======================================================================
ERROR: test_randint_empty_range (__main__.RandomCompareTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\mpmiller\Marks\Python_code\cythonlib\cyrandom_tests.py", line 222, in test_randint_empty_range
    self.assertRaisesRegex(ValueError, r'empty range',
AttributeError: 'RandomCompareTests' object has no attribute 'assertRaisesRegex'

======================================================================
ERROR: test_randrange_empty_range (__main__.RandomCompareTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\mpmiller\Marks\Python_code\cythonlib\cyrandom_tests.py", line 198, in test_randrange_empty_range
    self.assertRaisesRegex(ValueError, r'empty range',
AttributeError: 'RandomCompareTests' object has no attribute 'assertRaisesRegex'

======================================================================
ERROR: test_randrange_zero_step (__main__.RandomCompareTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\mpmiller\Marks\Python_code\cythonlib\cyrandom_tests.py", line 194, in test_randrange_zero_step
    self.assertRaisesRegex(ValueError, r'zero step size',
AttributeError: 'RandomCompareTests' object has no attribute 'assertRaisesRegex'

======================================================================
ERROR: test_invalid_state_1 (__main__.RandomCreationTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\mpmiller\Marks\Python_code\cythonlib\cyrandom_tests.py", line 40, in test_invalid_state_1
    self.assertRaisesRegex(ValueError, r'only Python 3',
AttributeError: 'RandomCreationTests' object has no attribute 'assertRaisesRegex'

======================================================================
ERROR: test_invalid_state_3 (__main__.RandomCreationTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\mpmiller\Marks\Python_code\cythonlib\cyrandom_tests.py", line 53, in test_invalid_state_3
    self.assertRaisesRegex(ValueError, r'state tuple.*length',
AttributeError: 'RandomCreationTests' object has no attribute 'assertRaisesRegex'

======================================================================
FAIL: test_randint_compare_to_python_negative (__main__.RandomCompareTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\mpmiller\Marks\Python_code\cythonlib\cyrandom_tests.py", line 219, in test_randint_compare_to_python_negative
    self.assertEqual(cy_val, py_val)
AssertionError: -704L != -751

======================================================================
FAIL: test_randint_compare_to_python_positive (__main__.RandomCompareTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\mpmiller\Marks\Python_code\cythonlib\cyrandom_tests.py", line 209, in test_randint_compare_to_python_positive
    self.assertEqual(cy_val, py_val)
AssertionError: 1360L != 1125

======================================================================
FAIL: test_randrange_compare_to_python_negative_step (__main__.RandomCompareTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\mpmiller\Marks\Python_code\cythonlib\cyrandom_tests.py", line 191, in test_randrange_compare_to_python_negative_step
    self.assertEqual(cy_val, py_val)
AssertionError: -464L != -419

======================================================================
FAIL: test_randrange_compare_to_python_positive_step (__main__.RandomCompareTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\mpmiller\Marks\Python_code\cythonlib\cyrandom_tests.py", line 180, in test_randrange_compare_to_python_positive_step
    self.assertEqual(cy_val, py_val)
AssertionError: 962L != 1010

----------------------------------------------------------------------
Ran 32 tests in 0.051s

FAILED (failures=4, errors=9)

Chris Barker

unread,
Sep 26, 2013, 3:29:04 PM9/26/13
to cython-users
I don't use py3, but this looks like it may be trying to run py3 code with py2...

-Chris

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris....@noaa.gov

Josh Ayers

unread,
Sep 26, 2013, 11:33:12 PM9/26/13
to cython...@googlegroups.com


On Thursday, September 26, 2013 9:10:48 AM UTC-7, Mark wrote:
Hi Josh:

I took out the encoding parameter of that function call and got things to compile without apparent issues. But I am seeing a number of errors when testing the code. If you are actively working on this, please let me know if you'd like any additional testing or input.
-Mark


This is a Py2/Py3 issue.  Until now, I've only used this code with Python 3. 

I updated the setup.py file, so the compilation problem should be fixed.  The test method assertRaisesRegex is named assertRaisesRegexp in Python 2, so that explains those errors.

As far as I can tell, the test failures in randint and randrange are due to a change in the algorithm for those two methods between Python 2 and 3.  I get different output from the builtin random module between the two versions.  My code matches Python 3's output.  Unless you have a need for random numbers that are identical to what's produced by Python 2's RNG, it should be fine to use.

Hope that helps,

Josh

Mark Miller

unread,
Sep 27, 2013, 11:58:26 AM9/27/13
to cython...@googlegroups.com

Thanks Josh. This just works for me, and I've found it to be really helpful. I am not concerned about generating identical numbers between Cython and Python. I'm just happy to be able to do this.

I run Monte-Carlo simulations where I never really know exactly how many random numbers will be needed. I have relied on numpy, but now I'll be switching over to this approach so that can just pull the numbers one at a time as needed.

Again, thanks for the original post to chime in on this topic. I bet I'm not the only person who will find this useful.

-Mark
Reply all
Reply to author
Forward
Message has been deleted
0 new messages