Custmo field numpyarray - why does this even work?

[Joakim Hove]

Hello;

I want to create a custom model field for storing a numpy array. It has currently been so simple that I fear I am overrlooking something and getting a very false positive?

<code>

class NumpyArrayField(BinaryField):

    dtype = numpy.float32

    @classmethod

    def load_numpy_array(cls , blob):

        return numpy.fromstring(blob , NumpyArrayField.dtype)

    def __init__(self , *args , **kwargs):

        kwargs['default'] = None

        super(NumpyArrayField , self).__init__(*args , **kwargs)

    def from_db_value(self, value, expression, connection, context):

        if value is None:

            return value

        return self.load_numpy_array( value )

    def to_python(self, value):

        if isinstance(value, numpy.ndarray):

            return value

        if value is None:

            return value

        return self.load_numpy_array( value )

</code>

As I see it this has (essenially two versions) of code for deserializing from a database value to a numpy array, but no method for the opposite operation. However it seemingly works?

I have created a small model with one of these fields:

<code>

class TimeSeries(Model):

    start = DateTimeField( )

    step = IntegerField( )

    data = NumpyArrayField( )

    @classmethod

    def createArray(cls , size = 0):

        return numpy.ndarray( shape = [size] , dtype = NumpyArrayField.dtype)

    def __getitem__(self , index):

        if self.data is None:

            raise IndexError

        else:

            return self.data[index]

    def __setitem__(self , index , value):

        if self.data is None:

            raise IndexError

        else:

            self.data[index] = value

    def __len__(self):

        if self.data is None:

            return 0

        else:

            shape = self.data.shape

            return shape[0]

</code>

And a test:

<code>

class TimeSeriesTest(TestCase):

    def test_create(self):

        ts = TimeSeries.objects.create( start = timezone.now(),

                                        step = 100 ,

                                        data = TimeSeries.createArray( ))

        self.assertEqual( len(ts) , 0 )

        with self.assertRaises(IndexError):

            ts[0]        

        ts.addValue( 1 )

        

        self.assertEqual( len(ts) , 1 )

        with self.assertRaises(IndexError):

            ts[1]

        self.assertEqual( ts[0] , 1 )

 

        # How on earth does the NumpyArrayField know how to save itself to the db?

        ts.save()

        ts2 = TimeSeries.objects.get( pk = 1 )

        self.assertEqual( ts2[0] , 1)

        self.assertEqual( len(ts2) , 1 )

</code>

I was expecting the test to fail spectacularly when calling ts.save() - however it seems to work, and I can even get the instance back from db - what gives?  I am using sqlite and have tried to force the db to create a file.

I was excpecting to implement the get_prep_db_value() method - but now when things seemingly work without it I get quite uncertain? Is there some magic trickery involved which says that this should indeed work - or am I seeing a completely false positive?

Joakim

[Joakim Hove]

Sorry;

how do you get the nice code formatting - I thought it was through the use of  <code> ... </code> blocks?