JsonField
Ric
a json data field, the code is simple as this, and it works with
lastest django release
from django.db import models
from django.core.serializers.json import simplejson, DjangoJSONEncoder
import StringIO
class JsonField(models.Field):
__metaclass__ = models.SubfieldBase
serialize_to_string = True
def get_internal_type(self):
return "TextField"
def value_to_string(self, obj):
return self.get_prep_value(self._get_val_from_obj(obj))
def get_prep_value(self, value):
if value:
stream = StringIO.StringIO()
simplejson.dump(value, stream, cls=DjangoJSONEncoder)
value = stream.getvalue()
stream.close()
return value
return None
def to_python(self, value):
if isinstance(value, (str, unicode)):
value = StringIO.StringIO(value)
return simplejson.load(value)
return value
you can set a standard json object as data for a field, and the field
encode this as a json object when you save the model
example
obj.shopping_list = {"fruit":["apple", "banana"]}
it is also possibile to pass an ugettext_lazy object inside the
encoder, if we put a method to encode/decode lazy objects inside
DjangoJSONEncoder
Russell Keith-Magee
> this is my proposition to have custom data inside a model field
>
> a json data field, the code is simple as this, and it works with
> lastest django release
See #12990 -- this is a ticket that has requested exactly this feature.
You'll also see that with one exception, the core team have all been
-1 or -0 on this idea.
Yours,
Russ Magee %-)
Ric
maybe there is no need to to create a class
JSONDateEncoder(json.JSONEncoder) because there is DjangoJSONEncoder
already.
then it would be great to implement a lazy translation object for
DjangoJSONEncoder to store ugettext_lazy objects.
for me there is no need to create set_%s_json and get_%s_json method
because the field should handle it directly, and we should use the
to_python method, like all other fields in django.
for example the datetime field directly push a datetime object to the
model (using to_python) and transform it to a string when we call save
method.
we should use the same logic here and encode a json object to string
only when we call the save method, no need for a get_%s_json in the
field api.
On 6 Nov, 01:36, Russell Keith-Magee <russ...@keith-magee.com> wrote:
Ric
schinckel
Ric
you wrote:
# TODO: Look for date/time/datetime objects within the structure?
and there is no need, because it's handled by djangojsonencoder
Tom Evans
> this is my proposition to have custom data inside a model field
>
> a json data field, the code is simple as this, and it works with
> lastest django release
>
The problem with something like this is that it is rarely a good idea
to stick opaque data into the database that cannot then be queried
upon. You may have a very good reason for doing this, but most of the
time that it happens it ends up being a bad design choice.
Adding this to core would suggest that bad design choices are a-ok in
Django, which is why I expect many people are -0/-1 on this. If I had
a vote, I'd be -1.
Cheers
Tom
Ric
when, like my app, you need to handle document, real documents (like
invoice, ddt, quotations) and other stuffs, it's a good practice to
keep all in one table, separating only the things that you need to
query, and leave a non relational field to handle evreything else.
read http://guide.couchdb.org/ there are a lot of ideas that are
simply brillant.
one of this is the validation of the non relational data, and django
has got a fantastic framework to validate data, so it would be grat to
implement a custom non relational field that can be validated, and
it's stored in one place, because for some applications the best
design practice is to put all the data in one place, because the date
is simply non relational.
django should be able to handle this kind of situation, and please
don't be arrogant.
you don't like non relational data, but non relational db are growing,
and maybe there is a reason.
the reason is that relational data not always is the best answer.
non relational data it's a tool, and in some situation it's the best
practice.
for me the first step it's to create a JsonField, than we can use
validators to let the developers validate non relational data.
On 7 Nov, 12:45, Tom Evans <tevans...@googlemail.com> wrote:
Russell Keith-Magee
> you say "is rarely a good idea", and that it is the only thing true.
>
> when, like my app, you need to handle document, real documents (like
> invoice, ddt, quotations) and other stuffs, it's a good practice to
> keep all in one table, separating only the things that you need to
> query, and leave a non relational field to handle evreything else.
>
> read http://guide.couchdb.org/ there are a lot of ideas that are
> simply brillant.
>
> one of this is the validation of the non relational data, and django
> has got a fantastic framework to validate data, so it would be grat to
> implement a custom non relational field that can be validated, and
> it's stored in one place, because for some applications the best
> design practice is to put all the data in one place, because the date
> is simply non relational.
>
> django should be able to handle this kind of situation, and please
> don't be arrogant.
>
> you don't like non relational data, but non relational db are growing,
> and maybe there is a reason.
>
> the reason is that relational data not always is the best answer.
This is not in dispute. What is in dispute is that when you have
non-relational data, you should be using a non-relational data store,
not trying to stuff non-relational data into your relational store.
Tom correctly summarized the argument against putting JSONField in
trunk -- it's the wrong tool for the job. If you have non-relational
data, you should be using a non-relational store. Alternatively, you
should be exploiting the non-relational data structures inside a
relational store (e.g., PostgreSQL's hstore).
At a project level, Django has a responsibility to advocate for best
practices. Using the wrong tool for the job isn't best practice. If
you really do need a JSONField, Django gives you the abilty to add one
yourself as an external dependency, and for my money, that's the right
place for JSONField to live.
Yours,
Russ Magee %-)
Torsten Bronger
Tom Evans writes:
> On Sat, Nov 5, 2011 at 11:48 AM, Ric wrote:
>
>> this is my proposition to have custom data inside a model field
>>
>> a json data field, the code is simple as this, and it works with
>> lastest django release
>
> The problem with something like this is that it is rarely a good
> idea to stick opaque data into the database that cannot then be
> queried upon. [...]
>
> Adding this to core would suggest that bad design choices are a-ok
> in Django, which is why I expect many people are -0/-1 on this.
While I appreciate that Django core is supposed to enforce good
design practices, one must be careful with that if some practice may
have valid use cases. You cannot prevent the user from shooting in
the foot anyway.
While we make extensive use of the relational model in our project,
I remembered to have a couple of JSON fields (though we don't have a
field class them them so far), so I skimmed through our code to see
why we did this.
In our code, a JSON field is an application of the KISS principle.
I see at least two valid use cases for such fields:
1. You say that you will never query for the content of a field.
For example, we store user preferences in a JSON field. I'll
never be interested in all users who set their page skin to
"classic". (And if I would, I could still do a "manual" query or
do the schema and data migration. This is okay as long as its
probability is sufficiently small.)
Moreover, in some cases, realising the JSON field explicitly in
the relational model would have led to many tiny models. This
may also be bad for performance, but it is certainly bad for
readability and maintenance.
2. Dynamic typing. In one case, we simply don't know a priori how
the data structure in the JSON field will look like. The Python
code interprets its content depending on another field value of
the model instance. The alternative would have been extensive
use of multi-table inheritance.
Tsch�,
Torsten.
--
Torsten Bronger Jabber ID: torsten...@jabber.rwth-aachen.de
or http://bronger-jmp.appspot.com
Tom Evans
> django should be able to handle this kind of situation, and please
> don't be arrogant.
Hi, I'm sorry you felt that I was being arrogant - I don't know the
situations of your project, it may well be exactly what is needed in
that situation.
I was only trying to point out why in general it is not a good idea to
do this, and why that would mean it shouldn't be in core.
Cheers
Tom
Tom Evans
<bro...@physik.rwth-aachen.de> wrote:
>
> While I appreciate that Django core is supposed to enforce good
> design practices, one must be careful with that if some practice may
> have valid use cases. You cannot prevent the user from shooting in
> the foot anyway.
>
> While we make extensive use of the relational model in our project,
> I remembered to have a couple of JSON fields (though we don't have a
> field class them them so far), so I skimmed through our code to see
> why we did this.
>
And so do I - the main point here is that we have both successfully
loaded the gun and aimed it at our foot without this field existing in
core. I think it's important to have that distinction between things
you can do and things you should do, in order to make you think
"should I really be doing this".
Cheers
Tom
diogobaeder
I agree that we should use the right tool for the job; But even a
relational database can be used for semi-structured data when it's not
going to be directly queried. Matter of fact, there's even a design
pattern recognized for that: http://martinfowler.com/eaaCatalog/serializedLOB.html
Example of use: think of a semi-structured data store (in a relational
database) that is going to be read through a search engine (think
about ElasticSearch of Solr, here).
But yes, if you're going to need to query for this data by parts of
the JSON field, then you're shooting your own foot. So, in the end, we
have to think a lot about what's the problem we're trying to solve -
it's not as simple as "don't use RDBMS for non-relational data" -,
IMHO.
Well, at least here's my 2 cents.
Cheers,
Diogo
On 8 nov, 08:34, Tom Evans <tevans...@googlemail.com> wrote:
> On Mon, Nov 7, 2011 at 11:48 PM, Torsten Bronger
>
Ric
the best thing to do is to have a non relational data store.
but you need a relational data store only if you need to query non
relational data, in the other cases you can use a relational db,
because relational db got foreign keys, a things that rocks!
i need to handle documents (real documents, paper), and what i have
done is to store cleaned_data inside a model and give to that a
number.
i need to do a lot of different documents (iso 9000 sheet, ddt,
invoices, quotations), and i don't need to query non relational data
that are inside (not all of them, of course).
what i need, instead, is to have some common field for all my document
(document date, document number, document prefix, document owner, and
maybe a content type field) to query and order them, and store them in
an unique db table.
that kind of schema is better than a non relational database, because
i can use foreign keys and index for fast query the things i need to
query, and create different kind of classes to handle different kind
of documents.
to archive that kind of job, a json field is great. and i dont't whink
i'm the only django user that needs stuff like that (business staff).
sometimes you might need to save raw html inside db, and sometimes you
may need to store raw json to db.
that's all.
Ric
i don't want to use a non relational db, because i need foreign key
and i need non relational data.
read the couchdb guide, i think that you can find inspiration to
create powerful things.
i think that some ability of non relational data store can be archived
on any datastore, with a framework.
validation of non relational data and creation of indexes
we can validate data on save, and we can create index of non
relational data on save.
that is just my opinion, but maybe i'm not a programmer, i'm just an
hacker!
Ric
right way.
i can use json field in my trunk and that's good for me.
i thought maybe, with validators, django could be be able to
accomplish some of the features of a non relational data store,
without messing up with low lowel database api, and use the powerful
model and queryset django api.
forget about it.
Ric
django use pickle to store non relational data with session...