Maintaining type via JSON

[Anthony Lukach]

Hi, I'm curious about django-hstore's design choice of casting many data types to string.  What is the purpose of this?  I understand that Hstore can only hold strings but would it not be simpler to cast all data as JSON string upon entry into the DB and loaded from JSON upon retrieval, transparently preserving data common types?  For the sake of backwards-compatibility, this could be a feature turned on via a field-parameter.

I did this (admittedly not so eloquently) on an fork of the old django-hstore module (https://github.com/alukach/django-hstore), which is now quite out of date with the djangonaut's version.  I just upgraded a project to use the official djangonaut version of the package and, to maintain this feature, created a bit of a shim using a mixin and some odd-usage of the @property decorator (https://gist.github.com/alukach/b6d0c52868cf0caba36d).

Is there any reason why the library is not written to support this out-of-the-box?

Thanks.

p.s. This is my first post to the group so apologies if this is a well-discussed topic or if I breached any etiquette. I was unable to find any information about it through the search.

Anthony

[Andrey Antukh]

Hi Anthony.

Maintain  type via json is a good idea but it has some drawbacks, it makes search slightly more complicated.

Your idea is very good but it can live out of django-hstore or as additional mixin, and anyone can choice the behavior that needs.

If you give me permissions, I'll try update docs with this tip referencing your gist code and add the proposed mixin in code making 

the proposed behavior available for all django-hstore users.

Federico, that do you think about that?

Thanks!

Andrey

--
You received this message because you are subscribed to the Google Groups "django-hstore" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-hstor...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Andrey Antukh - Андрей Антух - <andrei....@kaleidos.net> / <ni...@niwi.be>

http://www.niwi.be

https://github.com/niwibe

[Nemesis]

Same thought I had when I thought about storing the pickled values: I thought that I would store something in the database which is slightly altered which could introduce all sorts of problems. In case of json the alteration at least is low.

To me the best solution is to use the new schema mode:
https://github.com/djangonauts/django-hstore/pull/54

With this solution we can specify a schema and have real types by sitting on the shoulders of the standard django fields.
In the end all applications need some kind of internal schema.

I'm ok with adding it to the documentation but I would warn that the single page doc is growing a bit and for the future we should look into separating it in several pages to make it easier to digest.

[Anthony Lukach]

Hey, I totally dropped the ball on this discussion.

I still feel that storing data as JSON encoded to maintain type would be a major improvement to django-hstore.  The schema mode attempts to remedy the issue of typing but requires that one knows what data types will be stored in the model ahead of time, which isn't always accurate and seems to be a bit of a departure from schema-less storage.

@Andrey Feel free to add a mention in the Django Hstore documentation.  We've been working with a DjangoHstoreMixin at my work for the past few months, we've set the repo to public and the license to MIT.  If anyone wants to use it, have at it: https://bitbucket.org/ospreyinformatics/django-hstore-mixin/ .  Pull requests are very welcome!

Anthony

--
You received this message because you are subscribed to a topic in the Google Groups "django-hstore" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-hstore/AqG6seEEwx0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-hstor...@googlegroups.com.

[nemesis]

Hi Antony,

On Thursday, July 24, 2014 12:12:54 AM UTC+2, Anthony Lukach wrote:

Hi, I'm curious about django-hstore's design choice of casting many data types to string.  What is the purpose of this?  I understand that Hstore can only hold strings but would it not be simpler to cast all data as JSON string upon entry into the DB and loaded from JSON upon retrieval, transparently preserving data common types?  For the sake of backwards-compatibility, this could be a feature turned on via a field-parameter.

The idea is good, I think no one would object that it would be useful for many to be able to retain types without needing to define a schema in advance.

So, because the intention is clear, I think that discussing on a pull request would be much more interesting than writing words here.

Do you have time to work on a separate branch with a pull request open as we did for some other features? That way as you write code anybody can see it and send feedback.

I will be away for the Google Summer of Code Reunion until the end of october but I could find some spare time to contribute to the discussion.

Federico

[Anthony Lukach]

Alright, that sounds like a decent idea.  I'll try to find some time and put together a start of the code.  Right now I'm thinking that something like this would fit best in a separate model field type to best prevent conflicts with any existing Hstore fields in people's projects.

Anthony

--

[Anthony Lukach]

I put some work in over the past few evenings and took a stab at the implementation, take a look here: https://github.com/djangonauts/django-hstore/pull/80.  All further discussion should probably take place within the pull request.

To unsubscribe from this group and all its topics, send an email to django-hstore+unsubscribe@googlegroups.com.

[Anthony Lukach]

The Pull Request is now effectively 'feature complete' and passing all tests.  I feel that it's ready to be merged.  I'd appreciate some feedback on the branch in respect to both a) code/design style; and b) confirmation that it works on other people's machines.

https://github.com/djangonauts/django-hstore/pull/80

Anthony

[Nemesis]

Thanks Anthony,

I hope to be able to provide some feedback and review asap.

Federico

You received this message because you are subscribed to the Google Groups "django-hstore" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-hstor...@googlegroups.com.

[Anthony Lukach]

Apologies if this is uncouth, but bumping with a request for review.  Would love to get an idea of whether or not this will be merged, as I'm considering rolling it in to some projects of mine but am hesitant to build on anything that would require myself to maintain a fork of the module.

Anthony

To unsubscribe from this group and all its topics, send an email to django-hstor...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.