[pyDAL] Next "big" new features

[Giovanni Barillari]

Hi all,

I'd like to make a list of "big" new features we want to add in pyDAL in the next versions.

For "big" new features I mean everything which is not bugfixing and is quite consistent (so it needs testing and experimenting).

The main idea behind this is to give a priority to new features and attach them to one or more developers and to specific milestones (obviously we can reschedule, but as I stated some weeks ago, we should start organizing ourselves a bit..).

I'm starting the list with:

- python 3 support

I think is time to start moving, also because we're officially dropping 2.5.x support. I've made a separate branch on pyDAL repo, on which I'm working on (slowly, I know that). The main idea is to have a version passing the tests with python3, even if it's not enough to say "we support python 3", in fact we should start a discussion at least about the strings and text stuffs due to different approach to bytes/unicode from py2 to py3.

Then I saw the 'iterselect' optimization by ilvalle, which I consider one "big" new feature combined with consequents optimizations of the "Row" object.

So, what new features you want to propose?

/Giovanni

[Paolo Valleri]

Hi, I think

- iterselect

- optimization of Row (as I mentioned in an other mail)

- mongo db code cleanup

- serializers

- split BaseAdapter  (~1700 loc!) into smaller classes ?

- extended .table for index etc ?

I have the first two on my todolist

 Paolo

--
-- mail from:GoogleGroups "web2py-developers" mailing list
make speech: web2py-d...@googlegroups.com
unsubscribe: web2py-develop...@googlegroups.com
details : http://groups.google.com/group/web2py-developers
the project: http://code.google.com/p/web2py/
official : http://www.web2py.com/
---
You received this message because you are subscribed to the Google Groups "web2py-developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py-develop...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Leonel Câmara]

I would love to have full text search support for the backends that support it (like postgresql).

[Michele Comitini]

A big ambitious change would be make migrations a full engine itself.
Some goals in chronological order could be:
- .table files with more attributes and hints (for indexes and other constraints).
- history
- versioning and conflict management (to help manage merging diverting history)

[Massimo DiPierro]

Yes to python 3 as top.

--

[Jose]

Hi,

This message already wrote before, but it seems that appeared.

I have an application that uses a SQLServer database, which is part of another system. The database is latin1

when I create the DAL object I pass the parameter db_codec = 'latin1'

This solves the conversion to utf8 when I read data, but when I write it does in utf8.

I solved it by doing the following: If I create a new record from a SQLFORM use "dbio = False", then analyze and convert the dictionary form to latin1:

for k in form.vars:
         if isinstance (form.vars [k], unicode) or isinstance (form.vars [k], str):
             form.vars [k] = form.vars [k] .decode ('utf8'). encode ('latin1')

db.mytable.insert (** dict (form.vars))

Then, for a very simple task you have to write a lot of code.

The proposal is to become the edges to encoding necessary:

Database <---enconding1 (default utf8)---> DAL (utf8) <---encoding2 (default utf8)---> Application

Jose

[Giovanni Barillari]

Jose, this sounds more like an issue. You should open an issue on github for this!

/Giovanni

--

-- mail from:GoogleGroups "web2py-developers" mailing list
make speech: web2py-d...@googlegroups.com
unsubscribe: web2py-develop...@googlegroups.com
details : http://groups.google.com/group/web2py-developers
the project: http://code.google.com/p/web2py/
official : http://www.web2py.com/
---

You received this message because you are subscribed to a topic in the Google Groups "web2py-developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/web2py-developers/j133Nbl_w84/unsubscribe.
To unsubscribe from this group and all its topics, send an email to web2py-develop...@googlegroups.com.

[pbreit]

+1 on Postgres full text search and Postgres Hstore.

[Niphlod]

-1 on hstore, if the same base implementation can't be done in other adapters...

[Giovanni Barillari]

I've just added some of the proposed points to the issues on github, with [new feature] and [proposal] tags.

Feel free to discuss them on github, and add any other point I've missed.

/Giovanni

[Richard Vézina]

+1 Hstore despite Niphold says no

:D

Richard

--

-- mail from:GoogleGroups "web2py-developers" mailing list
make speech: web2py-d...@googlegroups.com
unsubscribe: web2py-develop...@googlegroups.com
details : http://groups.google.com/group/web2py-developers
the project: http://code.google.com/p/web2py/
official : http://www.web2py.com/
---

You received this message because you are subscribed to the Google Groups "web2py-developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to web2py-develop...@googlegroups.com.

[Luca de Alfaro]

What I would really like is to be able to specify the type of the "id" field of a table, with int (or long) as default, but with str as possible value. 

This would enable me to use DAL to access e.g. the Google Datastore using string keys for rows, and I think the same benefit would be there for MongoDB. 

Using string IDs is helpful for a number of reasons:

• Often, I need to come up with a row id before the row is actually written (e.g. to send it to the page via ajax, whatever), and the simplest is to set the id to a long random string.
• Lookup by key in Datastore is significantly faster, and has better consistency properties than lookup via an index on another field, and having keys as strings allows me to "manufacture" keys from other fields (e.g., username + ":" + str(homework_id)). 

This may seem a small point, but it's the inability of using str ids on top of Datastore that has kept me from using DAL in my latest work. 

Luca

[Niphlod]

uhm. isn't better to use something like, e.g. snowflake to generate random - but integer and incrementing - ids ?

I get the "we'd like to use whatever string we may" but there are some assumptions here and there that benefit (from a design standpoint) having an "integer-like" pkey.