Opinions on paginate changes
Randall
feedback.
The first is SQLAlchemy support. paginate.py uses SQLObject's
SelectResults to order the result set. SA has similar functionality in
an extension called SelectResults which can be used as a near drop in
replacement.
The second is session support. The URI's get long and complex, so if
sessions are in use, only the change information has to be passed in
the URI and everything else (like the data query params) can be stored
in a session. I've currently got it as an option in the decorator
params. @paginate(..., use_session=True). One thing I'm not sure
about yet is how to keep session data for the DataGrid widget from
overwriting another paginating DataGrid instance's session data. Right
now I'm using the id (id(func)) of the func passed into the decorator's
"decorated" function.
Third. All field sorting. Instead of only sorting by one field, each
sorting selection will build on the last.
My current implementation of these features does not change the default
behavior. The extra sorting capability is only present when using
sessions. Also it includes this bug fix:
http://trac.turbogears.org/turbogears/ticket/1044
which I submitted yesterday.
I'd like to hear opinions on these additions a well as requests for
other features.
Randall
Jorge Godoy
> The first is SQLAlchemy support. paginate.py uses SQLObject's
> SelectResults to order the result set. SA has similar functionality in
> an extension called SelectResults which can be used as a near drop in
> replacement.
This will be gladly welcome! I was going to look at this as soon as I could,
but I confess that it might take a while for that...
> The second is session support. The URI's get long and complex, so if
> sessions are in use, only the change information has to be passed in
> the URI and everything else (like the data query params) can be stored
> in a session. I've currently got it as an option in the decorator
> params. @paginate(..., use_session=True). One thing I'm not sure
> about yet is how to keep session data for the DataGrid widget from
> overwriting another paginating DataGrid instance's session data. Right
> now I'm using the id (id(func)) of the func passed into the decorator's
> "decorated" function.
Hmmm... This isn't nice either. I can open two tabs on the same grid to work
with it -- e.g. making comparisons -- and it would probably be a problem.
Using this kind of static data when one can have multiple accesses to
something doesn't sound like it will work to me. I can't say anything,
though, since I haven't yet used sessions here (don't have the need for it).
> Third. All field sorting. Instead of only sorting by one field, each
> sorting selection will build on the last.
What do you mean? Sorting by multiple fields? Or clicking a column to sort
data? And what happens with data on screen: will it all change or just the
current set being viewed? (I mean, if I am seeing records 1-10 sorted by
firstname and somehow when sorting by lastname I get 14-5, will it show me
14-5 or will it sort 1-10 using the selected column?)
--
Jorge Godoy <jgo...@gmail.com>
Randall
> > The second is session support. The URI's get long and complex, so if
> > sessions are in use, only the change information has to be passed in
> > the URI and everything else (like the data query params) can be stored
> > in a session. I've currently got it as an option in the decorator
> > params. @paginate(..., use_session=True). One thing I'm not sure
> > about yet is how to keep session data for the DataGrid widget from
> > overwriting another paginating DataGrid instance's session data. Right
> > now I'm using the id (id(func)) of the func passed into the decorator's
> > "decorated" function.
>
> Hmmm... This isn't nice either. I can open two tabs on the same grid to work
> with it -- e.g. making comparisons -- and it would probably be a problem.
>
Yea, you're right. Multiple windows or tabs of the same browser use
the same session.
> > Third. All field sorting. Instead of only sorting by one field, each
> > sorting selection will build on the last.
>
> What do you mean? Sorting by multiple fields? Or clicking a column to sort
> data? And what happens with data on screen: will it all change or just the
> current set being viewed? (I mean, if I am seeing records 1-10 sorted by
> firstname and somehow when sorting by lastname I get 14-5, will it show me
> 14-5 or will it sort 1-10 using the selected column?)
It works like the current implementation, but remembers previous sorts.
So if I sort by age, then again by name, the data will be sorted by
age then name. The current implementation sorts all data and goes to
page one, which seems like reasonable behaviour to me. So from your
example, it would show records 1-10. It's a minor change. Do you find
it would be better to sort only the currently viewed set? Maybe both
would be good.
Randall
Jorge Godoy
> Yea, you're right. Multiple windows or tabs of the same browser use
> the same session.
So this is a problem on your implementation using sessions :-( Too bad... I
was liking the idea...
> It works like the current implementation, but remembers previous sorts.
> So if I sort by age, then again by name, the data will be sorted by
> age then name. The current implementation sorts all data and goes to
> page one, which seems like reasonable behaviour to me. So from your
> example, it would show records 1-10. It's a minor change. Do you find
> it would be better to sort only the currently viewed set? Maybe both
> would be good.
Having both would be good. I don't know which one is better, it depends on
the application and what users expect from it.
--
Jorge Godoy <jgo...@gmail.com>
Randall
The current implementation uses the most prevalent approach from what
I've seen. The change I'm proposing will go unnoticed by most users,
but is quite nice when needed. For an example, look at this grid on a
website I built (non-TG).
http://starkvegas.com/real_estate/?_action_quickSearch=all&search_type=1
If you click on Price, then Rooms, the listings are sorted by rooms
then price. If you don't require the feature, it doesn't have any
negative impact.
Jorge Godoy
> The current implementation uses the most prevalent approach from what
> I've seen. The change I'm proposing will go unnoticed by most users,
> but is quite nice when needed. For an example, look at this grid on a
> website I built (non-TG).
>
>
> http://starkvegas.com/real_estate/?_action_quickSearch=all&search_type=1
>
> If you click on Price, then Rooms, the listings are sorted by rooms
> then price. If you don't require the feature, it doesn't have any
> negative impact.
I liked that approach. You do what I asked: change the records that appear on
the grid instead of just sorting the visible records. This is what makes most
sense to me in most cases. But, as I said, some people expect it
different... :-)
--
Jorge Godoy <jgo...@gmail.com>
Kevin Dangoor
>
> I'd like to add some features to TG's pagination and would like some
> feedback.
>
> The first is SQLAlchemy support. paginate.py uses SQLObject's
> SelectResults to order the result set. SA has similar
> functionality in
> an extension called SelectResults which can be used as a near drop in
> replacement.
Yeah,we need this. A few months back, we were trying to reconcile the
differences between this paging implementation and that of
WebHelpers. I think this is a fine solution for now. Pagination also
needs docs so that we can declare it as a feature :)
Kevin
Jose Soares
I'm using sqlalchemy and I have in my dev.cfg the line:
sqlalchemy.dburi="postgres://pippo:pluto@localhost/mydb"
I can run
$ python prj-start.py
but:
$ tg-admin toolbox
Traceback (most recent call last):
File "/usr/local/bin/tg-admin", line 7, in ?
sys.exit(
File
"/usr/lib/python2.4/site-packages/TurboGears-0.9a9-py2.4.egg/turbogears/command/base.py",
line 351, in main
command.run()
File
"/usr/lib/python2.4/site-packages/TurboGears-0.9a9-py2.4.egg/turbogears/command/base.py",
line 297, in run
root = SecureObject(toolbox.Toolbox(),from_any_host(self.hostlist),
exclude=['noaccess'])
File
"/usr/lib/python2.4/site-packages/TurboGears-0.9a9-py2.4.egg/turbogears/toolbox/base.py",
line 95, in __init__
self.toolbox = self.get_tools()
File
"/usr/lib/python2.4/site-packages/TurboGears-0.9a9-py2.4.egg/turbogears/toolbox/base.py",
line 107, in get_tools
setattr(self, i.name, tool())
File
"/usr/lib/python2.4/site-packages/TurboGears-0.9a9-py2.4.egg/turbogears/toolbox/catwalk/__init__.py",
line 112, in __init__
self.model = turbogears.util.get_model()
File
"/usr/lib/python2.4/site-packages/TurboGears-0.9a9-py2.4.egg/turbogears/util.py",
line 118, in get_model
package = __import__(package_name, {}, {}, ["model"])
File "/home/jose/turbosicer/sicer/model/__init__.py", line 6, in ?
from sicer.model import tables
File "/home/jose/turbosicer/sicer/model/tables.py", line 7, in ?
database.bind_meta_data()
File
"/usr/lib/python2.4/site-packages/TurboGears-0.9a9-py2.4.egg/turbogears/database.py",
line 49, in bind_meta_data
get_engine()
File
"/usr/lib/python2.4/site-packages/TurboGears-0.9a9-py2.4.egg/turbogears/database.py",
line 32, in get_engine
raise KeyError("No sqlalchemy database config found!")
KeyError: 'No sqlalchemy database config found!
Any ideas?
j
Tim Van Steenburgh
A feature I'd like to see added: add a sort_value kw arg to the DataGrid.Column constructor. There are lots of times where the value I want to sort on for a column and the value I actually want to display for that column are different.
Randall
> I'm currently using PaginateDataGrid to display sortable/pageable SQLAlchemy
> result sets (which are just lists by default). Had to modify
> my solution if the
> "right" way, but it works. Diffs are attached if anyone cares to look.
>
Using SelectResults for SQLAlchemy has the advantage of only pulling
records from the database that will be displayed and also the sorting
is done in the database. If you've got a large dataset (several pages)
this can make a difference. In addition, it keeps the code simpler in
paginate.py because SQLObject's and SQLAlchemy's SelectResults have the
same API. Creating a SelectResults instance is simple.
from sqlalchemy.ext.selectresults import SelectResults
sr = SelectResults(query)
If I read your code right, it looks like you added sorting for lists,
which is another feature that is needed I think.
Randall
Randall
http://trac.turbogears.org/turbogears/ticket/1115
Check it out and let me know what you think. It has SA support via
SelectResults and multi-field sorting. Use SA SelectResults like this:
from sqlalchemy.ext.selectresults import SelectResults
query = session.query(MySAClass)
sr = SelectResults(query)
Use sr for SQLAlchemy where you would normally use MySOClass.select()
for SQLObject.
Jorge Godoy
Is using sessions mandatory to have SA support? Sessions have that known
scalability problem besides the other we've pointed out here before...
--
Jorge Godoy <jgo...@gmail.com>
Randall
it like this:
from turbogears.database import session as tg_session
The main thing is that SA support takes a SA SelectResults instance
just like SO support takes a SO SelectResults instance.
There is no use of cherrypy sessions. Everything is passed thru the
URI.
Jorge Godoy
Cool!
I vote for this being included on TG, then. ;-)
I'll read the code better on your bug and see if I commit it if no one has any
objection to it.
--
Jorge Godoy <jgo...@gmail.com>
Randall
1. Line 130 is where I made a fix for the handling of submitted form
data. It is passed via URI, but must first be converted from a dict in
the case of CompoundWidget data. More info in the docstring. I just
wonder if there is a better way to do this.
2. The ordering data is stored as a dict and passed in the URI. Check
how I convert it from string to dict in line 261. I'm trying to keep
is secure since it is untrusted input.
3. Line 248 - I sure would like to be more explicit when checking for
an SQLObject column. InstanceType is quite generic. Any ideas.
I did not make any API changes (that I know of). I don't think there
are any existing tests for this module, so I can't verify I didn't
break something. I'll have to write tests from scratch, but I wanted
to get some feedback on it ASAP.
jose
my dev.cfg is like this one
---------------
[global]
sqlalchemy.dburi="postgres://pippo:pluto@localhost/mydb"
server.environment="development"
autoreload.package="gs"
[/static]
staticFilter.on = True
staticFilter.dir = "static"
---------------
but database.py doesn't find the token 'sqlalchemy.dburi' and I can't understand why. =-O
26 def get_engine():
27 "Retreives the engine based on the current configuration"
28 global _engine
29 if not _engine:
30 dburi = config.get("sqlalchemy.dburi")
31 if not dburi:
32 raise KeyError("No sqlalchemy database config found!")
33 _engine = sqlalchemy.create_engine(dburi)
34 metadata.connect(_engine)
35 elif not metadata.is_bound():
36 metadata.connect(_engine)
37 return _engine
Kevin Dangoor
not yet supported.
Kevin
--
Kevin Dangoor
TurboGears / Zesty News
email: k...@blazingthings.com
company: http://www.BlazingThings.com
blog: http://www.BlueSkyOnMars.com
Elvelind Grandin
--
cheers
elvelind grandin
Elvelind Grandin
both SA and identity. forgot to the it to get the sqlalchemy.dburi and
friends from dev.cfg at startup. should be fixed now.
--
cheers
elvelind grandin
Randall
jose
>I'm using it in my current app now and made a few fixes.
>
>http://trac.turbogears.org/turbogears/ticket/1115
>
>
>>
>
>
I see these fix are for version 0.9a5.
I suppose I cannot use them on 0.9a9 :-(
j
Adam Jones
working on some documentation for paginate (
http://docs.turbogears.org/1.0/PaginateDecorator ) and would like to
make sure it stays current.
-Adam
Randall
Jorge Godoy indicated earlier in this thread that he would accept it,
though it hasn't been applied yet. The changes I made, if accepted,
would only impact your documentation in two ways. First, the addition
of SQLAlchemy and second, removal of the 'reversed' attribute.
In the patched version, fields are cummulatively sorted and the sort
info is stored in a dict named tg.paginate.ordering of the form:
{'fieldname': [0, True]}
where the 0 indicates the field position (for multiple sorted fields)
and True indicates Ascending sort order (False would be Descending).
SQLAlchemy support uses sqlalchemy.ext.selectresults to mimic the
behaviour of the SQLObject SelectResults. It expects a SelectResults
instance just as SO does.
Jorge Godoy
> Very nice documentation.
>
> Jorge Godoy indicated earlier in this thread that he would accept it,
> though it hasn't been applied yet. The changes I made, if accepted,
> would only impact your documentation in two ways. First, the addition
> of SQLAlchemy and second, removal of the 'reversed' attribute.
I haven't done anything because I had no time -- yet. And I can't test it --
also "yet". But I'll have to take a look at a stopped pet project soon, and
there I'll try it.
If anybody else uses it and had no problems, please say it here.
I just don't want to break anything. :-)
--
Jorge Godoy <jgo...@gmail.com>
Randall
>
> If anybody else uses it and had no problems, please say it here.
>
> I just don't want to break anything. :-)
>
Agreed. The absence of tg.paginate.reversed could break someone's app,
but I hesitate to put it in because it would be cruft. On the other
hand, API changes shouldn't occur during beta. Suggestions?
That brings me to another subject, the interface to ordering. The
ordering attribute is what tracks the ordering of all the fields. For
three fields, it may look like this:
{'fieldone':[0, True], 'fieldtwo':[1, False], 'fieldthree':[2,
True]}
The values in the dictionary contain the field order and a boolean for
Ascending order. This structure was designed for easy internal usage,
not as a public API. Should the field ordering be exposed in some
other fashion?
For SQLAlchemy support, it accepts a SelectResults instance which is
very similar to SQLObject's SelectResults. I'm wondering if maybe it
should accept a query instance instead and create the SelectResults
instance internally. That might save the developer a little trouble.
So it would go from this:
from sqlalchemy.ext.selectresults import SelectResults
paginate_input = SelectResults(tg_session.query(MySAClass))
to this:
paginate_input = tg_session.query(MySAClass)
Note that I'm using tg_session (turbogears.databse.session) because I'm
not using ActiveMapper classes with a default session.
What do you think? I'm starting to prefer the latter. That way if
SelectResults becomes a problem later (not maintained, feature limited,
etc.), paginate can use a different approach internally without
breaking the API.
So I'm leaning towards:
1. Making a compatible reversed attribute.
2. Requiring only a SA query (not a SelectResults instance).
Thoughts?
Randall
field's sort direction. I didn't change SA support because a Query
instance can't contain clause information. The data object passed to
paginate will most likely have some conditions applied to it, which is
normally done using the select method in SA, but the select method
returns a list, which is not useful. The only way I see around this is
to pass a Query instance plus clauses to be applied to it, which I
think is awkward. So for now, requiring a SelectResults instance seems
to make the most sense.