WebHelpers developments

[Mike Orr]

WebHelpers is splitting into two projects. The "classic" WebHelpers will remain as-is, for compatibility with existing applications. I won't be developing it further, so it needs a maintainer if it's going to change any more. Its repository is https://bitbucket.org/bbangert/webhelpers . Its documentation is finally on ReadTheDocs (http://webhelpers.readthedocs.org/en/latest/), and a link from the Pyramid docs will be added soon.

I've started a WebHelpers2 project at https://github.com/mikeorr/WebHelpers2 . It will contain the core helpers (html, text, number, etc), and will be ported to Python 3. The large third-party subpackages will be spun off to standalone distributions. Feedgenerator is already on PyPI, and the Paginate and Grid developers are working on their distributions. Markdown and Textile have long been available separately and their WebHelpers subpackages offered little added value. The other Pylons-dependent helpers are gone too; use classic WebHelpers if you need those. There's no reason to port classic WebHelpers to Python 3 because Pylons is not being ported.

The biggest headache in WebHelpers maintenance has been the large third-party packages and the obscure Pylons depenencies, so this change will make WebHelpers2 more focused and easier to maintain.

I'm researching an object-based backend for the HTML tag builder with C speedups, but noting definitive yet.

Note that WebHelpers2 will be a distinct distro name and top-level package name, but it isn't yet, so if you install it now it'll overwrite your existing WebHelpers install. That won't be the case in production, because there are too many existing Pylons applications that would automatically upgrade to an incompatible version if it had the same distro name.

Feedgenerator users: the standalone distro is an independent derivation of the same Django original. It doesn't have the WebHelpers enhancements, and it probably inherits Django's longitude-latitude input rather than latitude-longitude. I'll discuss with the author whether he can merge the WebHelpers enhancements, but there's no word on that yet.

--
Mike Orr <slugg...@gmail.com>

[Jonathan Vanasco]

Is there a url for that Paginate project ?

I've been using this generic lib:

http://code.google.com/p/python-pagination/

Also, I don't know if you've considered it... but if there were an easy way to widgetize or templatize webhelpers so that it could generate HTML that is in line with Twitter-Bootstrap , Boilerplate, etc , i think that could be really useful and a big draw for people to adopt it.  

[Mike Orr]

On Thu, Nov 29, 2012 at 3:20 PM, Jonathan Vanasco <jvan...@gmail.com> wrote:

Is there a url for that Paginate project ?

http://pypi.python.org/pypi/paginate/0.3.2

However, the last release is 2008. But I expect the next release will be here.
 

--
Mike Orr <slugg...@gmail.com>

[Christoph Haas]

On 30.11.2012 17:51, Mike Orr wrote:
> On Thu, Nov 29, 2012 at 3:20 PM, Jonathan Vanasco <jvan...@gmail.com
> <mailto:jvan...@gmail.com>> wrote:
>
> Is there a url for that Paginate project ?
>
> http://pypi.python.org/pypi/paginate/0.3.2
>
> However, the last release is 2008. But I expect the next release will be
> here.

For quite a while the "paginate" project was maintained as part of the
"webhelpers" module. As discussed with Mike I will make it a standalone
module again. I'm right now working on that. Expect a releaes at
http://pypi.python.org/pypi/paginate/ soon. I will announce that on this
list, too.

Cheers
Christoph
--
A distributed system is one in which I cannot get something done
because a machine I've never heard of is down. (Leslie Lamport)

[Zak]

Christoph -

Would you mind also putting the paginate module on github? Will the paginate update support both python 2.7 and python 3?

[Christoph Haas]

I'm actively using GitHub for the development already. You can find the
project here:
https://github.com/Signum/paginate

Yes, "paginate" now supports Python 3.

It does not have the exact same API as webhelpers.paginate for Pylons. I
ignorantly simplified the module, made it independent from Pylons or
Pyramid and threw out dozens of lines of magical URL generation code.

There is a TODO file that shows what I'm working on. Currently my
biggest challenge is moving the SQLAlchemy stuff into a seperate module.
My idea was to let "paginate" just work on any list-like object.
Additional functionality could go into further PyPi modules like
"paginate-sqlalchemy" which would subclass "paginate" but override a
method to make the Page() object work with SQLAlchemy Selects or
ORM-mapped objects. That would reduce the number of dependencies and
allow others to contribute pagination functionality for other kinds of
data objects. I would probably provide a MongoEngine pagination
extension, too.

Feel free to track the GitHub project. I'm open to contributions, ideas
and feedback and grateful for ideas and beta testers.

�Christoph

> --

[Christoph Haas]

I have just released version 0.4.0 of the paginate module:
http://pypi.python.org/pypi/paginate/0.4.0

Changes:

- Module is now standalone and not included as webhelpers.paginate any more.
Once the module was deemed stable, webhelpers can drop it.
- API overhaul. You shouldn't try to include this new version in your
Pylons/Pyramid
projects instead of webhelpers.paginate. It will work slightly
differently.
But I'm sure you'll love it. :)
- No more direct support for SQLAlchemy Select and ORM/Query objects.
This will
follow short in an additional Python module. I decided to put support
for further
data sources (e.g. SQLAlchemy, MongoEngine, CouchDB) into seperated
modules.
That should remove functionality not everybody needs and make it
easier for
other contributors to write paginate.Page subclasses.
- Python 3 compatibility (yeah!)
- Improved test suite
- Much simpler URL generation. Does not depend on any web framework.
- More pythonic.

I will add further documentation and examples to "readthedocs". The
module works for me already. But I would still consider it "beta".
Feedback welcome.

�Christoph

[Mike Orr]

On Thu, Dec 6, 2012 at 5:56 AM, Christoph Haas <em...@christoph-haas.de> wrote:

I have just released version 0.4.0 of the paginate module:
http://pypi.python.org/pypi/paginate/0.4.0

Changes:

- Module is now standalone and not included as webhelpers.paginate any more.
  Once the module was deemed stable, webhelpers can drop it.

WebHelpers 1.3 won't be changing so paginate will remain there.

WebHelpers2 is a separate repository and will be a separate PyPI distribution, and paginate is already deleted in it.
 

[Mike Orr]

On Tue, Dec 4, 2012 at 7:36 AM, Christoph Haas <em...@christoph-haas.de> wrote:

Yes, "paginate" now supports Python 3.

It does not have the exact same API as webhelpers.paginate for Pylons. I
ignorantly simplified the module, made it independent from Pylons or
Pyramid and threw out dozens of lines of magical URL generation code.

Yaay..
 

[Zak]

When will WebHelpers2 available on PyPI?

[Mike Orr]

When it's ready, :) I may have an alpha ready this weekend. For the final I want to see if my tag-builder idea is feasable. I tracked down the non-template generator I was thinking of, but it didn't do what I thought.

On Mon, Dec 10, 2012 at 3:19 AM, Zak <zakd...@gmail.com> wrote:

When will WebHelpers2 available on PyPI?

--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To view this discussion on the web visit https://groups.google.com/d/msg/pylons-discuss/-/YoAC9wgWik8J.
To post to this group, send email to pylons-...@googlegroups.com.
To unsubscribe from this group, send email to pylons-discus...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en.

--
Mike Orr <slugg...@gmail.com>

[Laurent DAVERIO]

Hello Christoph, hello list,

keeping in mind the warnings about the beta quality of the software, I
have tried integrating it in an existing Pyramid project.

Long story: I wanted to replace the Xapian indexer with Apache-Solr,
using the "sunburnt" module to interface Python with Solr.

With Xapian, I could retrieve all results of an index query, and feed
them into webhelpers.paginate.Page. Retrieving all the documents would
take 2 seconds on my old machine. Although inefficient, it was
acceptable for the users, and easy to program.

But with Solr, a query retrieving all documents takes much longer
(that's why Solr defaults to "partial" queries, I guess), so pagination
need to be integrated upstream. It can take place in Solr itself :

> si = SolrInterface( url = 'http://localhost:8080/solr',
> http_connection = httplib2.Http(cache='/tmp/solr_cache')
> )
> query = si.query(**search_args).paginate(start=10, rows=30)
> results = query.execute()

This works fine, but then you need to setup "fake" results if you want
to use the paginate module, something like :

> page = Page([], item_count=results.result.numFound)

Luckily, the sunbunrt query object provides a __getitem__ method, so I
can write :

> page = Page(query)

This is a very welcome change as compared with webhelpers.paginate,
where only SQLAlchemy query were allowed :-)

==

IMHO, the pager needs some more polish:

1/ I had to handle URL generation myself, here is how I did is (cleaner
solutions welcome :-))

> qsd = urlparse.parse_qs(request.query_string)
> qsd.update(page='$page')
> page_url = request.path_url + '?' + urllib.urlencode(qsd,True).replace('%24page', '$page')

2/ Unicode characters such as "�" and "�" (angled double quotes) were
not accepted in symbol_previous and symbol_next, so I kept the defaults
("<" and ">").

3/ The paginate module has lost webhelpers.paginate's ability to handle
partial page loads with AJAX (via the "onclick" argument).

Hope this helps,

Laurent.

[Laurent DAVERIO]

Hello again,

after due consideration, the removal of "onclick" is not such a problem,
it even allows for cleaner HTML code. I added the following Javascript
and it solved the problem for me:

> $('#${prefix}pager a').click(function () {
> var partial_url = this.href + (this.href.indexOf('?')==-1 ? '?' : '&') + 'partial=1';
> $('#results').load(partial_url);
> return false
> }

I think I'll add "paginate" to my toolbox permanently. Thanks for the
good work :-)

Laurent.

> IMHO, the pager needs some more polish:
>
> 1/ I had to handle URL generation myself, here is how I did is (cleaner
> solutions welcome :-))
>
>> qsd = urlparse.parse_qs(request.query_string)
>> qsd.update(page='$page')
>> page_url = request.path_url + '?' + urllib.urlencode(qsd,True).replace('%24page', '$page')
>

> 2/ Unicode characters such as "�" and "�" (angled double quotes) were

[Laurent DAVERIO]

> 2/ Unicode characters such as "�" and "�" (angled double quotes) were

> not accepted in symbol_previous and symbol_next, so I kept the defaults
> ("<" and ">").

This one can be solved easily, too: in file paginate/__init__.py, change
line 541 from :

tag_string += '{0}</{1}>'.format(text, tag)
to:
tag_string += u'{0}</{1}>'.format(text, tag)

For good measure, you should probably apply the same change to lines 534
and 537.


Laurent.