Towards Plone 3.0 compatibility
Maurits van Rees
Today I started working on Plone 3 compatibility of getpaid. Below is
a list of the most important things that I did. You can also skip to
the summary at the bottom.
buildout
--------
Most importantly, I created a buildout for Plone 3. I copied the
current buildout of 2.5 and modernised it based on what I got out of a
fresh paster buildout. For instance, I replaced
inquant.recipe.download with the default plone.recipe.plone. If there
is a compelling reason for using that inquant recipe in the trunk
buildout I would be interested in hearing it.
Five was removed as Zope 2.10 has a more recent version. BTW, I also
removed Five from the develop-products directory on trunk, as Five was
also installed in parts/five, so this was double.
CMFonFive was removed as that has been integrated in the CMF used by
Plone 3. URL:
https://getpaid.googlecode.com/svn/trunk/getpaid-30.buildout
The current svn layout is not what I am used to, so I may make some
wrong decisions about where to put things. If it is better to move
this buildout to the branches directory, just say so.
Fixing imports
--------------
I have not made a branch of anything for the 3.0 work, except the
buildout. I added backwards compatibility for things that have
changed in 3.0.
I think all unit tests (run with bin/test from the buildout directory)
passed immediately. So great work everyone, especially Kapil
probably! There only were some DeprecationWarnings which I fixed.
In Zope 2.10 (needed for 3.0) zope.app.annotation was moved to
zope.annotation, which unfortunately needs some messing with
sys.modules in order to keep a configure.zcml working when it uses
this package.
Some events were moved in 2.10, so I added some try/excepts.
After these fixes, all unit tests (called with bin/test) work on 2.5
and 3.0 without DeprecationWarnings.
Tests in PloneGetPaid
---------------------
I worked on getting the tests of PloneGetPaid to run on both 2.5 and
3.0. These are run with:
bin/instance test -s Products.PloneGetPaid
I saw two failures on 2.5; also some on 3.0, including lots of errors.
So some more fixes were done there. The change having the biggest
impact is probably that I moved the call to installProducts('Five') in
tests/base.py. Before I did that, I got a ComponentLookupError about
IIntIds when clicking "Donate!" in line 93 of
tests/test_functional_browser_checkout.txt
After that I ran into errors a few lines down. The testbrowser gave
ambiguity errors as for example he did not know whether with the
'City' control we meant the billing or the shipping city. I fixed a
few errors like that. Probably more fixes like that need to be done.
Then I ran into errors in that same file when setting the state, as
the state box was empty. So it seems the vocabulary for that field is
not hooked up correctly in the test. Feel free to fix that test, as
for me it is long past the time to go home now. :)
Summary
-------
- There is a buildout for getpaid on Plone 3.0
https://getpaid.googlecode.com/svn/trunk/getpaid-30.buildout
- The getpaid product (PloneGetPaid) and packages (getpaid.core and
hurry.workflow) have gotten compatibility fixes on trunk. So no
branches needed for now.
- All unit tests pass on 2.5 and 3.0.
- There are test failures for Products.PloneGetPaid on both 2.5 and
3.0.
Bye,
--
Maurits van Rees | http://maurits.vanrees.org/ [NL]
Work | http://zestsoftware.nl/
"I am trapped in a Klein Bottle and can't get in."
John Lenton
>
> Five was removed as Zope 2.10 has a more recent version. BTW, I also
> removed Five from the develop-products directory on trunk, as Five was
> also installed in parts/five, so this was double.
the one in develop-products was a very specific version to avoid a
bug (in Five). Maybe it has been fixed; did you check?
hmm... the problem isn't in an issue, but you can read about it in
the archive,
http://groups.google.com/group/getpaid-dev/browse_thread/thread/b57969810f50ac44/09fffb629dd5deea
Cheers,
--
John Lenton (jo...@except.com.ar) -- http://except.com.ar/
except - responsible free software developers for hire.
Kapil Thangavelu
On 10/17/07, Maurits van Rees <maurits...@gmail.com> wrote:
>
> Hi everyone!
>
> Today I started working on Plone 3 compatibility of getpaid. Below is
> a list of the most important things that I did. You can also skip to
> the summary at the bottom.
>
>
> buildout
> --------
>
> Most importantly, I created a buildout for Plone 3. I copied the
> current buildout of 2.5 and modernised it based on what I got out of a
> fresh paster buildout. For instance, I replaced
> inquant.recipe.download with the default plone.recipe.plone. If there
> is a compelling reason for using that inquant recipe in the trunk
> buildout I would be interested in hearing it.
>
> Five was removed as Zope 2.10 has a more recent version. BTW, I also
> removed Five from the develop-products directory on trunk, as Five was
> also installed in parts/five, so this was double.
>
> CMFonFive was removed as that has been integrated in the CMF used by
> Plone 3. URL:
>
> https://getpaid.googlecode.com/svn/trunk/getpaid-30.buildout
>
> The current svn layout is not what I am used to, so I may make some
> wrong decisions about where to put things. If it is better to move
> this buildout to the branches directory, just say so.
this layout is there to make it easy for us to tag/branch the whole project.
>
>
> Fixing imports
> --------------
>
> I have not made a branch of anything for the 3.0 work, except the
> buildout. I added backwards compatibility for things that have
> changed in 3.0.
>
> I think all unit tests (run with bin/test from the buildout directory)
> passed immediately. So great work everyone, especially Kapil
> probably! There only were some DeprecationWarnings which I fixed.
all the pgp tests get mad props from the sprinters.. i was mostly
working on getpaid.* tests cause they don't need to wait for plone to
start.
>
> In Zope 2.10 (needed for 3.0) zope.app.annotation was moved to
> zope.annotation, which unfortunately needs some messing with
> sys.modules in order to keep a configure.zcml working when it uses
> this package.
don't mess with sys.modules!.. eeek
use conditional imports, and if needed refactor to a base
compatibility module ie.
try:
from zope.annotation.interfaces import IAnnotation
except ImportError:
from zope.app.annotation.interfaces import IAnnotation
use zcml:condition (see pgp use of this for payment processors) to
import the zcml.
>
> Some events were moved in 2.10, so I added some try/excepts.
cool.
>
> After these fixes, all unit tests (called with bin/test) work on 2.5
> and 3.0 without DeprecationWarnings.
>
sweet.
great, thanks for making this happen.
cheers,
kapil
Kapil Thangavelu
> Maurits van Rees wrote:
> >
> > Five was removed as Zope 2.10 has a more recent version. BTW, I also
> > removed Five from the develop-products directory on trunk, as Five was
> > also installed in parts/five, so this was double.
>
> the one in develop-products was a very specific version to avoid a
> bug (in Five). Maybe it has been fixed; did you check?
>
> hmm... the problem isn't in an issue, but you can read about it in
> the archive,
>
> http://groups.google.com/group/getpaid-dev/browse_thread/thread/b57969810f50ac44/09fffb629dd5deea
>
this should be a non issue with zope 2.10 and the distributed five
product won't be needed anymore in plone 3.
we will also need to adjust the scripts/nightly to take this into
account for our tarball generation.
cheers,
kapil
Maurits van Rees
> don't mess with sys.modules!.. eeek
>
> use conditional imports, and if needed refactor to a base
> compatibility module ie.
Getting it right without sys.modules in python code works fine. The
zcml is the problem. In Zope 2.10 the python code in
zope.app.annotation has nice DeprecationWarnings. But the
configure.zcml is not there anymore, so you cannot include the package
in your own zcml file:
IOError: [Errno 2] No such file or directory:
'.../parts/zope2/lib/python/zope/app/annotation/configure.zcml'
BTW, plone.memoize uses my trick too, so I do not feel too bad having
added it to getpaid. But see below.
> try:
> from zope.annotation.interfaces import IAnnotation
> except ImportError:
> from zope.app.annotation.interfaces import IAnnotation
>
> use zcml:condition (see pgp use of this for payment processors) to
> import the zcml.
I tried that earlier, but failed. Hm, there should be a way. Ah yes,
you need to figure out the right imports to test for with a zcml
condition. It works with this:
<!-- zope.annotation is used in Zope 2.10 -->
<include
zcml:condition="installed zope.annotation"
package="zope.annotation" />
<!-- zope.app.annotation is used in Zope 2.9
We know that zope.app.annotation.tests is only installed there. -->
<include
zcml:condition="installed zope.app.annotation.tests"
package="zope.app.annotation" />
I added fixes to all the code so I do not mess with sys.modules
anymore. And it still works in 2.5 and 3.0. :)
Kapil Thangavelu
>
> On Thu, Oct 18, 2007 at 10:13:32AM -0400, Kapil Thangavelu wrote:
> > don't mess with sys.modules!.. eeek
> >
> > use conditional imports, and if needed refactor to a base
> > compatibility module ie.
>
> Getting it right without sys.modules in python code works fine. The
> zcml is the problem. In Zope 2.10 the python code in
> zope.app.annotation has nice DeprecationWarnings. But the
> configure.zcml is not there anymore, so you cannot include the package
> in your own zcml file:
>
> IOError: [Errno 2] No such file or directory:
> '.../parts/zope2/lib/python/zope/app/annotation/configure.zcml'
>
yes, i'm aware of this issue, hence the zcml condition stuff for zcml.
> BTW, plone.memoize uses my trick too, so I do not feel too bad having
> added it to getpaid. But see below.
just because there is someone does something bad somewhere else,
doesn't make it okay to keep repeating the same mistake, imo.
>
> > try:
> > from zope.annotation.interfaces import IAnnotation
> > except ImportError:
> > from zope.app.annotation.interfaces import IAnnotation
> >
> > use zcml:condition (see pgp use of this for payment processors) to
> > import the zcml.
>
> I tried that earlier, but failed. Hm, there should be a way. Ah yes,
> you need to figure out the right imports to test for with a zcml
> condition. It works with this:
>
> <!-- zope.annotation is used in Zope 2.10 -->
> <include
> zcml:condition="installed zope.annotation"
> package="zope.annotation" />
> <!-- zope.app.annotation is used in Zope 2.9
> We know that zope.app.annotation.tests is only installed there. -->
> <include
> zcml:condition="installed zope.app.annotation.tests"
> package="zope.app.annotation" />
>
> I added fixes to all the code so I do not mess with sys.modules
> anymore. And it still works in 2.5 and 3.0. :)
excellent, thanks!
cheers,
kapil
Maurits van Rees
> Maurits van Rees wrote:
> >
> > Five was removed as Zope 2.10 has a more recent version. BTW, I also
> > removed Five from the develop-products directory on trunk, as Five was
> > also installed in parts/five, so this was double.
>
> the one in develop-products was a very specific version to avoid a
> bug (in Five). Maybe it has been fixed; did you check?
It was Five 1.4.2. The point is that this was in
develop-products/externals.txt *and* also in parts/five. Ah, that
last one was added by jefroche only five days ago in buildout.cfg in
revision 856:
[five]
recipe = plone.recipe.distros
urls =
http://codespeak.net/z3/five/release/Five-1.4.2.tgz
The one in the externals.txt pointed to
svn://svn.zope.org/repos/main/Products.Five/tags/1.4.2
so they are the same really.
jefroche: next time also add it to the products section of the
instance please. :) Ah, in revision 857 you did it correctly for
test-recorder. Good. :) I fixed it for five yesterday.
Summary
-------
Compared with one week ago, Five was moved from develop-products/Five
to parts/five/Five and all keeps working fine.
Maurits van Rees
> > BTW, plone.memoize uses my trick too, so I do not feel too bad having
> > added it to getpaid. But see below.
>
> just because there is someone does something bad somewhere else,
> doesn't make it okay to keep repeating the same mistake, imo.
This is the changeset of plone.memoize that introduces the sys.modules
hack: http://dev.plone.org/plone/changeset/15030
When you follow the link pointed to in that commit message you will go
to a mailling list thread started by me. At the end I present the
sys.modules hack as the way to solve this. So I am at the root of
this evil. :)
Might as well stop it myself then. I just fixed it in the
eXtremeManagement product, which is why I started that thread in the
first place.
I will see about fixing it in plone.memoize too. Yes, fixed here:
http://dev.plone.org/plone/changeset/17715
Thanks for keeping my on my toes Kapil. :)
Christopher Johnson
I am thinking we can add 3.0 compatibility to the release features given the update, which is very cool!
Do we have some volunteers for testing this on Plone 3.0? Can people please report in about how it is working?
Also, are there any updates on the status of the tests that are failing? I think Reed and Jean were the last working on those - can you let us know the current state?
Cheers,
Chris
www.ifpeople.net
t: 678-608-3408
130 Boulevard NE, #6
Atlanta, GA 30312
Maurits van Rees
> This is awesome - thanks, Maurits!
>
> I am thinking we can add 3.0 compatibility to the release features given the
> update, which is very cool!
>
> Do we have some volunteers for testing this on Plone 3.0? Can people please
> report in about how it is working?
On 3.0 portlets have been switched to new style portlets today. And
they are added as context portlets to the portal root when you
reinstall. It could be done differently, see some comments in
Extensions/install.py:install_plone3_portlets. But this works.
Hm, it *does* add portlets a second time when you reinstall again...
On 2.5 this code never gets called so it uses the old and trusted way.
> Also, are there any updates on the status of the tests that are failing? I
> think Reed and Jean were the last working on those - can you let us know the
> current state?
Currently I see one failure both on 2.5 and 3.0 and one extra on 3.0.
Failure 2.5 and 3.0
-------------------
Traceback is here (adapter slightly for readability):
----------------------------------------------------------------------
File ".../PloneGetPaid/tests/test_functional_browser_checkout.txt",
line 100, in test_functional_browser_checkout.txt
Failed example:
browser.getControl(name='form.bill_state').value = ('US-HI',)
Exception raised:
Traceback (most recent call last):
File ".../lib/python/zope/testing/doctest.py", line 1348, in __run
compileflags, 1) in test.globs
File "<doctest test_functional_browser_checkout.txt[66]>", line 1, in ?
browser.getControl(name='form.bill_state').value = ('US-HI',)
File ".../lib/python/zope/testbrowser/browser.py", line 91, in __setattr__
object.__setattr__(self, name, value)
File ".../lib/python/zope/testbrowser/browser.py", line 483, in fset
self.mech_control.value = value
File ".../lib/python/ClientForm.py", line 1926, in __setattr__
self._set_value(value)
File ".../lib/python/ClientForm.py", line 1947, in _set_value
self._single_set_value(value)
File ".../lib/python/ClientForm.py", line 1970, in _single_set_value
on, off = self._get_items(value[0])
File ".../lib/python/ClientForm.py", line 1954, in _get_items
raise ItemNotFoundError(
ItemNotFoundError: insufficient items with name 'US-HI'
----------------------------------------------------------------------
This tests that you can set a state when going to
http://localhost:8080/.../@@getpaid-checkout-wizard
And that works just fine in the browser, both in 2.5 and 3.0. I see
this in the html code:
<option value="US-HI">Hawaii</option>
So this goes fine in reality, but not in the test. So the test is
somehow wrong.
Extra failure 3.0
-----------------
That extra one is in PloneGetPaid/tests/test_notifications.py. It
throws an AttributeError: context in PloneGetPaid/notifications.py,
line 84. site.context does not exist in 3.0. We have:
82 site = component.getSiteManager()
On Plone 2.5 this gives:
(Pdb) site
<Products.Five.site.localsite.FiveSiteManager object at 0xb513acc>
And on 3.0 this gives:
(Pdb) site
<PersistentComponents /plone>
So if someone knows a way to make this work in 3.0 too, that would be
nice. I have no idea currently.
Kapil Thangavelu
> Extra failure 3.0
> -----------------
>
> That extra one is in PloneGetPaid/tests/test_notifications.py. It
> throws an AttributeError: context in PloneGetPaid/notifications.py,
> line 84. site.context does not exist in 3.0. We have:
>
> 82 site = component.getSiteManager()
>
> On Plone 2.5 this gives:
>
> (Pdb) site
> <Products.Five.site.localsite.FiveSiteManager object at 0xb513acc>
>
> And on 3.0 this gives:
>
> (Pdb) site
> <PersistentComponents /plone>
>
> So if someone knows a way to make this work in 3.0 too, that would be
> nice. I have no idea currently.
this code is a gymastics exercise to the mailhost so we can send out
email, 'the plone way'.
the notification handler is an event subscriber on the order's
workflow transition. it needs to access context to get the mail host.
previously we used an implementation detail of the site manager, and
got it from there. zope 2.10 (z 3.3) has changed a bit in this regard
and has a new way of doing persistent components (much better for
fwiw).
so the question is how to get access to the portal context in plone
3.0 from the site manager, as i recall... cmf 2.1 and plone 3.0, both
utilize five.lsm, which figures out some ways to do just that so it
can magic acquisition chains onto utilities, i think a look through
there should give an idea.. i'm not sure if the tools as utilites
debacle ever got resolved to having tools as utilities, but if they
are then this becomes pretty easy (ie.
getUtility(IPortalURL).getPortalObject()), else.. we'll see.
hth,
kapil
Jean-François Roche
This won't ever work in testbrowser as this needs javascript to work. Just
disable your javascript on your browser and it won't work anymore. This is
anyway bad, Plone takes care to work with and without JS, getpaid should
continue the same way. Reed gave a hint to fix this, print there a simple
input text and in javascript replace this input with the drop down list.
Regards,
Jeff
Maurits van Rees
>
> On 10/19/07, Maurits van Rees <maurits...@gmail.com> wrote:
>
> > Extra failure 3.0
> > -----------------
> >
> > That extra one is in PloneGetPaid/tests/test_notifications.py.
>
> email, 'the plone way'.
>
> the notification handler is an event subscriber on the order's
> workflow transition. it needs to access context to get the mail host.
> previously we used an implementation detail of the site manager, and
> got it from there. zope 2.10 (z 3.3) has changed a bit in this regard
> and has a new way of doing persistent components (much better for
> fwiw).
I fixed this with a try/except so it works on 2.5 and 3.0. See
changeset 966:
site = component.getSiteManager()
- portal = site.context
- mailer = portal.MailHost
+ try:
+ portal = getToolByName(site, 'portal_url').getPortalObject()
+ except AttributeError:
+ # BBB for Zope 2.9
+ portal = site.context
+ mailer = getToolByName(portal, 'MailHost')
(Trying getToolByName(site, 'MailHost') would have worked as well, but
we need the portal object too a few lines further on.)
Now both on 2.5 and 3.0 we are down to just that error in the states,
which is caused by Javascript not being available in the testbrowser.
Maurits van Rees
> > ItemNotFoundError: insufficient items with name 'US-HI'
> > ----------------------------------------------------------------------
> >
> > This tests that you can set a state when going to
> > http://localhost:8080/.../@@getpaid-checkout-wizard
> > And that works just fine in the browser, both in 2.5 and 3.0. I see
> > this in the html code:
> >
> > <option value="US-HI">Hawaii</option>
> >
> > So this goes fine in reality, but not in the test. So the test is
> > somehow wrong.
>
> This won't ever work in testbrowser as this needs javascript to work. Just
> disable your javascript on your browser and it won't work anymore. This is
> anyway bad, Plone takes care to work with and without JS, getpaid should
> continue the same way. Reed gave a hint to fix this, print there a simple
> input text and in javascript replace this input with the drop down list.
I changed the default javascript-less code so that it indeed just
shows a simple input text field. The user then has to somehow know
himself that he needs to type "US-HI" for Hawaii or say "NL-ZH" for
The Netherlands, South Holland, which is not good, but at least it
gives the user a small chance.
I fixed the test accordingly. It now breaks a few lines below that
because it cannot find a link to Disclaimer. That might be javascript
related too.
BTW, I can understand that the State field is needed for the USA, but in
The Netherlands it is not necessary.