[Plone-developers] plone.api – back to the future?

38 views
Skip to first unread message

Malthe Borch

unread,
Feb 26, 2012, 3:39:31 AM2/26/12
to Plone Developers
I have been seeing some checkins on Github to a `plone.api` package.

Where's the PLIP for this? Zope tried to do the same very thing with
"zapi" and it's been deprecated since Zope 3.5.

I think there's a two camps in terms of where and how Plone's "app"
code should live, i.e. fully split up into atoms of responsibility
with the `plone.app.*` family (the current state of affairs; this is
the majority), or as a single, or a couple of major eggs, e.g. `Plone`
(wishful thinking; the minority).

Where does this new API fit in? Arguably in the second camp, because
it's got to have dependencies to a great many `plone.app.*` eggs
and/or CMF products. If you update a dependency, you need to update
the API. It's the "mother egg" and her dependencies are strong.

I understand the idea of approaching the problem of Plone's
problematic codebase from the outside, taming the beast with clean,
modernized wrappers, but it doesn't make much sense in terms of the
dependency graph -and- it's a step away from eventually cleaning up
and refactoring the codebase altogether:

– get rid of the very deep class hierarchy of Plone-objects;
– lose code weight; currently it's likely an inverse 80/20 rule (20%
code used by 80% installations).
– refactor and improve existing code so that it fits the bill as an "API".

Add to this that existing third party code will only very slowly or
never migrate to the API.

\malthe

------------------------------------------------------------------------------
Virtualization & Cloud Management Using Capacity Planning
Cloud computing makes use of virtualization - but cloud computing
also focuses on allowing computing to be delivered as a service.
http://www.accelacomm.com/jaw/sfnl/114/51521223/
_______________________________________________
Plone-developers mailing list
Plone-de...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/plone-developers

Jan-Carel Brand

unread,
Feb 26, 2012, 6:10:20 AM2/26/12
to Malthe Borch, Plone Developers
On Sun, 2012-02-26 at 09:39 +0100, Malthe Borch wrote:
> I have been seeing some checkins on Github to a `plone.api` package.
>
> Where's the PLIP for this? Zope tried to do the same very thing with
> "zapi" and it's been deprecated since Zope 3.5.

> I think there's a two camps in terms of where and how Plone's "app"
> code should live, i.e. fully split up into atoms of responsibility
> with the `plone.app.*` family (the current state of affairs; this is
> the majority), or as a single, or a couple of major eggs, e.g. `Plone`
> (wishful thinking; the minority).
>
> Where does this new API fit in? Arguably in the second camp, because
> it's got to have dependencies to a great many `plone.app.*` eggs
> and/or CMF products. If you update a dependency, you need to update
> the API. It's the "mother egg" and her dependencies are strong.

Products.CMFPlone currently depends on 103 packages, so much of the
heavy lifting of tracking dependencies and maintaining compatibility is
already being done there.

Would an API need to depend on packages outside of these 103 CMFPlone
dependencies?

> I understand the idea of approaching the problem of Plone's
> problematic codebase from the outside, taming the beast with clean,
> modernized wrappers, but it doesn't make much sense in terms of the
> dependency graph -and- it's a step away from eventually cleaning up
> and refactoring the codebase altogether:
>
> – get rid of the very deep class hierarchy of Plone-objects;

How would an API implemented by wrappers hinder efforts to get rid of
the deep class hierarchy?

For integrators who use the API, the changing of classes behind it would
be less disruptive, since the API stays constant.

> – lose code weight; currently it's likely an inverse 80/20 rule (20%
> code used by 80% installations).

AFAIU, the idea of a Plone API is to provide a consistent, pythonic way
of doing CRUD operations that integrators will do over and over again.

People are not going to stop changing workflow states or setting member
properties.

> – refactor and improve existing code so that it fits the bill as an "API".

A laudable goal, but very difficult (if not impossible) to achieve when
using disparate packages from hundreds of authors.

> Add to this that existing third party code will only very slowly or
> never migrate to the API.

This is true, but is it a showstopper?

Unmaintainted 3rd party code will in any case become unusable as Plone
evolves.

JC

ajung

unread,
Feb 26, 2012, 7:46:01 AM2/26/12
to plone-de...@lists.sourceforge.net
To make it short...my 2 cents on plone.api. I really love the idea from the
integrators point of view (although I know how to use the real API
methods)...I would love to see the same API being exposed as a web-service.
wsapi4plone is already doing a good job right now..having a consistent
programmatic API and a consistent web-service API together in one would be a
cool addition for integrators.

-aj


J-C Brand wrote

> Plone-developers@.sourceforge
> https://lists.sourceforge.net/lists/listinfo/plone-developers
>


--
View this message in context: http://plone.293351.n2.nabble.com/plone-api-back-to-the-future-tp7318823p7319176.html
Sent from the Core Developers mailing list archive at Nabble.com.

Martin Aspeli

unread,
Feb 26, 2012, 8:48:58 AM2/26/12
to ajung, plone-de...@lists.sourceforge.net

On 26 Feb 2012, at 12:46, ajung <li...@zopyx.com> wrote:

> To make it short...my 2 cents on plone.api. I really love the idea from the
> integrators point of view (although I know how to use the real API
> methods)...I would love to see the same API being exposed as a web-service.
> wsapi4plone is already doing a good job right now..having a consistent
> programmatic API and a consistent web-service API together in one would be a
> cool addition for integrators.
>

I like the idea too. It solves a big problem we have with API discoverability.

However, API design is hard and a bad API is worse than none at all in this case. I'd like to make sure we take documentation and review if this very seriously before any release. I'd like to volunteer to help with this type of review.

Martin

Domen Kožar

unread,
Feb 26, 2012, 8:58:03 AM2/26/12
to Martin Aspeli, plone-de...@lists.sourceforge.net, ajung
There are few people at the sprint working on design docs for review
and idea was discussed on the last few successful drinkups.

I guess code is there only for experimenting with underlying implementation.

Currently the main blocker is a clear vision of goals plone.api is trying achieve.

Laurence Rowe

unread,
Feb 26, 2012, 12:30:09 PM2/26/12
to Martin Aspeli, plone-de...@lists.sourceforge.net, ajung
On 26 February 2012 14:48, Martin Aspeli <opti...@gmail.com> wrote:
>
>
> On 26 Feb 2012, at 12:46, ajung <li...@zopyx.com> wrote:
>
>> To make it short...my 2 cents on plone.api. I really love the idea from the
>> integrators point of view (although I know how to use the real API
>> methods)...I would love to see the same API being exposed as a web-service.
>> wsapi4plone is already doing a good job right now..having a consistent
>> programmatic API and a consistent web-service API together in one would be a
>> cool addition for integrators.
>>
>
> I like the idea too. It solves a big problem we have with API discoverability.
>
> However, API design is hard and a bad API is worse than none at all in this case. I'd like to make sure we take documentation and review if this very seriously before any release. I'd like to volunteer to help with this type of review.

We already have the @@context_state and @@portal_state views from
plone.app.layout along with @@folderListing from
plone.app.contentlisting. Rather than create a whole new API, I think
it's important to build on these, perhaps adding further views along
similar lines, and making them more easily reachable from pure python
code. Being views makes them easy to use from page templates.

Also note that getSite() does not always return the portal object
itself. KSS views setup their own site manager and I expect subsite
type add-ons like lineage will do so in the future if we enable
traversal events in Zope2/4.

Laurence

ajung

unread,
Feb 26, 2012, 12:46:44 PM2/26/12
to plone-de...@lists.sourceforge.net
I think that the concept of a view must be kept under the hood. It is ok
re-using
existing views but they should be exposed through a simple API. Complexity
must be hidden. E.g. traversing a folder hierarchy should follow the
natural __getitem__() API, traversal should work similar to
restrictedTraverse()....setting/getting object schema attributes should
follow the __setattr__, __getattr__ API while implementation specific
differences (AT mutator/accessor methods vs. AT FieldProperties vs.
Dexterity) must be be hidden.

-aj


Laurence Rowe wrote
>
> On 26 February 2012 14:48, Martin Aspeli &lt;optilude@&gt; wrote:

> Plone-developers@.sourceforge
> https://lists.sourceforge.net/lists/listinfo/plone-developers
>


--
View this message in context: http://plone.293351.n2.nabble.com/plone-api-back-to-the-future-tp7318823p7319558.html


Sent from the Core Developers mailing list archive at Nabble.com.

------------------------------------------------------------------------------

Nejc Zupan

unread,
Feb 26, 2012, 10:02:21 PM2/26/12
to plone-de...@lists.sourceforge.net
Hi all,

as some of you have already noticed, work has begun on a new package: plone.api.

The main goal is, quoting it's documentation, "a simple entry point for 20% of tasks
we do 80% of the time."

At this moment we have defined which methods should be in plone.api, their parameters

We also have a narrative documentation that guides new Plone developers through
the process of managing content, users and groups:

Note that *no code* has been written so far. All focus went into getting consensus on
what to include and how the API should look like. In this light:

Please don't go around adding/fixing API without first getting a few "+1"s from
*non-rockstar* developers. It is absolutely crucial that whatever gets into plone.api
to be understandable without years of Plone experience. I'm sure we missed some
common use-cases, let's get them out there in the open to discuss whether to include
them in plone.api.

Actual method implementations are very welcome, but we'd like the API to stay mostly
frozen unless the majority agrees on changing it. Baby steps. Use branches and
pull-requests, keep master clean. 


On behalf of Plone Konferenz sprinters,
Nejc

David Glick

unread,
Feb 27, 2012, 1:36:03 AM2/27/12
to Laurence Rowe, plone-de...@lists.sourceforge.net, ajung, Martin Aspeli
On 2/26/12 9:30 AM, Laurence Rowe wrote:
> On 26 February 2012 14:48, Martin Aspeli<opti...@gmail.com> wrote:
>>
>> On 26 Feb 2012, at 12:46, ajung<li...@zopyx.com> wrote:
>>
>>> To make it short...my 2 cents on plone.api. I really love the idea from the
>>> integrators point of view (although I know how to use the real API
>>> methods)...I would love to see the same API being exposed as a web-service.
>>> wsapi4plone is already doing a good job right now..having a consistent
>>> programmatic API and a consistent web-service API together in one would be a
>>> cool addition for integrators.
>>>
>> I like the idea too. It solves a big problem we have with API discoverability.
>>
>> However, API design is hard and a bad API is worse than none at all in this case. I'd like to make sure we take documentation and review if this very seriously before any release. I'd like to volunteer to help with this type of review.
> We already have the @@context_state and @@portal_state views from
> plone.app.layout along with @@folderListing from
> plone.app.contentlisting. Rather than create a whole new API, I think
> it's important to build on these, perhaps adding further views along
> similar lines, and making them more easily reachable from pure python
> code. Being views makes them easy to use from page templates.
>
> Also note that getSite() does not always return the portal object
> itself. KSS views setup their own site manager and I expect subsite
> type add-ons like lineage will do so in the future if we enable
> traversal events in Zope2/4.
>
Yes, I suggested using getUtility(ISiteRoot) instead.
David


----------
David Glick
Web Developer
david...@groundwire.org
206.286.1235x32

The Engagement Party 2012. So much more fun than the wedding reception.

http://www.npoengagementparty.com

------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2

David Glick

unread,
Feb 27, 2012, 1:37:57 AM2/27/12
to Nejc Zupan, plone-de...@lists.sourceforge.net

Nejc,
Thanks for the overview and congrats to the sprinters; it looks like
this is off to a great start! It remains to be seen whether it will be
successful in the way zapi was not for Zope, but I am encouraged by the
approach of documentation first and preferring input from non-rockstars.
So kudos!

Some suggestions:

1. Regarding the calls like get_site_url() and get_request(), it's a bit
misleading to say that these will *always* be available. It would be
good to warn that the API is intended for use within code published by
the ZPublisher (which sets up the necessary thread locals), as opposed
to external methods or code run via bin/instance run or whatever.
Uh...I'll let you guys translate that into non-Zope-addled-brain
language. :)

2. In Dexterity we have plone.dexterity.utils.createContentInContainer
which is similar to plone.api.content.create(). One thing I've found
very useful in the Dexterity version is the checkConstraints=False flag
if I want to add a content item and ignore the normal security checks
for whether the item is allowed (useful for example in a browser view
intended to let anonymous users submit something). Can we add something
like this to api.content.create()? If not, please at least clarify in
the documentation which API calls check permissions and which don't;
this info is currently omitted.

3. This is me channeling wiggy: For the user API, can we please use
"user_id" as the argument name instead of "username"? "Username" and
"user id" are slightly different concepts in PAS and it gets endlessly
confusing when the terminology changes from place to place. (We should
probably supporting getting a user either by user_id or login.)

4. The documentation for get_tool should include a list of the tools
that are available in a standard Plone installation.

5. Some additional things I think it would be useful to include in the
API (or at least brainstorm further). (I came up with this list by going
through the .py files of a recent client project and looking for things
that could let me remove an import statement if it were part of the
api....perhaps this is a useful exercise for others to do too.) Calls to:
a. get a content item's UID, and to get a content item from a UID
b. list the names of a content item's schema fields (no matter if it
is Archetypes or Dexterity) and get/set a field value by name
c. grant or revoke a permission to a role within a particular context
d. get the navigation root (which may be different than the site root)
e. get/set plone.app.registry record values
f. format a date/time or number for the current locale
g. make sure a given path exists (i.e. add several levels of folder
structure with one call)
h. activate, upgrade, or deactivate an add-on
i. As Laurence and Andreas mentioned, the things currently included
in the views in plone.app.layout.globals...this also requires some
discussion of what parts of the API to make accessible to untrusted code
(i.e. page templates and TALES expressions).
j. An is_homepage() check (this is a FAQ, how do I check if I'm on
the homepage)

One final word -- I hope others will see the new plone.api project not
as an excuse to never simplify the APIs of the existing underlying Zope
and Plone packages, but rather as an inspiration to do more of that. :)

cheers,
David

----------
David Glick
Web Developer
david...@groundwire.org
206.286.1235x32

The Engagement Party 2012. So much more fun than the wedding reception.

http://www.npoengagementparty.com

------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2

matthew lange

unread,
Feb 27, 2012, 1:37:59 AM2/27/12
to Nejc Zupan, plone-de...@lists.sourceforge.net
Oh my gooodness, thank you so much...this looks awesome.
Only writing python code <5% of my time, and an even smaller % for other plone bits--I am so happy to see this initiative.  

I often find myself knowing what needs to be done, but hating that I'd forgotten how to do it--or that the methods had changed, etc.--and then of course, hating writing code even more.

A few things I didn't see this in the links provided that I do think it would be good to be able to do:

list the members of a group/role (based on credentials of course)...
a simple find/union/intersect functionality--similar to the get function listed:

SomeContentCol = api.content.find(path='/somePath', type='someContentType', attributeName='someAtributeName', attributeValue='someAtributeValue')

SomeOtherContentCol = api.content.find(path='/posSomeOtherPath', type=' posSomeOtherContentType', attributeName=' posSomeOtherAtributeName', attributeValue='posSomeOtheAtributeValue') 

IntersectedContent = api.content.intersect (SomeContentCol, SomeOtherContentCol, attrNamesValuesToMatch, attrNamesValuesToFilter)
UnionedContent = api.content.union (SomeContentCol, SomeOtherContentCol, attrNamesValuesToMatch, attrNamesValuesToFilter)

Above is simplified of course, the attribute name/values would have to be arrayable...but this kind of easily readable/teachable interface for content interrogation would really go a long way toward getting me--and perhaps many others--off of SQL RDBMS...which is still my biggest holdback for total embracement of Plone.

Great work.
~mc

> ------------------------------------------------------------------------------
> Try before you buy = See our experts in action!
> The most comprehensive online learning library for Microsoft developers
> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
> Metro Style Apps, more. Free future releases when you subscribe now!
> http://p.sf.net/sfu/learndevnow-dev2

Martin Aspeli

unread,
Feb 27, 2012, 2:49:14 AM2/27/12
to David Glick, plone-de...@lists.sourceforge.net, ajung

... which works, except I believe it doesn't get the RequestContainer
fake aq parent, so anything doing context.REQUEST with that (or a
traversed-to child of it) as the context won't work.

Martin

David Glick

unread,
Feb 27, 2012, 2:55:10 AM2/27/12
to Martin Aspeli, plone-de...@lists.sourceforge.net, ajung
I think it says something that I then have no idea what the right way is
to get the site root 100% of the time :-/
David


----------
David Glick
Web Developer
david...@groundwire.org
206.286.1235x32

The Engagement Party 2012. So much more fun than the wedding reception.

http://www.npoengagementparty.com

------------------------------------------------------------------------------

Martin Aspeli

unread,
Feb 27, 2012, 5:24:34 AM2/27/12
to Nejc Zupan, plone-de...@lists.sourceforge.net
On 27 February 2012 03:02, Nejc Zupan <nejc....@gmail.com> wrote:
> Hi all,
>
> as some of you have already noticed, work has begun on a new package:
> plone.api.

I think this is really great :)

There is a risk that we'll end up in a situation where all of our own
packages depend on plone.api (I think this was Malthe's concern),
creating a somewhat challenging dependency management situation. We
need to think about how we address that, perhaps by mandating that
Plone core packages can't use plone.api, i.e. it's a "consumer-facing"
package only. Another option would be to make plone.api have 'soft'
dependencies only i.e. the API functions are available if the relevant
packages are in sys.path, otherwise they give an error, but there is
no install_requires dependency for things like Products.CMFPlone.

> The main goal is, quoting it's documentation, "a simple entry point for 20%
> of tasks
> we do 80% of the time."
>
> At this moment we have defined which methods should be in plone.api, their
> parameters
> and their return
> values: http://readthedocs.org/docs/ploneapi/en/latest/api.html

A bunch of thoughts follow:

First off - these methods use lowercase_and_underscore() methods. That
is what PEP8 (now!) says, but it's not the Zope standard. Since you
can't replicate every function on every object, you can't hide all the
mixedCase() methods. Dexterity started out as the former and had a
painful transition to the latter as API code became really confusing.
I think we just need to accept mixedCase() and stick with it.

get_tool() should be called getToolByName(). That one is already an
API and having two things importable from two places that do the same
thing with similar-but-different names is confusing. This is mainly
about old documentation and code that we can't expect to update in one
fell swoop.

get_all() for users is likely to be impossible in many scenarios (e.g.
LDAP/AD) and very, very slow. Probably best not to expose this as an
API. A search-based API is better.

We should consider the API on objects returned by items as well. At
the very least, dict-like access to folderish items (which works, even
for AT content), but also common methods like getId().

I was hoping to see some content search/catalog APIs.

There is no API for for developing new functionality, e.g. registering
new views, viewlets etc. I'd suggest trying to incorporate those into
plone.api may be a fallacy, as they are well-established elsewhere.
However, the documentation should cover where these are and give some
examples of important constructs. At the very least, I'd cover views.

We should either subsume or extend the plone_portal_state and
plone_context_state views for some of this. Access from TAL (including
expressions on things like actions) will be important. They also allow
for per-request caching which is important for performance. A simple
way to do that may be to replicate the relevant APIs on those views
and have the module-global function look up the view and call the
method.

> We also have a narrative documentation that guides new Plone developers
> through
> the process of managing content, users and groups:
> http://readthedocs.org/docs/ploneapi/en/latest/index.html#narrative-documentation

This is fantastic. :-)

> Note that *no code* has been written so far. All focus went into getting
> consensus on
> what to include and how the API should look like. In this light:
>
> Please don't go around adding/fixing API without first getting a few
> "+1"s from
> *non-rockstar* developers. It is absolutely crucial that whatever gets into
> plone.api
> to be understandable without years of Plone experience. I'm sure we missed
> some
> common use-cases, let's get them out there in the open to discuss whether to
> include
> them in plone.api.

I'll have a look through chapter 9 of PP4D and pull out anything I
felt was important enough to include there that may be missing.

We should also look at the Dexterity Developer Manual, which intends
to expose useful content operations, even if not exactly Dexterity
specific.

David also added a bunch of potentially useful functions.

> Actual method implementations are very welcome, but we'd like the API to
> stay mostly
> frozen unless the majority agrees on changing it. Baby steps. Use branches
> and
> pull-requests, keep master clean.

Good ideas. I think writing the implementations will be relatively easy.

Martin

------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2

Martin Aspeli

unread,
Feb 27, 2012, 5:26:11 AM2/27/12
to David Glick, plone-de...@lists.sourceforge.net, ajung
On 27 February 2012 07:55, David Glick <david...@groundwire.org> wrote:

> I think it says something that I then have no idea what the right way is to
> get the site root 100% of the time :-/

I don't think it's 100% possible every scenario if you don't have a
valid acquisition context. If you do, use the potral_url tool.

The fix is simple, though: a traversal hook like the one that sets the
'site' threadlocal that getSite() returns that only triggers on the
site root and not other component site managers.

Martin

Malthe Borch

unread,
Feb 27, 2012, 5:38:17 AM2/27/12
to Martin Aspeli, plone-de...@lists.sourceforge.net
On 27 February 2012 11:24, Martin Aspeli <optilud...@gmail.com> wrote:
> There is a risk that we'll end up in a situation where all of our own
> packages depend on plone.api (I think this was Malthe's concern),
> creating a somewhat challenging dependency management situation. We
> need to think about how we address that, perhaps by mandating that
> Plone core packages can't use plone.api, i.e. it's a "consumer-facing"
> package only. Another option would be to make plone.api have 'soft'
> dependencies only i.e. the API functions are available if the relevant
> packages are in sys.path, otherwise they give an error, but there is
> no install_requires dependency for things like Products.CMFPlone.

I don't think it's a worry for `plone.*` packages, because they
typically interact with packages that do have reasonable APIs (e.g.
plone.session, keyring and friends).

The `plone.app.*` packages – these would probably benefit from using
the standard API. Actually, they'd benefit from not being separate
packages at all:

plone/app/layout
plone/app/locales
... etc ...

Then, you'd simply have top-level imports:

from plone import get_members_by_id

That's *simple* – and I believe it's also the intention with the project.

It's unnessary to use the `plone.api` namespace, because this will be
just functions, and a name requirement could be that the published
names be sufficiently long and descriptive, and contain at least one
underscore (as illustrated by the example).

\malthe

Nejc Zupan

unread,
Feb 27, 2012, 8:11:34 AM2/27/12
to David Glick, plone-de...@lists.sourceforge.net
On 27/02/2012, at 07:37, David Glick wrote:

Some suggestions:

1. Regarding the calls like get_site_url() and get_request(), it's a bit misleading to say that these will *always* be available. It would be good to warn that the API is intended for use within code published by the ZPublisher (which sets up the necessary thread locals), as opposed to external methods or code run via bin/instance run or whatever.  Uh...I'll let you guys translate that into non-Zope-addled-brain language. :)

Definitely. If someone can give me a definitive list of when get_request() and such won't work,
and possibly links to in-depth explanations why, I'm happy to add this to the API documentation
(as in plone.api limitations). See https://github.com/plone/plone.api/issues/1.


2. In Dexterity we have plone.dexterity.utils.createContentInContainer which is similar to plone.api.content.create(). One thing I've found very useful in the Dexterity version is the checkConstraints=False flag if I want to add a content item and ignore the normal security checks for whether the item is allowed (useful for example in a browser view intended to let anonymous users submit something). Can we add something like this to api.content.create()?

I like the idea, I'll put it up for discussion. See https://github.com/plone/plone.api/issues/2


If not, please at least clarify in the documentation which API calls check permissions and which don't; this info is currently omitted.

I'll need help here, someone going through the methods one by one and writing down what the
security implications are. See https://github.com/plone/plone.api/issues/3.


3. This is me channeling wiggy: For the user API, can we please use "user_id" as the argument name instead of "username"? "Username" and "user id" are slightly different concepts in PAS and it gets endlessly confusing when the terminology changes from place to place. (We should probably supporting getting a user either by user_id or login.)

What the underlying terminology uses is not that relevant. The main question is How should the
API look like. In an ideal world, disregarding what goes on underneath, do we want the
api.user.create() to have:
 - ``username`` and ``user`` parameters OR
 -  ``userid`` and ``user`` parameters

I vote the former. What do you think?


4. The documentation for get_tool should include a list of the tools that are available in a standard Plone installation.



5. Some additional things I think it would be useful to include in the API (or at least brainstorm further). (I came up with this list by going through the .py files of a recent client project and looking for things that could let me remove an import statement if it were part of the api....perhaps this is a useful exercise for others to do too.)

Yes, this is exactly how we did it :).


Calls to:
 a. get a content item's UID

Should this just be documented or do we need an actual api helper method. Isn't is just content.UID()?


and to get a content item from a UID



 b. list the names of a content item's schema fields (no matter if it is Archetypes or Dexterity)

Yes! And also list user properties. See https://github.com/plone/plone.api/issues/5.


and get/set a field value by name

Also good. We already have api.user.get_property and api.user.set_property, so having the same for
content object is expected. See https://github.com/plone/plone.api/issues/6.


 c. grant or revoke a permission to a role within a particular context

Cannot this be easily performed through UI. Does it fall in the 20% of most common tasks?


 d. get the navigation root (which may be different than the site root)

If you have a site where navigation root is different than site root, you are doing something funky
and should know what you are doing without an abstraction layer covering it up for you.
A use-case that plone.api does not want to cover.


 e. get/set plone.app.registry record values



 f. format a date/time or number for the current locale



 g. make sure a given path exists (i.e. add several levels of folder structure with one call)

Shall this be a separate method or a flag in create/move/copy?


 h. activate, upgrade, or deactivate an add-on

Uhm ... again, is it in the 20%? Isn't the UI approach good enogh and we document that 
rather than adding another method to plone.api (I really want to keep it sleek).


 i. As Laurence and Andreas mentioned, the things currently included in the views in plone.app.layout.globals...this also requires some discussion of what parts of the API to make accessible to untrusted code (i.e. page templates and TALES expressions).

If something in there proves usefull, we can easily add a method to plone.api that gets the relevant view
and then the relevant method. Is there something in globals that is used all the time and we don't yet
have in plone.api?


 j. An is_homepage() check (this is a FAQ, how do I check if I'm on the homepage)


One final word -- I hope others will see the new plone.api project not as an excuse to never simplify the APIs of the existing underlying Zope and Plone packages, but rather as an inspiration to do more of that. :)

On the contrary, plone.api will clearly show which parts of Plone APIs suck the most and need to get fixed.


z.

Nejc Zupan

unread,
Feb 27, 2012, 8:16:57 AM2/27/12
to matthew lange, plone-de...@lists.sourceforge.net
On 27/02/2012, at 07:37, matthew lange wrote:

A few things I didn't see this in the links provided that I do think it would be good to be able to do:

list the members of a group/role (based on credentials of course)...

Yep, missed that one, please see https://github.com/plone/plone.api/issues/10.


a simple find/union/intersect functionality--similar to the get function listed:

SomeContentCol = api.content.find(path='/somePath', type='someContentType', attributeName='someAtributeName', attributeValue='someAtributeValue')

SomeOtherContentCol = api.content.find(path='/posSomeOtherPath', type=' posSomeOtherContentType', attributeName=' posSomeOtherAtributeName', attributeValue='posSomeOtheAtributeValue') 

IntersectedContent = api.content.intersect (SomeContentCol, SomeOtherContentCol, attrNamesValuesToMatch, attrNamesValuesToFilter)
UnionedContent = api.content.union (SomeContentCol, SomeOtherContentCol, attrNamesValuesToMatch, attrNamesValuesToFilter)

If you can further elaborate on how these should work, what parameters do you expect it to
have, etc. and write it up as an issue on GitHub so we can all comment on it there? Including examples
of how you would it in your code?


z.

Nejc Zupan

unread,
Feb 27, 2012, 8:17:57 AM2/27/12
to David Glick, plone-de...@lists.sourceforge.net, ajung, Martin Aspeli
On 27/02/2012, at 08:55, David Glick wrote:
> I think it says something that I then have no idea what the right way is
> to get the site root 100% of the time :-/

:)

Yuri

unread,
Feb 27, 2012, 8:20:50 AM2/27/12
to plone-de...@lists.sourceforge.net
Shouldn't the software adhere to the apis, instead of the apis adhere to
the software? :)

Nejc Zupan

unread,
Feb 27, 2012, 8:31:26 AM2/27/12
to Martin Aspeli, plone-de...@lists.sourceforge.net
On 27/02/2012, at 11:24, Martin Aspeli wrote:

On 27 February 2012 03:02, Nejc Zupan <nejc....@gmail.com> wrote:
Hi all,

as some of you have already noticed, work has begun on a new package:
plone.api.

I think this is really great :)

Thanks!

There is a risk that we'll end up in a situation where all of our own
packages depend on plone.api (I think this was Malthe's concern),
creating a somewhat challenging dependency management situation. We
need to think about how we address that, perhaps by mandating that
Plone core packages can't use plone.api, i.e. it's a "consumer-facing"
package only.

Absolutely. This package is meant for consumers, for integrators.
If you are a core developer, than you better know your shit and use the
underlying APIs and not the plone.api abstraction layer. I strongly discourage
any core code using plone.api because it's goal is to cover 20% of use
cases and core code needs to cover 100% of use cases (or close).


Another option would be to make plone.api have 'soft'
dependencies only i.e. the API functions are available if the relevant
packages are in sys.path, otherwise they give an error, but there is
no install_requires dependency for things like Products.CMFPlone.

We need Products.CMFPlone as an install_requires so that we get
an auto-generated API from method docstrings. Sphinx needs to import
all code in order to generate this: http://readthedocs.org/docs/ploneapi/en/latest/api.html


First off - these methods use lowercase_and_underscore() methods. That
is what PEP8 (now!) says, but it's not the Zope standard. Since you
can't replicate every function on every object, you can't hide all the
mixedCase() methods. Dexterity started out as the former and had a
painful transition to the latter as API code became really confusing.
I think we just need to accept mixedCase() and stick with it.

-1, but mostly on a personal preference level


get_tool() should be called getToolByName(). That one is already an
API and having two things importable from two places that do the same
thing with similar-but-different names is confusing. This is mainly
about old documentation and code that we can't expect to update in one
fell swoop.

-1
Old documentation and code will still work and can be kept that way.
For completely new developers (I'm hauling 10 new students into Plone 
this July!) it's just so much easier to remember get_tool(), especially
if it lists all possible tools in it's docs.


get_all() for users is likely to be impossible in many scenarios (e.g.
LDAP/AD) and very, very slow. Probably best not to expose this as an
API.

-1
We want to cover 20% of usecases. If you have AD with thousands of 
users, you better know your shit. Though we definitely should state it docs
the limitations of this method (as in, it will be slow if you have a lot of users)

A search-based API is better.

And can be similarly abused: api.user.search("*"), not solving anything.


We should consider the API on objects returned by items as well. At
the very least, dict-like access to folderish items (which works, even
for AT content), but also common methods like getId().

The dict-like access to folderish items is already documented.

And also why we don't want to use wrappers on returned objects:


I was hoping to see some content search/catalog APIs.

Definitely on the list of ideas, but we couldn't reach a consensus of how it should
behave. If we come up with a proposal that we mostly agree on, it goes in.


There is no API for for developing new functionality, e.g. registering
new views, viewlets etc. I'd suggest trying to incorporate those into
plone.api may be a fallacy, as they are well-established elsewhere.
However, the documentation should cover where these are and give some
examples of important constructs. At the very least, I'd cover views.

Again, there were a lot of discussions, but no consensus reached. Proposals
need further work, so we can all agree on them. 


We should either subsume or extend the plone_portal_state and
plone_context_state views for some of this. Access from TAL (including
expressions on things like actions) will be important. They also allow
for per-request caching which is important for performance. A simple
way to do that may be to replicate the relevant APIs on those views
and have the module-global function look up the view and call the
method.

I'm sorry but I cannot wrap my brain around this last paragraph. Can you
re-phrase please? 


Note that *no code* has been written so far. All focus went into getting
consensus on
what to include and how the API should look like. In this light:

Please don't go around adding/fixing API without first getting a few
"+1"s from
*non-rockstar* developers. It is absolutely crucial that whatever gets into
plone.api
to be understandable without years of Plone experience. I'm sure we missed
some
common use-cases, let's get them out there in the open to discuss whether to
include
them in plone.api.

I'll have a look through chapter 9 of PP4D and pull out anything I
felt was important enough to include there that may be missing.

Nice! Please add proposals as issues on GitHub.


We should also look at the Dexterity Developer Manual, which intends
to expose useful content operations, even if not exactly Dexterity
specific.

Same here, issues on GitHub.


Actual method implementations are very welcome, but we'd like the API to
stay mostly
frozen unless the majority agrees on changing it. Baby steps. Use branches
and
pull-requests, keep master clean.

Good ideas. I think writing the implementations will be relatively easy.

Yep, I agree. 


z.

Nejc Zupan

unread,
Feb 27, 2012, 8:33:16 AM2/27/12
to Yuri, plone-de...@lists.sourceforge.net
On 27/02/2012, at 14:20, Yuri wrote:

> Shouldn't the software adhere to the apis, instead of the apis adhere to
> the software? :)

Yes, that is the idea.


z.

Martin Aspeli

unread,
Feb 27, 2012, 8:50:15 AM2/27/12
to Nejc Zupan, plone-de...@lists.sourceforge.net
Hi,

On 27 February 2012 13:31, Nejc Zupan <nejc....@gmail.com> wrote:

> Another option would be to make plone.api have 'soft'
> dependencies only i.e. the API functions are available if the relevant
> packages are in sys.path, otherwise they give an error, but there is
> no install_requires dependency for things like Products.CMFPlone.
>
>
> We need Products.CMFPlone as an install_requires so that we get
> an auto-generated API from method docstrings. Sphinx needs to import
> all code in order to generate
> this: http://readthedocs.org/docs/ploneapi/en/latest/api.html

You can do that with an 'extra' for the purposes of generating the docs.

> First off - these methods use lowercase_and_underscore() methods. That
> is what PEP8 (now!) says, but it's not the Zope standard. Since you
> can't replicate every function on every object, you can't hide all the
> mixedCase() methods. Dexterity started out as the former and had a
> painful transition to the latter as API code became really confusing.
> I think we just need to accept mixedCase() and stick with it.
>
>
> -1, but mostly on a personal preference level

That's a bullshit reason. ;-)

Please don't make the same mistake that Dexterity did (and which was
painful to reverse). How on earth is a new developer supposed to
understand that imports from plone.api use one convention, but methods
on objects returned from its calls - and most other parts of Plone -
use another? PEP8 actually says existing convention takes priority
over fidelity to the PEP8 suggestions.

> get_tool() should be called getToolByName(). That one is already an
> API and having two things importable from two places that do the same
> thing with similar-but-different names is confusing. This is mainly
> about old documentation and code that we can't expect to update in one
> fell swoop.
>
>
> -1
> Old documentation and code will still work and can be kept that way.
> For completely new developers (I'm hauling 10 new students into Plone
> this July!) it's just so much easier to remember get_tool(), especially
> if it lists all possible tools in it's docs.

The point is that I may read some code like this:

portal_url = getToolByName(context, 'portal_url')

and then somewhere else, I may read:

portal_url = get_tool(context, 'portal_url')

The import statement is scrolled out of view, or not part of the
example. The overlap is confusing. And getToolByName is probably one
of the few 'API-like' functions we have today, an indirection instead
of doing aq_acqiure(context, 'portal_url') or whatever.

Your 10 new users will pretty soon move beyond the pristinely simple
API. We need to consider that they shouldn't drop off a cliff,
learning-curve-wise, when they do. We have a legacy codebase and some
kind of compromise in consideration of it will always be necessary.

> get_all() for users is likely to be impossible in many scenarios (e.g.
> LDAP/AD) and very, very slow. Probably best not to expose this as an
> API.
>
>
> -1
> We want to cover 20% of usecases. If you have AD with thousands of
> users, you better know your shit. Though we definitely should state it docs
> the limitations of this method (as in, it will be slow if you have a lot of
> users)

No, seriously. This is another one of those 'fall of a cliff' things.
As soon as your site scales, get_all() becomes one of those things
that makes it possible to DoS your site in seconds. People should not
be expected to know of this danger.

> A search-based API is better.
>
>
> And can be similarly abused: api.user.search("*"), not solving anything.

Yes, but here it is more obvious what you're doing. It's a question of
what you encourage people to do. With a get_all() people will call
that, loop over the results and apply their own filter. It can be
disastrous performance-wise, and will not work in all cases because
underlying user storage mechanisms don't allow it for the very same
reason.

If we're trying to make things simpler for people, we also need to
consider nudging them towards good implementation choices without them
having to be an expert on the various bottlenecks and problems in the
solution. Saying that the only way to get users in bulk is to search,
is on that scale.

> We should consider the API on objects returned by items as well. At
> the very least, dict-like access to folderish items (which works, even
> for AT content), but also common methods like getId().
>
>
> The dict-like access to folderish items is already documented.
> See http://readthedocs.org/docs/ploneapi/en/latest/content.html#get-content-example.

Great!

We also need to consider attribute/field value access, as David said.

> And also why we don't want to use wrappers on returned objects:
> http://readthedocs.org/docs/ploneapi/en/latest/rationale.html#faq

Completely agree. Wrappers are a huge headache.

> I was hoping to see some content search/catalog APIs.
>
>
> Definitely on the list of ideas, but we couldn't reach a consensus of how it
> should
> behave. If we come up with a proposal that we mostly agree on, it goes in.

I think the portal_catalog __call__() API is fine, but we should
shield people from having to get the portal_catalog tool (which has
all kinds of scary methods) just to get to the method. And we should
call it search(), not searchResults(), a retarded method name if ever
there was one. You pass a dict and get back a lazy list of brains.

We just need to document the common indexes and explain where in the
ZMI you look for other indexes. We also need to document brains and
metadata.

> There is no API for for developing new functionality, e.g. registering
> new views, viewlets etc. I'd suggest trying to incorporate those into
> plone.api may be a fallacy, as they are well-established elsewhere.
> However, the documentation should cover where these are and give some
> examples of important constructs. At the very least, I'd cover views.
>
>
> Again, there were a lot of discussions, but no consensus reached. Proposals
> need further work, so we can all agree on them.

Yep. Maybe best to keep separate for now anyway, as it's effectively a
different 'class' of API.

> We should either subsume or extend the plone_portal_state and
> plone_context_state views for some of this. Access from TAL (including
> expressions on things like actions) will be important. They also allow
> for per-request caching which is important for performance. A simple
> way to do that may be to replicate the relevant APIs on those views
> and have the module-global function look up the view and call the
> method.
>
>
> I'm sorry but I cannot wrap my brain around this last paragraph. Can you
> re-phrase please?

See Laurence's comment (in the other thread?), essentially. It needs
to be possible to call API methods from untrusted code/TAL/actions,
which means we want them to be accessible by traversing to a view. We
have some 'API' functions in plone.app.layout.globals, and we want to
make sure we have those in plone.api and have a way to get to the API
functions via a view lookup like
context/@@plone_portal_state/navigationRootUrl or whatever.

> Note that *no code* has been written so far. All focus went into getting

Martin

Martin Aspeli

unread,
Feb 27, 2012, 9:05:35 AM2/27/12
to Nejc Zupan, plone-de...@lists.sourceforge.net
On 27 February 2012 13:11, Nejc Zupan <nejc....@gmail.com> wrote:
> On 27/02/2012, at 07:37, David Glick wrote:
>
> Some suggestions:
>
> 1. Regarding the calls like get_site_url() and get_request(), it's a bit
> misleading to say that these will *always* be available. It would be good to
> warn that the API is intended for use within code published by the
> ZPublisher (which sets up the necessary thread locals), as opposed to
> external methods or code run via bin/instance run or whatever.  Uh...I'll
> let you guys translate that into non-Zope-addled-brain language. :)
>
>
> Definitely. If someone can give me a definitive list of when get_request()
> and such won't work,
> and possibly links to in-depth explanations why, I'm happy to add this to
> the API documentation
> (as in plone.api
> limitations). See https://github.com/plone/plone.api/issues/1.

The basic idea is, if your code is:

- within a view, viewlet or portlet
- in a page template (TAL)
- in a TTW "script (python)"
- in a GenericSetup setup handler
- in an External method (yes, David, they are published too...)
- in a method on a content item (most likely)
- ... anything else?

then your code is being executed as a result of the publisher finding
that resource based on the inbound path traversal and calling it. In
this case, you have traversed over the Plone site root at least, and
various threadlocals are set so that things like getRequest() from
zope.globalrequest, getSite() and context.REQUEST work. If you are
running a script through bin/instance debug or bin/instance run or in
a unit test, then no traversal has happened.

I think for most people, it's not going to be an issue except perhaps
in the unit test scenario.

> If not, please at least clarify in the documentation which API calls check
> permissions and which don't; this info is currently omitted.
>
>
> I'll need help here, someone going through the methods one by one and
> writing down what the
> security implications are. See https://github.com/plone/plone.api/issues/3.

The main thing to warn about are the methods that do explicit security
checks (or, we make the API not do that). They are:

- invokeFactory()
- portal_workflow.doActionFor()
- manage_objectPaste

and possibly some others.

> 3. This is me channeling wiggy: For the user API, can we please use
> "user_id" as the argument name instead of "username"? "Username" and "user
> id" are slightly different concepts in PAS and it gets endlessly confusing
> when the terminology changes from place to place. (We should probably
> supporting getting a user either by user_id or login.)
>
>
> What the underlying terminology uses is not that relevant. The main question
> is How should the
> API look like. In an ideal world, disregarding what goes on underneath, do
> we want the
> api.user.create() to have:
>  - ``username`` and ``user`` parameters OR
>  -  ``userid`` and ``user`` parameters
>
> I vote the former. What do you think?

The point is that userid and username are separate concepts. One is a
unique id that never changes. The other is a name you use to log in
with and which may change over time. Various APIs expect one or the
other. In many cases, userid and username are the same, but then
sometimes they're not and your code breaks, which can happen in config
e.g. when moving to an integrated environment. It's important to be
clear about the distinction.

For this reason, I wouldn't say 'user' because it's (a) ambiguous and
(b) could refer to a user *object* rather than the user name or id
(which are strings).

Again, I appreciate this is irritating complexity, but it's complexity
that people will hit at the wrong time, blindly, unless we (a) nudge
them towards using the right approach (b) are consistent in our use of
terms and (c) provide helpful documentation, including "common
gotchas".

> Calls to:
>  a. get a content item's UID
>
>
> Should this just be documented or do we need an actual api helper method.
> Isn't is just content.UID()?

No, it's

from plone.uuid.interfaces import IUUID
uuid = IUUID(context)

The former is AT-specific. A convenience method would be useful.

>  c. grant or revoke a permission to a role within a particular context
>
>
> Cannot this be easily performed through UI. Does it fall in the 20% of most
> common tasks?

It's a fairly common thing to want to do, especially in
bootstrapping/setup code and tests. And, the underlying API is awkward
to use.

>  d. get the navigation root (which may be different than the site root)
>
>
> If you have a site where navigation root is different than site root, you
> are doing something funky
> and should know what you are doing without an abstraction layer covering it
> up for you.
> A use-case that plone.api does not want to cover.

I'm not sure I agree, though I agree it's less common. I think it's
very trivial to add both, though. More importantly, I think we should
promote 'navigation root' as the primary concept. Most people care
about "the home page", i.e. what users get to when they click the site
logo or on a 'home' link. That's always the nav root, but only
sometimes the site root. The site root can be hidden from view with
virtual hosting (config, not code!). For things like catalog searches
with a path filter or URL construction, navigation root is a better
concept.

>  f. format a date/time or number for the current locale
>
>
> Also good, see https://github.com/plone/plone.api/issues/8.

Maybe also string localisation?

>  g. make sure a given path exists (i.e. add several levels of folder
> structure with one call)
>
>
> Shall this be a separate method or a flag in create/move/copy?

I think you want a method to test, even if you also have a policy flag
on those methods.

>  h. activate, upgrade, or deactivate an add-on
>
>
> Uhm ... again, is it in the 20%? Isn't the UI approach good enogh and we
> document that
> rather than adding another method to plone.api (I really want to keep it
> sleek).

I think this is mainly done with GenericSetup or UI, so I'd agree with
keeping this out for now.

>  i. As Laurence and Andreas mentioned, the things currently included in the
> views in plone.app.layout.globals...this also requires some discussion of
> what parts of the API to make accessible to untrusted code (i.e. page
> templates and TALES expressions).
>
>
> If something in there proves usefull, we can easily add a method to
> plone.api that gets the relevant view
> and then the relevant method. Is there something in globals that is used all
> the time and we don't yet
> have in plone.api?

The main thing is that you want the API to be available from TAL. I'd
also just do a bit of a cross-check between plone_portal_state and
plone_context_state and plone.api.

Finally, another source of API-ish calls:
http://pypi.python.org/pypi/plone.app.testing/4.0.2#common-test-patterns

Martin

Nejc Zupan

unread,
Feb 27, 2012, 9:07:51 AM2/27/12
to Martin Aspeli, plone-de...@lists.sourceforge.net
On 27/02/2012, at 14:50, Martin Aspeli wrote:

On 27 February 2012 13:31, Nejc Zupan <nejc....@gmail.com> wrote:

Another option would be to make plone.api have 'soft'
dependencies only i.e. the API functions are available if the relevant
packages are in sys.path, otherwise they give an error, but there is
no install_requires dependency for things like Products.CMFPlone.


We need Products.CMFPlone as an install_requires so that we get
an auto-generated API from method docstrings. Sphinx needs to import
all code in order to generate
this: http://readthedocs.org/docs/ploneapi/en/latest/api.html

You can do that with an 'extra' for the purposes of generating the docs.

Would this approach also work on ReadTheDocs.org? We're somewhat
limit there in terms of controlling the build. If it works, I'm definitely +1.


First off - these methods use lowercase_and_underscore() methods. That
is what PEP8 (now!) says, but it's not the Zope standard. Since you
can't replicate every function on every object, you can't hide all the
mixedCase() methods. Dexterity started out as the former and had a
painful transition to the latter as API code became really confusing.
I think we just need to accept mixedCase() and stick with it.


-1, but mostly on a personal preference level

That's a bullshit reason. ;-)

True :)


PEP8 actually says existing convention takes priority over fidelity to the PEP8 suggestions.

Yep, and it also says "although this is also an opportunity to clean up
 someone else's mess". 


get_tool() should be called getToolByName(). That one is already an
API and having two things importable from two places that do the same
thing with similar-but-different names is confusing. This is mainly
about old documentation and code that we can't expect to update in one
fell swoop.


-1
Old documentation and code will still work and can be kept that way.
For completely new developers (I'm hauling 10 new students into Plone
this July!) it's just so much easier to remember get_tool(), especially
if it lists all possible tools in it's docs.

The point is that I may read some code like this:

portal_url = getToolByName(context, 'portal_url')

and then somewhere else, I may read:

portal_url = get_tool(context, 'portal_url')

Nope, you read the following (omit context):
portal_url = get_tool(tool='portal_url')


The import statement is scrolled out of view, or not part of the
example.

It should be. We had an openspace here in Munich and *no one* could
spell out the import statement for getToolByName from the top of 
their head. No one. People have to look it up all the time (not everyone
is using mr.igor).


The overlap is confusing. And getToolByName is probably one
of the few 'API-like' functions we have today, an indirection instead
of doing aq_acqiure(context, 'portal_url') or whatever.

Doesn't mean that it cannot be further simplified.


Your 10 new users will pretty soon move beyond the pristinely simple
API. We need to consider that they shouldn't drop off a cliff,
learning-curve-wise, when they do. We have a legacy codebase and some
kind of compromise in consideration of it will always be necessary.

Yes, this is true. 


get_all() for users is likely to be impossible in many scenarios (e.g.
LDAP/AD) and very, very slow. Probably best not to expose this as an
API.


-1
We want to cover 20% of usecases. If you have AD with thousands of
users, you better know your shit. Though we definitely should state it docs
the limitations of this method (as in, it will be slow if you have a lot of
users)

No, seriously. This is another one of those 'fall of a cliff' things.
As soon as your site scales, get_all() becomes one of those things
that makes it possible to DoS your site in seconds. People should not
be expected to know of this danger.

A proposal: only allow get_all() if the users and groups control panel
allows listing users (in other words: "many users?" is un-checked)?


A search-based API is better.


And can be similarly abused: api.user.search("*"), not solving anything.

Yes, but here it is more obvious what you're doing. It's a question of
what you encourage people to do. With a get_all() people will call
that, loop over the results and apply their own filter. It can be
disastrous performance-wise, and will not work in all cases because
underlying user storage mechanisms don't allow it for the very same
reason.

We can still leave methods that have performance implications in plone.api
but have big red banners next to them explaining what can go wrong if 
they abuse them.


If we're trying to make things simpler for people, we also need to
consider nudging them towards good implementation choices without them
having to be an expert on the various bottlenecks and problems in the
solution. Saying that the only way to get users in bulk is to search,
is on that scale.

That is a fair point. Having search only, and then adding an example of
how to use search to get all users (with implications of what that means, 
performance wise).


We should consider the API on objects returned by items as well. At
the very least, dict-like access to folderish items (which works, even
for AT content), but also common methods like getId().


The dict-like access to folderish items is already documented.
See http://readthedocs.org/docs/ploneapi/en/latest/content.html#get-content-example.

Great!

We also need to consider attribute/field value access, as David said.



I was hoping to see some content search/catalog APIs.


Definitely on the list of ideas, but we couldn't reach a consensus of how it
should
behave. If we come up with a proposal that we mostly agree on, it goes in.

I think the portal_catalog __call__() API is fine, but we should
shield people from having to get the portal_catalog tool (which has
all kinds of scary methods) just to get to the method. And we should
call it search(), not searchResults(), a retarded method name if ever
there was one. You pass a dict and get back a lazy list of brains.

Good points.


We just need to document the common indexes and explain where in the
ZMI you look for other indexes. We also need to document brains and
metadata.

True.


There is no API for for developing new functionality, e.g. registering
new views, viewlets etc. I'd suggest trying to incorporate those into
plone.api may be a fallacy, as they are well-established elsewhere.
However, the documentation should cover where these are and give some
examples of important constructs. At the very least, I'd cover views.


Again, there were a lot of discussions, but no consensus reached. Proposals
need further work, so we can all agree on them.

Yep. Maybe best to keep separate for now anyway, as it's effectively a
different 'class' of API.

Indeed, yes.


We should either subsume or extend the plone_portal_state and
plone_context_state views for some of this. Access from TAL (including
expressions on things like actions) will be important. They also allow
for per-request caching which is important for performance. A simple
way to do that may be to replicate the relevant APIs on those views
and have the module-global function look up the view and call the
method.


I'm sorry but I cannot wrap my brain around this last paragraph. Can you
re-phrase please?

See Laurence's comment (in the other thread?), essentially. It needs
to be possible to call API methods from untrusted code/TAL/actions,
which means we want them to be accessible by traversing to a view. We
have some 'API' functions in plone.app.layout.globals, and we want to
make sure we have those in plone.api and have a way to get to the API
functions via a view lookup like
context/@@plone_portal_state/navigationRootUrl or whatever.

Ah, got it now, thanks!


z.

Martin Aspeli

unread,
Feb 27, 2012, 9:12:28 AM2/27/12
to Nejc Zupan, plone-de...@lists.sourceforge.net
Hi,

Your lack-of-chevron-quoting is making this hard to read, but...

On 27 February 2012 14:07, Nejc Zupan <nejc....@gmail.com> wrote:

> You can do that with an 'extra' for the purposes of generating the docs.
>
>
> Would this approach also work on ReadTheDocs.org? We're somewhat
> limit there in terms of controlling the build. If it works, I'm definitely
> +1.

Try it ;-)

> A proposal: only allow get_all() if the users and groups control panel
> allows listing users (in other words: "many users?" is un-checked)?

APIs that change based on UI config is probably not great. I think we
should just have a search API, make it clear how to 'get all' using
it, and put a big warning on it. This is a case where making it a bit
'hard' to perform an operation is good. It makes it clear this isn't
the way Plone was intended to be used.

> That is a fair point. Having search only, and then adding an example of
> how to use search to get all users (with implications of what that means,
> performance wise).

Yeah, this feels more right to me.

Nejc Zupan

unread,
Feb 27, 2012, 9:24:14 AM2/27/12
to Martin Aspeli, plone-de...@lists.sourceforge.net
'user' is always (in the context of plone.api) the object, same as 'group'
is always the object.
'username' is always a string, same as 'groupname' is always a string.


Let's pick good names and stick to them (in the context of plone.api).


Again, I appreciate this is irritating complexity, but it's complexity
that people will hit at the wrong time, blindly, unless we (a) nudge
them towards using the right approach (b) are consistent in our use of
terms and (c) provide helpful documentation, including "common
gotchas".

I agree. I want the plone.api narrative documentation to nudge people
in the right direction, always.


Calls to:
 a. get a content item's UID


Should this just be documented or do we need an actual api helper method.
Isn't is just content.UID()?

No, it's

from plone.uuid.interfaces import IUUID
uuid = IUUID(context)

The former is AT-specific. A convenience method would be useful.

Ah, I just learned something new. In this case I agree a convenience method


 c. grant or revoke a permission to a role within a particular context


Cannot this be easily performed through UI. Does it fall in the 20% of most
common tasks?

It's a fairly common thing to want to do, especially in
bootstrapping/setup code and tests. And, the underlying API is awkward
to use.



 d. get the navigation root (which may be different than the site root)


If you have a site where navigation root is different than site root, you
are doing something funky
and should know what you are doing without an abstraction layer covering it
up for you.
A use-case that plone.api does not want to cover.

I'm not sure I agree, though I agree it's less common. I think it's
very trivial to add both, though. More importantly, I think we should
promote 'navigation root' as the primary concept. Most people care
about "the home page", i.e. what users get to when they click the site
logo or on a 'home' link. That's always the nav root, but only
sometimes the site root. The site root can be hidden from view with
virtual hosting (config, not code!). For things like catalog searches
with a path filter or URL construction, navigation root is a better
concept.

So why just don't make api.get_site() return navigation root instead of site root?


 f. format a date/time or number for the current locale


Also good, see https://github.com/plone/plone.api/issues/8.

Maybe also string localisation?

Definitely, please add it to a comment on GitHub.


 g. make sure a given path exists (i.e. add several levels of folder
structure with one call)


Shall this be a separate method or a flag in create/move/copy?

I think you want a method to test, even if you also have a policy flag
on those methods.

So, if I understand correctly: have something like "check_if_exists(path)"
and a flag in create/move/copy? Seams reasonable to me.


 i. As Laurence and Andreas mentioned, the things currently included in the
views in plone.app.layout.globals...this also requires some discussion of
what parts of the API to make accessible to untrusted code (i.e. page
templates and TALES expressions).


If something in there proves usefull, we can easily add a method to
plone.api that gets the relevant view
and then the relevant method. Is there something in globals that is used all
the time and we don't yet
have in plone.api?

The main thing is that you want the API to be available from TAL. I'd
also just do a bit of a cross-check between plone_portal_state and
plone_context_state and plone.api.

Yep, I agree.
Tnx, it went completely under my radar.


z.

Nejc Zupan

unread,
Feb 27, 2012, 9:28:51 AM2/27/12
to Martin Aspeli, plone-de...@lists.sourceforge.net
On 27/02/2012, at 15:12, Martin Aspeli wrote:

> Your lack-of-chevron-quoting is making this hard to read, but...

URL with how to do that? I'm willing to adhere to quoting policies, I just need
to know what they are. Pointers to how to wrangle Mail.app around them also
welcome :).


> On 27 February 2012 14:07, Nejc Zupan <nejc....@gmail.com> wrote:
>
>> You can do that with an 'extra' for the purposes of generating the docs.
>>
>> Would this approach also work on ReadTheDocs.org? We're somewhat
>> limit there in terms of controlling the build. If it works, I'm definitely
>> +1.
>
> Try it ;-)

Heh, setup.py is still more or less black magic for me, but I will :).


>> A proposal: only allow get_all() if the users and groups control panel
>> allows listing users (in other words: "many users?" is un-checked)?
>
> APIs that change based on UI config is probably not great. I think we
> should just have a search API, make it clear how to 'get all' using
> it, and put a big warning on it. This is a case where making it a bit
> 'hard' to perform an operation is good. It makes it clear this isn't
> the way Plone was intended to be used.

+1, I'll write it up and RFC it.


z.

Martin Aspeli

unread,
Feb 27, 2012, 9:43:57 AM2/27/12
to Nejc Zupan, plone-de...@lists.sourceforge.net
On 27 February 2012 14:28, Nejc Zupan <nejc....@gmail.com> wrote:
> On 27/02/2012, at 15:12, Martin Aspeli wrote:
>
>> Your lack-of-chevron-quoting is making this hard to read, but...
>
> URL with how to do that? I'm willing to adhere to quoting policies, I just need
> to know what they are. Pointers to how to wrangle Mail.app around them also
> welcome :).

Not sure. Plain text email, maybe? I've seen it with others, too.

Guido Stevens

unread,
Feb 27, 2012, 11:15:33 AM2/27/12
to plone-de...@lists.sourceforge.net
On 02/27/2012 03:05 PM, Martin Aspeli wrote:
> On 27 February 2012 13:11, Nejc Zupan <nejc....@gmail.com> wrote:
>> d. get the navigation root (which may be different than the site root)
>>
>>
>> If you have a site where navigation root is different than site root, you
>> are doing something funky

Like running a multi-lingual site? I don't consider that to be an edge
case for Plone, certainly in Europe.

:*CU#
--
*** Guido A.J. Stevens *** tel: +31.43.3618933 ***
*** guido....@cosent.nl *** Postbus 619 ***
*** http://www.cosent.nl *** 6200 AP Maastricht ***

s h a r i n g m a k e s s e n s e

RT @StKonrath: Facebook spies on phone users' smartphone text
messages, report says http://t.co/yY1g5vNU

http://twitter.com/GuidoStevens

Nejc Zupan

unread,
Feb 27, 2012, 11:52:08 AM2/27/12
to Guido Stevens, plone-de...@lists.sourceforge.net
On 27/02/2012, at 17:15, Guido Stevens wrote:

> On 02/27/2012 03:05 PM, Martin Aspeli wrote:
>> On 27 February 2012 13:11, Nejc Zupan <nejc....@gmail.com> wrote:
>>> d. get the navigation root (which may be different than the site root)
>>>
>>>
>>> If you have a site where navigation root is different than site root, you
>>> are doing something funky
>
> Like running a multi-lingual site? I don't consider that to be an edge
> case for Plone, certainly in Europe.

Ah, yes, multi-lingual slipped under our radars too.


z.

Sean Upton

unread,
Feb 27, 2012, 12:06:36 PM2/27/12
to Nejc Zupan, plone-de...@lists.sourceforge.net
On Mon, Feb 27, 2012 at 6:11 AM, Nejc Zupan <nejc....@gmail.com> wrote:
> If you have a site where navigation root is different than site root, you
> are doing something funky
> and should know what you are doing without an abstraction layer covering it
> up for you.
> A use-case that plone.api does not want to cover.

Allow be to be a bit blunt: this scares me. This is not all that
strange. We cannot have a simplified API that pleases everybody, but
on the other side of the coin, we cannot just fiat these design
decisions at whim.

* A lot of work in various Plone 3 releases went into removing all of
the hacks that product-developers had to go through to use alternate
navigation roots. Much of Plone's internals for search, navigation,
etc use this idea.

* I maintain an add-on that uses this, quite a few others rely on this as well.

* IIRC, multi-lingual sites do (and will likely going forward) want to
take advantage of this.

Thinking aloud a bit more:

* Quoting Chris McDonough on the subject of API design:
"Unfortunately, when designing a framework, you get exactly one shot
at creating 'the right' API." We ought to ponder ways to route around
this axiom.

* We need to proceed with great care and deliberation, and getting the
(nominally official sounding) API right (and stable) should take a
while (and live in beta for some good amount of time).

* You should be able to use a newer plone.api with a slightly older
(supported) Plone version, and the decision of what API package
version to use is an add-on developer or integrator's decision to
override the defaults of the KGS. Some reasonable efforts towards
compatibility need to be made to allow developers to choose to use a
slightly newer (bleeding edge) API version in a recently released
Plone to allow for the API to evolve with a bit longer tether from the
core Plone version.

* Is the eventual intent for plone.api.* to be a namespace package?
Is this a good idea or a bad idea? I really think that certain area
experts or people working repeatedly on certain types of problems
would be best to guide and maintain APIs in parts, not as one
monolithic whole (for example, I have interest in user/member related
APIs as I keep building simplified adapters for this area on top of
PAS and the stock tools for my own use, and I would be interested in
collaborating with others interested in creating a better API for just
this problem only).

* If plone.api is a grassroots effort, I think the FWT needs to
provide (and solicit from developers here and on product-developers
list) steering, input, and design critique, review periodically (at
least a few rounds of review every few months before any API is ever
to be treated as initially stable)?

* There needs to be infrastructure for discussion and code-review
about each interfaces and each method, or some convention for using
our existing tools (github, trac, etc) to facilitate discussion about
the things in question (IMHO, mailing lists are too coarse-grained to
dabble in the minutiae of specific method signatures, etc). With the
right tools, the community can design an API by merit, not by
happenstance.


Sean


[1] http://plope.com/pyramid_auth_design_api_postmortem

Jan-Carel Brand

unread,
Feb 27, 2012, 11:49:23 AM2/27/12
to Martin Aspeli, plone-de...@lists.sourceforge.net
On Mon, 2012-02-27 at 13:50 +0000, Martin Aspeli wrote:
<snip>

> > First off - these methods use lowercase_and_underscore() methods. That
> > is what PEP8 (now!) says, but it's not the Zope standard. Since you
> > can't replicate every function on every object, you can't hide all the
> > mixedCase() methods. Dexterity started out as the former and had a
> > painful transition to the latter as API code became really confusing.
> > I think we just need to accept mixedCase() and stick with it.
> >
> >
> > -1, but mostly on a personal preference level
>
> That's a bullshit reason. ;-)
>
> Please don't make the same mistake that Dexterity did (and which was
> painful to reverse). How on earth is a new developer supposed to
> understand that imports from plone.api use one convention, but methods
> on objects returned from its calls - and most other parts of Plone -
> use another? PEP8 actually says existing convention takes priority
> over fidelity to the PEP8 suggestions.

Can you please elaborate on how and when you recognised that adhering to
PEP8 for Dexterity was a mistake and what the consequences were?

Thanks
J-C

Sean Upton

unread,
Feb 27, 2012, 12:30:49 PM2/27/12
to Jan-Carel Brand, plone-de...@lists.sourceforge.net
On Mon, Feb 27, 2012 at 9:49 AM, Jan-Carel Brand <li...@opkode.com> wrote:
> On Mon, 2012-02-27 at 13:50 +0000, Martin Aspeli wrote:
> <snip>
>
>> > First off - these methods use lowercase_and_underscore() methods. That
>> > is what PEP8 (now!) says, but it's not the Zope standard. Since you
>> > can't replicate every function on every object, you can't hide all the
>> > mixedCase() methods. Dexterity started out as the former and had a
>> > painful transition to the latter as API code became really confusing.
>> > I think we just need to accept mixedCase() and stick with it.
>> >
>> >
>> > -1, but mostly on a personal preference level
>>
>> That's a bullshit reason. ;-)
>>
>> Please don't make the same mistake that Dexterity did (and which was
>> painful to reverse). How on earth is a new developer supposed to
>> understand that imports from plone.api use one convention, but methods
>> on objects returned from its calls - and most other parts of Plone -
>> use another? PEP8 actually says existing convention takes priority
>> over fidelity to the PEP8 suggestions.
>
> Can you please elaborate on how and when you recognised that adhering to
> PEP8 for Dexterity was a mistake and what the consequences were?

I would be interested in understanding the impacts of this as well. I
like consistency, but I also prefer and use LCUS style for method and
identifier names exclusively in my own code. I see the merits to
both, but I may be blind to the downsides.

The "cultural" argument:

There is an indirect "feeling" about the tenor and personality of an
API -- I cannot explain why, but from an "API" standpoint, LCUS
(preferably with as few underscores) makes for APIs that look more
friendly and less daunting. Dunno, maybe it just looks less like
Java; maybe it just seems more humble. Maybe it forces you to write
more resource-centric code to avoid ugly underscores (e.g. make
abstractions look like mappings instead of controllers of some other
resource's state -- see the way PAS deals with groups for an example).
As ugly as underscore word delimiters are, they are less ugly than
mixed-case?

I would guess that there are cultural pre-dispositions among the
target audience of a simplified API (including Python devs outside of
Plone/Zope pedigree) that would find LCUS identifier style more
natural to find friendly.

Sean

Fulvio Casali

unread,
Feb 27, 2012, 2:54:15 PM2/27/12
to Sean Upton, plone-de...@lists.sourceforge.net
On Mon, Feb 27, 2012 at 6:30 PM, Sean Upton <sdu...@gmail.com> wrote:
>
>
>
> I would be interested in understanding the impacts of this as well. I
> like consistency, but I also prefer and use LCUS style for method and
> identifier names exclusively in my own code. I see the merits to
> both, but I may be blind to the downsides.
>

LCUS = ?

Google is failing me here, sorry.

Thanks,
fulv

Martin Aspeli

unread,
Feb 27, 2012, 3:03:52 PM2/27/12
to Jan-Carel Brand, plone-de...@lists.sourceforge.net

On 27 Feb 2012, at 16:49, Jan-Carel Brand <li...@opkode.com> wrote:

> On Mon, 2012-02-27 at 13:50 +0000, Martin Aspeli wrote:
> <snip>
>
>>> First off - these methods use lowercase_and_underscore() methods. That
>>> is what PEP8 (now!) says, but it's not the Zope standard. Since you
>>> can't replicate every function on every object, you can't hide all the
>>> mixedCase() methods. Dexterity started out as the former and had a
>>> painful transition to the latter as API code became really confusing.
>>> I think we just need to accept mixedCase() and stick with it.
>>>
>>>
>>> -1, but mostly on a personal preference level
>>
>> That's a bullshit reason. ;-)
>>
>> Please don't make the same mistake that Dexterity did (and which was
>> painful to reverse). How on earth is a new developer supposed to
>> understand that imports from plone.api use one convention, but methods
>> on objects returned from its calls - and most other parts of Plone -
>> use another? PEP8 actually says existing convention takes priority
>> over fidelity to the PEP8 suggestions.
>
> Can you please elaborate on how and when you recognised that adhering to
> PEP8 for Dexterity was a mistake and what the consequences were?

Not PEP8, just the lowercase/underscore convention for functions. It's a mistake in a Zope context because the Zope standard is mixedCase (it predates PEP8 making a recommendation on this) and this leads to confused code.

I feel quite strongly about this. Consistency is important. See above for why.

We discovered prior to 1.0 betas because all the docs were confused and code using Dexterity was a hodge podge of the two styles. Fixing it (and breaking early adopter code) was a blocker for making Dexterity more public. Had we had more than three or four users at that point, it would have been impossible to go back.

Martin

Martin Aspeli

unread,
Feb 27, 2012, 3:06:40 PM2/27/12
to Sean Upton, plone-de...@lists.sourceforge.net

I'd favour a namespace package with extremely tight (FWT) control over who can use the namespace and for what.


> * If plone.api is a grassroots effort, I think the FWT needs to
> provide (and solicit from developers here and on product-developers
> list) steering, input, and design critique, review periodically (at
> least a few rounds of review every few months before any API is ever
> to be treated as initially stable)?
>

Agreed,

> * There needs to be infrastructure for discussion and code-review
> about each interfaces and each method, or some convention for using
> our existing tools (github, trac, etc) to facilitate discussion about
> the things in question (IMHO, mailing lists are too coarse-grained to
> dabble in the minutiae of specific method signatures, etc). With the
> right tools, the community can design an API by merit, not by
> happenstance.

Substantial alpha periods may help this.

Martin

Martin Aspeli

unread,
Feb 27, 2012, 3:18:33 PM2/27/12
to Sean Upton, plone-de...@lists.sourceforge.net
On 27 February 2012 17:30, Sean Upton <sdu...@gmail.com> wrote:

>> Can you please elaborate on how and when you recognised that adhering to
>> PEP8 for Dexterity was a mistake and what the consequences were?
>
> I would be interested in understanding the impacts of this as well.  I
> like consistency, but I also prefer and use LCUS style for method and
> identifier names exclusively in my own code.  I see the merits to
> both, but I may be blind to the downsides.
>
> The "cultural" argument:
>
> There is an indirect "feeling" about the tenor and personality of an
> API -- I cannot explain why, but from an "API" standpoint, LCUS
> (preferably with as few underscores) makes for APIs that look more
> friendly and less daunting.  Dunno, maybe it just looks less like
> Java;

I f-ing *hate* this argument. It crops up more than once in Python
circles. It's the kind of snobbery that makes open source developers
sound spoilt and petty.

(I realise I'm shooting the messenger here - sorry Sean)

> maybe it just seems more humble.  Maybe it forces you to write
> more resource-centric code to avoid ugly underscores (e.g. make
> abstractions look like mappings instead of controllers of some other
> resource's state -- see the way PAS deals with groups for an example).
>  As ugly as underscore word delimiters are, they are less ugly than
> mixed-case?
>
> I would guess that there are cultural pre-dispositions among the
> target audience of a simplified API (including Python devs outside of
> Plone/Zope pedigree) that would find LCUS identifier style more
> natural to find friendly.

The boat sailed a long time ago on this for Zope. Seriously. We have a
consistency problem already, but this would make it ten times worse.

For example:

import plone.api
import plone.api.content

catalog = get_tool(name='portal_catalog')

for item in catalog.searchResults(portal_type='News Item'):
obj = item.getObject()
plone.api.show_message("Found object %s" % obj.getId())

request = plone.api.get_request()
path = request['path']

obj = plone.api.content.get(path=path)
obj.setTitle("New title")

Okay, fairly non-sensical code, but the point here is that we have API
functions written in one style that return objects that use another
for their methods. To people learning Plone, this is a big deal. It
means you have to rely on rote learning to figure out what style to
use. Only once you've memorised the APIs (or figured out some kind of
obtuse heuristic based on the historical context of each class and
module) do you have a chance of guessing right most of the time.

We're not going to change the API of existing Zope and Plone
functions. Zope has had an explicit mixedCase policy for a long time,
which largely (but, sadly, not always) Plone has followed. Breaking
that now under the banner of trying to make things 'easier' for people
not familiar with Plone is just dumb.

Martin

Jon Stahl

unread,
Feb 27, 2012, 3:29:19 PM2/27/12
to Martin Aspeli, plone-de...@lists.sourceforge.net
On Mon, Feb 27, 2012 at 12:18 PM, Martin Aspeli
<optilud...@gmail.com> wrote:

> We're not going to change the API of existing Zope and Plone
> functions. Zope has had an explicit mixedCase policy for a long time,
> which largely (but, sadly, not always) Plone has followed. Breaking
> that now under the banner of trying to make things 'easier' for people
> not familiar with Plone is just dumb.

I usually don't weigh in on technical issues like this, but, FWIW, I
think that Martin's arguments are spot-on. Internal consistency is
really important.

US$0.02,
jon

David Glick (GW)

unread,
Feb 27, 2012, 3:45:32 PM2/27/12
to Martin Aspeli, plone-de...@lists.sourceforge.net


For the record, I also hate Zope and Plone's use of camel case. I'd like to find a way to move away from it in the long term, and I think plone.api is probably one of our best opportunities to set foot in the right direction.

Any argument for following the patterns of Zope/Plone API for the sake of consistency makes me laugh. Consistency? We ain't got none to follow.

That said, the topic feels like a silly one to spend a lot of community energy on, so this will be my last post on the matter. In the end what matters is that the APIs created are simple and *documented*, not how the words are separated in the names.

David

----------
David Glick
Web Developer
david...@groundwire.org
206.286.1235x32

The Engagement Party 2012. So much more fun than the wedding reception.

http://www.npoengagementparty.com

Domen Kožar

unread,
Feb 27, 2012, 3:44:05 PM2/27/12
to David Glick (GW), plone-de...@lists.sourceforge.net
Not that it counts, but +1 what David said. I'm all for consistency, but there is clearly no policy at the moment.

Jan-Carel Brand

unread,
Feb 27, 2012, 4:03:21 PM2/27/12
to David Glick (GW), plone-de...@lists.sourceforge.net

FWIW +1

> Any argument for following the patterns of Zope/Plone API for the sake of consistency makes me laugh. Consistency? We ain't got none to follow.

Plone will presumably in the future be using an increasing share of code
from the broader python/pyramid community that *does* adhere to PEP8.

Also, currently quite a few packages under the plone namespace already
use underscore method names.

> That said, the topic feels like a silly one to spend a lot of community energy on, so this will be my last post on the matter.
> In the end what matters is that the APIs created are simple and *documented*, not how the words are separated in the names.

Agreed, but I'm glad I could use this opportunity to weigh in.

Martin Aspeli

unread,
Feb 27, 2012, 4:05:07 PM2/27/12
to David Glick (GW), plone-de...@lists.sourceforge.net
On 27 February 2012 20:45, David Glick (GW) <david...@groundwire.org> wrote:

> For the record, I also hate Zope and Plone's use of camel case. I'd like to find a way to move away from it in the long term, and I think plone.api is probably one of our best opportunities to set foot in the right direction.

> Any argument for following the patterns of Zope/Plone API for the sake of consistency makes me laugh. Consistency? We ain't got none to follow.

I think it would be valuable to do some quick analysis. I think it's
more consistent than it may appear (though far, far from perfect).

Please bear in mind that I created an entire API for Dexterity and
then spent a long time redoing it, fixing up all the Dexterity code
and two projects using Dexterity alpha, because I was convinced the
API inconsistency was to the detriment of Dexterity's adoption and
ease of use for new developers.

Of course, "because optilude did it" is hardly a good reason to do
anything, but I think this is a more serious issue than it may appear
at first given plone.api's goals.

> That said, the topic feels like a silly one to spend a lot of community energy on, so this will be my last post on the matter. In the end what matters is that the APIs created are simple and *documented*, not how the words are separated in the names.

You are absolutely right, of course, but to borrow from some 'lean'
software development thinking: it's worth spending time "up front" on
decisions that are expensive to reverse. Naming conventions are
expensive to reverse once something is in use.

Martin

Nejc Zupan

unread,
Feb 27, 2012, 4:07:29 PM2/27/12
to Domen Kožar, plone-de...@lists.sourceforge.net
On 27/02/2012, at 21:44, Domen Kožar wrote:

> Not that it counts, but +1 what David said. I'm all for consistency, but there is clearly no policy at the moment.

+1

Also, when using any non-plone python package in a plone project you again get inconsistency.

Moreover, any developer trying to use general purpose Plone packages in a non-plone project
gets camel case into his/her project.


z.


David Glick (GW)

unread,
Feb 27, 2012, 4:27:05 PM2/27/12
to Nejc Zupan, plone-de...@lists.sourceforge.net


There, I Fixed It: ;-)


import re
from collections import MutableMapping


def camel_to_under(name):
s1 = re.sub('(.)([A-Z][a-z]+)', r'\1_\2', name)
return re.sub('([a-z0-9])([A-Z])', r'\1_\2', s1).lower()


class UnderscoreNamespace(MutableMapping):

def __init__(self):
self.ns = {}

def __getitem__(self, key):
return self.ns[camel_to_under(key)]

def __delitem__(self, key):
del self.ns[camel_to_under(key)]

def __setitem__(self, key, value):
self.ns[camel_to_under(key)] = value

def __len__(self):
return len(self.ns)

def __iter__(self):
return iter(self.ns)


if __name__ == '__main__':
ns = UnderscoreNamespace()
exec '''if 1:
fooBar = 42
print foo_bar
''' in {}, ns


(h/t Armin Ronacher -- https://github.com/mitsuhiko/badideas/blob/master/caseinsensitive.py)

Sean Upton

unread,
Feb 27, 2012, 4:44:37 PM2/27/12
to Fulvio Casali, plone-de...@lists.sourceforge.net
On Mon, Feb 27, 2012 at 12:54 PM, Fulvio Casali
<ful...@webcollective.coop> wrote:
> LCUS = ?
>
> Google is failing me here, sorry.

Sorry for jargon; LCUS == "lower-case underscore-separated".

Sean

Daniel Kraft

unread,
Feb 27, 2012, 4:28:29 PM2/27/12
to plone-de...@lists.sourceforge.net
Hi,

as plone.api leaked very early, even so early that we haven't had the chance to describe what we do and why, here's an explanation.


This document is there to describe the current state of the idea *plone.api*.

At the german Plone Konferenz we had a open space with eleddy and realized at some point that the documentation about plone is not only badly organized, but that it's also pretty hard to write good documentation.

If you want to teach users how to get the site root, there are about five different methods and none of them is recommended for a 100% usecase even from plone professionals. In addition you first have to teach the whole ZCA for the required knowledge about adapters. An acquisition tutorial would also be required.

Then, as we tried to find ideas how to make this all easier, we came up with the idea *plone.api*. As soon as we had this idea, we saw endless possibilities to make our lifes easier with the very same thing. We all know that there's a manage_something method to copy an object. But what's that *something*? And wasn't there a paste-method too?

So we discussed how this *plone.api* could look like. But there was one thing we agreed on **from the beginning**. This was it:

### Document first.

By starting with the documentation we will make sure that there will be

1. plenty of room for discussion
2. not much to re-program if we fail with the first try
3. a nice looking and usable api because we already use it (in the docs)

## Ideas

So we started with three possible structural layouts for the API.

### PHP style

Just create a bunch of methods to make life simple. Name them like what they do (e.g. remove_user_from_group). That way, the API would be easy to document and use. But not very pretty.

``` python
get_object(path=path)
create_object(path=path, id='some', title='...')
create_object(parent=site, id='some', title='...')
set_password(user_id='some', password='qwert')
```

### Namespaces style

That's like the PHP style, but we group similar functions into namespaces. Like to put all functions to do something on users into plone.api.users. Inside, there would be a function called 'leave_group'. Function names would be shorter, because the context of the function would be the group namespace.

``` python
content.get(path=path)
content.create(path=path, id='some', title='...')
users.set_password(user_id='some', password='qwert')
```

### Dict style

Another approach was to wrap us around plone. We wanted to emulate dict-style access to anything anywhere.

``` python
site = plone.api.site()
site['folder']['obj'] = plone.api.create(title='The object')
users['bob'].password = 'asdfg'
users['eliza'] = plone.api.create_user(password='secret')
```

# Current state

In this document I don't want to get into details about the advantages or disadvantages of the seperate possibilites for the api. All I want to say is that we are not 100% sure about it ourselves. So this whole idea is still subject to discussion.

As I already mentioned: Right now, we only set up the framework for the documentation. No code has been written. We are trying out if we can reach the goals to simplify documentation and to make our lifes as programmers easier.

We have a good selection of functions that we think make sense for a start of all this. You can find it here (currently in namespace style) as narrative and API documentation: http://readthedocs.org/docs/ploneapi/en/latest/index.html

Any discussion and examples of different approaches are very welcome. Not only about the amount or arguments of functions, but also about the style of the api. We will now slow down and let this settle.

We know that we must make the api right in the first place. When the api will be widely used (maybe, but hopefully not by plone core products), we won't be able to change it anymore. We know about that responsibility.


Cheers,
Daniel


P.S. This document will be available and possibly bugfixed at the github wiki for the project: https://github.com/plone/plone.api/wiki

Sean Upton

unread,
Feb 27, 2012, 5:09:15 PM2/27/12
to Martin Aspeli, plone-de...@lists.sourceforge.net
On Mon, Feb 27, 2012 at 1:18 PM, Martin Aspeli <optilud...@gmail.com> wrote:

> I f-ing *hate* this argument. It crops up more than once in Python
> circles. It's the kind of snobbery that makes open source developers
> sound spoilt and petty.
>
> (I realise I'm shooting the messenger here - sorry Sean)

Ah, it isn't a problem. I'm really on the fence about this sort of
argument and not invested enough about case-style one way or another
-- like others, this is mostly preference.

Side-note: I tend to think that often times there tends to be a
(incidental?) coupling of over-descriptive method names to bad API
design (usually means some class controlling state for something that
should control its own state, e.g. why don't we have an interface for
PAS groups as resources in-and-of-themselves... so of course the
groups plugin has descriptive names like listAssignedPrincipals() --
spelling this as group.keys() where group is a mapping alleviates the
need to really care about case, since we use an idiomatic succinct
interface like the Python mapping interface instead of something that
needs to be more self-describing).

> The boat sailed a long time ago on this for Zope. Seriously. We have a
> consistency problem already, but this would make it ten times worse.

Yeah, I see it in my own stuff -- most recently, I have adapters to
abstract common user/group operations in my own APIs, but these are
typically mappings returning IPropertiedUser objects, so I'm stuck
with my calling code mixing case conventions. I'm not sure what I
think about this.

Sean

Daniel Nouri

unread,
Feb 27, 2012, 5:41:18 PM2/27/12
to Daniel Kraft, plone-de...@lists.sourceforge.net
On Mon, Feb 27, 2012 at 10:28 PM, Daniel Kraft <daniel...@d9t.de> wrote:
> ### PHP style
> [...]
> ### Namespaces style
> [...]

> ### Dict style
>
> Another approach was to wrap us around plone. We wanted to emulate dict-style access to anything anywhere.
>
> ``` python
> site = plone.api.site()
> site['folder']['obj'] = plone.api.create(title='The object')
> users['bob'].password = 'asdfg'
> users['eliza'] = plone.api.create_user(password='secret')
> ```

FWIW, I still like what I'll call 'adapter-style' best:

>>> from plone.api import get_user
>>> user = get_user('eliza') # returns a plone.api.IUser
>>> user.password = 'asdfg'

>>> from plone.api import get_content
>>> content = api.get_content('/about-us') # returns a plone.api.IContent
>>> content.workflow_state
'private'
>>> content.transition('publish')
>>> french_content = content.get_translation('fr')

(Where it wouldn't matter if 'content' used LinguaPlone or
plone.app.multilingual for translations; the adapter would take care
of that.)

The "design decisions" [1] argue that it's too hard to explain to
users, and too much of an effort to write transparent adapters. Which
I don't think is true. On the contrary, I believe the adapter-style
would make client code look more Pythonic, for one, it would reduce
the number of imports one would have to make from plone.api. (The
example above would require four additional imports if function-style
were to be used.)


[1] http://readthedocs.org/docs/ploneapi/en/latest/rationale.html#faq
--
http://danielnouri.org

Sean Upton

unread,
Feb 27, 2012, 6:03:59 PM2/27/12
to Daniel Kraft, plone-de...@lists.sourceforge.net
On Mon, Feb 27, 2012 at 2:28 PM, Daniel Kraft <daniel...@d9t.de> wrote:
> ### PHP style
>
> Just create a bunch of methods to make life simple.

Better described as global functions than global methods, no? Yeah, I
think everyone wants namespaces, thus...

> ### Namespaces style
>
> That's like the PHP style, but we group similar functions into namespaces. Like to put all functions to do something on users into plone.api.users. Inside, there would be a function called 'leave_group'. Function names would be shorter, because the context of the function would be the group namespace.

Right, function names are shorter, but not quite short enough...

>
> ``` python
> content.get(path=path)
> content.create(path=path, id='some', title='...')
> users.set_password(user_id='some', password='qwert')
> ```

I see a more natural alternative -- make the object relationships
create natural namespaces, not just coarse-grained module namespaces:

>>> from plone import api
>>> me = api.users.get('some').set_password('qwert')
>>> assert me.id in api.users.groups.get('coolkids')

(implementation assumption: users is module, methods like get adapt
site-root just-in-time with an adapter that does the work, gets the
user, returns it). Performance concern/caveat: this API provides no
way for a view to pass an optional request to the API for purposes of
caching the various adapter lookups and construction -- where if this
were done in a view, the adaptation could happen once per request by
stuffing the adapter created into an annotation on the request object.

Maybe this could be alleviated by entering the API as an *adapter as
optional view of site* like such:

>>> from plone.api import api
>>> site = api.site() # no request, no caching of lookups, but simple...
>>> assert 'me' in site.users # __contains__ via uncached users adapter
>>> assert 'you' in site.users # underlying adapter must be
constructed again.
>>> class MyView(object):
... def __init__(self, context, request):
... self.context = context
... self.request = request
... self.site = None # don't set up in ctor, no security
manager yet
... def update(self, *args, **kwargs):
... if self.site is None:
... self.site = api.site(self.request)
... assert 'me' in site.users # __contains__ via cached
users adapter
... assert 'you' in site.users # underlying adapter
already cached for this req.
...
>>>

The implementation assumption is that the api object's site
method/factory can work with no request passed (default), but can pass
a request from downstream calling code where it makes lots of sense --
both for optimizing runtime performance, and for request-is-necessary
features, like getting the navigation root.

This style assumes that the "namespaces" you use are not module
namespaces, but attributes of the site object you obtain as an entry
point into the API. I think this is reasonable, and seems natural.
The site object then becomes an proxy to the actual ISiteRoot object
that provides ISiteRoot, OFS interfaces for the site root (or some
subsets or analogues) and exposes the topical API namespaces like
"users" and "workflow" etc.

> ### Dict style
>
> Another approach was to wrap us around plone. We wanted to emulate dict-style access to anything anywhere.
>
> ``` python
> site = plone.api.site()
> site['folder']['obj'] = plone.api.create(title='The object')
> users['bob'].password = 'asdfg'
> users['eliza'] = plone.api.create_user(password='secret')
> ```

Things that are resources in collections should be modeled as
mappings. For weak-typed containers (folders), use __setitem__(), but
the C in CRUD could better be spelled using an alternate add() method
on singly-typed containers/mappings, IMHO:

>>> site = plone.api.site()
>>> # for untyped heterogeneous mappings like folders:
>>> site['myfolder']['that'] =
plone.api.create(portal_type='mytype', title=u'That')
>>> # but for typed mappings:
>>> users.add('eliza', password='secret') # simpler to read, easier to type

I hope these observations are useful and not just bike-shedding.

Cheers,
Sean

Sean Upton

unread,
Feb 27, 2012, 6:10:59 PM2/27/12
to Daniel Nouri, plone-de...@lists.sourceforge.net
On Mon, Feb 27, 2012 at 3:41 PM, Daniel Nouri <daniel...@gmail.com> wrote:
> The "design decisions" [1] argue that it's too hard to explain to
> users, and too much of an effort to write transparent adapters.  Which
> I don't think is true.  On the contrary, I believe the adapter-style
> would make client code look more Pythonic, for one, it would reduce
> the number of imports one would have to make from plone.api.  (The
> example above would require four additional imports if function-style
> were to be used.)

If I understand correctly, I agree... this just commits you to writing
a new plone.api interface for every possible resource the system
exposes, instead of using native API resources:

plone.api.interfaces.IUser # instead of IPropertiedUser, this may be good [1]
plone.api.interfaces.IContent # instead of IContentish
plone.api.interfaces.IFolder # instead of OFS/CMF interfaces, also good [2]
...etc...


I don't see this as intrinsically bad thing, just more work. OTOH, it
lets you be internally consistent on the
underscore_separated_lower_case vs. mixedCase front by controlling the
vast majority of the interface. You do accept the performance
penalties of adaptation anywhere, though.

(notes below)


Sean

[1] IPropertiedUser is an okay user interface, but things like
portraits CMF defines in
why-on-earth-is-external-controller-of-state-being used style.

[2] Dexterity types all have same meta-type so OFS filters like
BTreeFolder2's _mt_index are not useful, but hey, we have the catalog,
and we could use portal_type instead of meta_type!

Laurence Rowe

unread,
Feb 27, 2012, 6:12:48 PM2/27/12
to Daniel Kraft, plone-de...@lists.sourceforge.net

I'm really glad you're all looking into this, and seeing how it all
looks when you document it is completely the right way to go about it.
I do feel a little uneasy about creating yet another way to do things
(see: http://xkcd.com/927/) but we absolutely have to settle on a
default way of doing things as a community (our documentation is full
of alternatives.) That might even just be collecting together a number
of existing util functions for easy access and then documenting those.

One thing that's really important though is getting this right in a
way that works from templates too. The portal_state and context_state
(and soon folderListing) views are all aimed at providing that one way
to get at information. When starting out, a lot of users will be
working only from templates as they are easily customisable TTW (it's
certainly how I started out with Plone, even before learning Python.)
Anything we can do to ease that transition to writing view based code
will help people a lot. Making those util views easily available would
be a great start and perhaps adding additional ones for content
creation and other common tasks.

I could imagine something like:

from plone.api import portal_state

def my_external_method(self):
return portal_state.getNavigationRootUrl()

This way we would emphasise continuity between code in templates and
code in Python.

Laurence

Daniel Nouri

unread,
Feb 27, 2012, 6:29:54 PM2/27/12
to Laurence Rowe, plone-de...@lists.sourceforge.net
On Tue, Feb 28, 2012 at 12:12 AM, Laurence Rowe <l...@lrowe.co.uk> wrote:
> from plone.api import portal_state
>
> def my_external_method(self):
>    return portal_state.getNavigationRootUrl()
>
> This way we would emphasise continuity between code in templates and
> code in Python.

+0

portal_state is by the way defined here:
https://github.com/plone/plone.app.layout/blob/master/plone/app/layout/globals/interfaces.py#L93

Daniel Nouri

unread,
Feb 27, 2012, 6:41:01 PM2/27/12
to Sean Upton, plone-de...@lists.sourceforge.net
On Tue, Feb 28, 2012 at 12:10 AM, Sean Upton <sdu...@gmail.com> wrote:
> On Mon, Feb 27, 2012 at 3:41 PM, Daniel Nouri <daniel...@gmail.com> wrote:
>> The "design decisions" [1] argue that it's too hard to explain to
>> users, and too much of an effort to write transparent adapters.  Which
>> I don't think is true.  On the contrary, I believe the adapter-style
>> would make client code look more Pythonic, for one, it would reduce
>> the number of imports one would have to make from plone.api.  (The
>> example above would require four additional imports if function-style
>> were to be used.)
>
> If I understand correctly, I agree... this just commits you to writing
> a new plone.api interface for every possible resource the system
> exposes, instead of using native API resources:
>
> plone.api.interfaces.IUser  # instead of IPropertiedUser, this may be good [1]
> plone.api.interfaces.IContent  # instead of IContentish
> plone.api.interfaces.IFolder  # instead of OFS/CMF interfaces, also good [2]
> ...etc...

Yes, this is what I'm thinking could be a good idea.

> I don't see this as intrinsically bad thing, just more work. OTOH, it
> lets you be internally consistent on the
> underscore_separated_lower_case vs. mixedCase front by controlling the
> vast majority of the interface.  You do accept the performance
> penalties of adaptation anywhere, though.

I don't think there's a relevant performance penalty here. (One extra
function call for the adaptation, and one extra for every method?)
Also consider that, for convenience reasons, functions such as
``plone.app.get_translation(content, 'fr')`` would probably also want
to work with adapters in the background for pluggability (one for
p.a.multilingual, one for LinguaPlone).

Daniel

Malthe Borch

unread,
Feb 28, 2012, 1:25:27 AM2/28/12
to Sean Upton, plone-de...@lists.sourceforge.net
On 28 February 2012 00:03, Sean Upton <sdu...@gmail.com> wrote:
> I hope these observations are useful and not just bike-shedding.

I didn't quite know where to make a comment, so I thought this was a
nice starting point.

It seems to me that you want to take the natural object graph and lay
it out on the file system so that `acl_users` becomes
`plone.api.users` and then use the site hooks to wire it back
together.

I believe this runs counter to the idea of an object graph. Instead of
following the parent chain up to the API plugging point (which
admittedly could be better defined), you'll rely on site hooks to make
that same choice – only this time it feels a lot more magical.

If we don't like `context.acl_users._plugins[<some interface>].<some
method>` – then let's improve it:

context.acl_users.get_member_by_id("me")

If you want to get rid of `acl_users` altogether and the tool-style
way of managing local state and behavior, then do that instead and put
it all on the site object (or make an adapter proxy).

Is that better? I'm not sure, but it sounds like a lot of work.

You can put lip-stick on a pig.

I can't help but feel there's a inverse to `plone.api`, which is to
factor out the logic from the legacy CMF and Plone code – into more
discoverable methods and then make the legacy methods call into those.

I.e. if we think there's a problem with objects X, Y and Z – and there
probably is – then let's fix it where the problem is, not
cosmetically.

In terms of discoverability, then I agree that it could be much better.

How about this:

>>> help(context.acl_users)
Help on ImplicitAcquisitionWrapper in module
Products.PluggableAuthService.PluggableAuthService object:

acl_users = class ImplicitAcquisitionWrapper(ExtensionClass.Base)
| Wrapper object for implicit acquisition

[snip]

| __abs__(...)
| x.__abs__() <==> abs(x)
|
| __add__(...)
| x.__add__(y) <==> x+y
|
| __and__(...)
| x.__and__(y) <==> x&y

That's not very helpful. It might instead describe what that object is
about, how it fits into your site, what's its path and UUID, and
mostly importantly, what are its methods and how do I use them
(examples included).

\malthe

------------------------------------------------------------------------------
Keep Your Developer Skills Current with LearnDevNow!


The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!

http://p.sf.net/sfu/learndevnow-d2d

Sean Upton

unread,
Feb 28, 2012, 2:12:10 AM2/28/12
to Malthe Borch, plone-de...@lists.sourceforge.net
On Mon, Feb 27, 2012 at 11:25 PM, Malthe Borch <mbo...@gmail.com> wrote:
> If you want to get rid of `acl_users` altogether and the tool-style
> way of managing local state and behavior, then do that instead and put
> it all on the site object (or make an adapter proxy).

I'm somewhat resigned in my own current work to keep acl_users and PAS
under the hood, but abstract the things (that I think) PAS does wrong
(or things that get put into too many places / CMF tools) into a
simple set of adapters, spelled like ISiteMembers(portal) (mapping +
CRUD for users), IGroups(portal) (enumerable mapping of all groups),
and IGroups(portal)['mygroup'] (mapping of a group to its user
keys/values). I'm using adapters on the site for all of this in my
own internal project.

> Is that better? I'm not sure, but it sounds like a lot of work.

Maybe -- going whole-hog on this approach requires replacing much
more, adapters for the site root, but generators and state proxies for
things like user objects (some new, better interface) where
IPropertiedUser is not sufficient.

> I can't help but feel there's a inverse to `plone.api`, which is to
> factor out the logic from the legacy CMF and Plone code – into more
> discoverable methods and then make the legacy methods call into those.

Taking the (admittedly narrow) example/niche of user management, I
can't help but agree in principle, but in practice, this has to be
done bottom-up, staring with PAS -- addressing the plugin
compatibility and/or porting story, which is more complicated. Some
*interim* middle ground is a new API that builds on top of PAS (for
now) but below the three CMF tools that manage users in the system.
Then portal_registration, portal_memberdata, and portal_membership can
be deprecated and eventually put out to pasture.

> I.e. if we think there's a problem with objects X, Y and Z – and there
> probably is – then let's fix it where the problem is, not
> cosmetically.

There is a short-term problem to be solved (simple API abstractions as
a needed coat of varnish), and a longer-term problem to be solved
(make underpinnings of this stuff easier to evolve with a modern
bottom-up API re-design, piecewise). I suspect some measure of both
is the pragmatic approach.

> In terms of discoverability, then I agree that it could be much better.
>
> How about this:
>
>   >>> help(context.acl_users)

Right. If we can ship ipython with installers, even better. But this
argument applies to any context, be it a module of interfaces, a
persistent site root, etc.

Sean

Martin Aspeli

unread,
Feb 28, 2012, 3:06:53 AM2/28/12
to Sean Upton, plone-de...@lists.sourceforge.net

The problem with this is that you end up with adapters or proxies
around everything to create the API you want. But that won't always
work. Many calls (e.g. a catalog search) returns objects where we
can't easily insert a proxy. And even if we could, you then need some
kind of 'unwrap' or unholy __getattr__ pass-through to get to the
'real' object.

I think the API design above would be nicer if we had complete control
over all parts of the system and designed it into the object model. We
don't. Every time we try to pain over that with proxies of some kind,
it's very painful and convoluted. Things like Acquisition and security
checks suddenly behave funny (or stop working). In the end, the user
of the API would need to know whether he was working with a 'real'
object or a proxy one at all times. Nightmare.

Martin

------------------------------------------------------------------------------
Keep Your Developer Skills Current with LearnDevNow!

The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!

http://p.sf.net/sfu/learndevnow-d2d

Malthe Borch

unread,
Feb 28, 2012, 3:14:33 AM2/28/12
to Sean Upton, plone-de...@lists.sourceforge.net
On 28 February 2012 08:12, Sean Upton <sdu...@gmail.com> wrote:
> I'm somewhat resigned in my own current work to keep acl_users and PAS
> under the hood, but abstract the things (that I think) PAS does wrong
> (or things that get put into too many places / CMF tools) into a
> simple set of adapters, spelled like ISiteMembers(portal) (mapping +
> CRUD for users), IGroups(portal) (enumerable mapping of all groups),
> and IGroups(portal)['mygroup'] (mapping of a group to its user
> keys/values).  I'm using adapters on the site for all of this in my
> own internal project.

While that's probably a neat improvement over the actual plugin APIs,
I'll suggest that the benefits don't outshine the complexity of having
that extra layer.

I do think PAS itself – as in the `acl_users` object (or possibly an
adapter on top of it) – could implement a number of convenience
methods which all acted on a high level:

acl_users.iter_groups()
acl_users.user_property_proxy_for("me")

I don't think we should dumb down the API. There's a lot of
indirection and abstraction in there and it's got to be visible to the
developer. I.e. in the example above, if it's a proxy, tell that it's
a proxy.

> There is a short-term problem to be solved (simple API abstractions as
> a needed coat of varnish), and a longer-term problem to be solved
> (make underpinnings of this stuff easier to evolve with a modern
> bottom-up API re-design, piecewise).  I suspect some measure of both
> is the pragmatic approach.

We can't make the object graph go away.

There's an object called "acl_users" and it's integral to the way the
stack currently does authentication / authorization. Why not just
embrace it and make it easier to work with?

That XKCD sketch previously linked to in this thread really does
apply. If you're going to make one more indirection in the name of
making things easier, you're bound to fail.

>>   >>> help(context.acl_users)
>
> Right.  If we can ship ipython with installers, even better.  But this
> argument applies to any context, be it a module of interfaces, a
> persistent site root, etc.

What I meant is just that it might be a great point of documentation:

1) Use PDB. It's your friend.
2) Discover the API by calling `help` on the objects you find.

\malthe

Daniel Nouri

unread,
Feb 28, 2012, 5:02:43 AM2/28/12
to Malthe Borch, plone-de...@lists.sourceforge.net
On Tue, Feb 28, 2012 at 9:14 AM, Malthe Borch <mbo...@gmail.com> wrote:
>> There is a short-term problem to be solved (simple API abstractions as
>> a needed coat of varnish), and a longer-term problem to be solved
>> (make underpinnings of this stuff easier to evolve with a modern
>> bottom-up API re-design, piecewise).  I suspect some measure of both
>> is the pragmatic approach.
>
> We can't make the object graph go away.
>
> There's an object called "acl_users" and it's integral to the way the
> stack currently does authentication / authorization. Why not just
> embrace it and make it easier to work with?
>
> That XKCD sketch previously linked to in this thread really does
> apply. If you're going to make one more indirection in the name of
> making things easier, you're bound to fail.

I think that at a minimum, and this is something that probably
everyone agrees on, is that plone.api will import some of the most
useful names that are currently spread out between a lot of individual
packages, like getSite, getTool, what have you. The advantage here is
that people don't have to remember, "was it Products.CMFCore.utils or
Products.CMFPlone.utils, or plone.app.thisandthat that I have to
import from?".

The same argument might apply to the object graph: is it
portal.portal_memberdata or portal.portal_membership or
portal.acl_users that I have to look into to get a user's email
address?

Once you agree that collecting the most important things and putting
them into one package is a good idea, and that fixing the current APIs
is not going to make this problem go away, then you're tempted to
think about not just aliasing the 'raw' functionality, but put a nice
coating around it for the sake of consistency and easier
understanding.

This could be done without touching the original code, and made work
today (and possibly for Plone 3). And once the original code (say
acl_users) follows the new, say, IUsers interface, plone.api would no
longer need to adapt it. So it's still a useful starting point for
making the underlying code saner.

Daniel

Elizabeth Leddy

unread,
Feb 28, 2012, 2:06:15 PM2/28/12
to Daniel Nouri, plone-de...@lists.sourceforge.net
On Feb 28, 2012, at 11:02 AM, Daniel Nouri wrote:

> On Tue, Feb 28, 2012 at 9:14 AM, Malthe Borch <mbo...@gmail.com> wrote:
>>> There is a short-term problem to be solved (simple API abstractions as
>>> a needed coat of varnish), and a longer-term problem to be solved
>>> (make underpinnings of this stuff easier to evolve with a modern
>>> bottom-up API re-design, piecewise). I suspect some measure of both
>>> is the pragmatic approach.
>>
>> We can't make the object graph go away.
>>
>> There's an object called "acl_users" and it's integral to the way the
>> stack currently does authentication / authorization. Why not just
>> embrace it and make it easier to work with?
>>
>> That XKCD sketch previously linked to in this thread really does
>> apply. If you're going to make one more indirection in the name of
>> making things easier, you're bound to fail.
>
> I think that at a minimum, and this is something that probably
> everyone agrees on, is that plone.api will import some of the most
> useful names that are currently spread out between a lot of individual
> packages, like getSite, getTool, what have you. The advantage here is
> that people don't have to remember, "was it Products.CMFCore.utils or
> Products.CMFPlone.utils, or plone.app.thisandthat that I have to
> import from?".

This is so important. It's impossible.

>
> The same argument might apply to the object graph: is it
> portal.portal_memberdata or portal.portal_membership or
> portal.acl_users that I have to look into to get a user's email
> address?
>
> Once you agree that collecting the most important things and putting
> them into one package is a good idea, and that fixing the current APIs
> is not going to make this problem go away, then you're tempted to
> think about not just aliasing the 'raw' functionality, but put a nice
> coating around it for the sake of consistency and easier
> understanding.
>
> This could be done without touching the original code, and made work
> today (and possibly for Plone 3). And once the original code (say
> acl_users) follows the new, say, IUsers interface, plone.api would no
> longer need to adapt it. So it's still a useful starting point for
> making the underlying code saner.

+100. It's super easy to be theoretical and nitpick about layers but a big reason that this layer is needed is because no one is actually fixing the underlying layers and based on the pace of things I doubt it will happen overnight, or even in the next X years. I doubt if we even have a list of all the things that need to be done. For example, what EXACTLY needs to be done to deprecate CMF? If you really hate the layer idea, please start a new thread on how to get going on the rest of this stuff.

It doesn't matter if plone.api isn't used in the end. I actually hope it disappears in a few years because it will mean that we got our shit together. In the meanwhile, it will offer people some sanity and a place for people who aren't familiar with plone to transition into the hardcore parts of it. Furthermore, as the API changes there will be 1 and only 1 place to mark that change, and it will actually indicate that it IS deprecated.

Also just want to mention that the guys at the sprint actually talked to people about how they want to code. And by people I don't mean hardcore plone devs who already know getToolByName. They talked to integrators and average developers and have been really open to feedback. It wasn't a sit down and masturbate over rst session. Before balking over choices like PEP8, just remember that this is what people wanted. Who cares if it was always done a certain way - old folks can always use the old way and new folks can wait a few more months before wanting to smash their monitor. My favorite demotivator says "Just because it has always been done that way doesn't mean it's not incredibly stupid".

This is the first time I've seen a fully documented API without a single line of code and furthermore people actually asking for input on it before moving forward. I hope many people look at this process and really respect what these guys did. More python, more feedback, less zope. Onwards and upwards.

Liz

Martin Aspeli

unread,
Feb 28, 2012, 2:39:04 PM2/28/12
to Elizabeth Leddy, plone-de...@lists.sourceforge.net
On 28 February 2012 19:06, Elizabeth Leddy <elizabe...@gmail.com> wrote:

> Also just want to mention that the guys at the sprint actually talked to people about how they want to code. And by people I don't mean hardcore plone devs who already know getToolByName. They talked to integrators and average developers and have been really open to feedback. It wasn't a sit down and masturbate over rst session. Before balking over choices like PEP8, just remember that this is what people wanted. Who cares if it was always done a certain way - old folks can always use the old way and new folks can wait a few more months before wanting to smash their monitor. My favorite demotivator says "Just because it has always been done that way doesn't mean it's not incredibly stupid".

I would love it if all of Zope/Plone were PEP8. My concern is that we
can't (and don't want to) hide/supplant *all* the methods people will
call in their code, and so we end up making it harder to guess what
the correct call is by actively breaking the Zope convention (which is
imperfectly applied, but it's not as random as it may seem in isolated
cases).

This is based on real experience of trying to do the "right" thing
with PEP8 with Dexterity and ultimately deciding it was an un-winnable
battle.

I suspect if you ask "do you want PEP8 or not" the answer is "I do".
If the question was, "do you want PEP8 for some things, even if other
parts of the API stay with a different conventino" it may not be as
clear cut.

PEP8 itself actually explicitly says: "where an existing library has a
different style, internal consistency is preferred."

It may be incredibly stupid, but it's consistently stupid and
sometimes that's better than partly stupid and partly somewhat less
stupid. In any case, the "pure" PEP8 argument sounds to me like a
purist one rather than a pragmatic one.

Anyway.

This will be my last attempt at making this point. plone.api is a
great effort, underscores or not, and I don't want to be stop energy.
I desperately want it to succeed.

Martin

Daniel Nouri

unread,
Feb 29, 2012, 5:04:58 AM2/29/12
to Martin Aspeli, plone-de...@lists.sourceforge.net
On Tue, Feb 28, 2012 at 8:39 PM, Martin Aspeli <optilud...@gmail.com> wrote:
> PEP8 itself actually explicitly says: "where an existing library has a
> different style, internal consistency is preferred."

I agree that camel-case might be a good idea.

There's actually a much more controversial 'design decision' in the
current plone.api docs, which I totally disagree with: "No positional
arguments. Only named (keyword) arguments [in all of plone.api]".

Basically, the authors want us to write "api.move(from_=obj,
to=folder)" every time explicitly, instead of being able to call the
function using positional arguments: "api.move(obj, folder)" -- that's
not very user friendly.

And then folks still have very different ideas about whether we should
use functions, or adapters/methods. That's
"api.getTranslation(context, 'fr')" versus
"api.content(context).getTranslation('fr')". I prefer adapters,
though I'm starting to care less about this.

Designing plone.api is hard, even more so because it's not just any
other package, but wants to be used by everyone. I.e. we need some
experienced folks to look at it, and be critical.


Daniel

------------------------------------------------------------------------------
Virtualization & Cloud Management Using Capacity Planning
Cloud computing makes use of virtualization - but cloud computing
also focuses on allowing computing to be delivered as a service.
http://www.accelacomm.com/jaw/sfnl/114/51521223/

Rok Garbas - gmail.com

unread,
Feb 29, 2012, 6:42:33 AM2/29/12
to plone-de...@lists.sourceforge.net

On 2012-02-29 10:04:58 +0000, Daniel Nouri said:


On Tue, Feb 28, 2012 at 8:39 PM, Martin Aspeli <optilud...@gmail.com> wrote:

PEP8 itself actually explicitly says: "where an existing library has a

different style, internal consistency is preferred."


I agree that camel-case might be a good idea.


i agree with optilude that: plone.api.users.get('someuser').setProperties(..) is not nice since mixing camelcase and pep8 style just feels weird. but there is a way to follow pep8 style all the way.


by creating a wrapper object (i think nouri called it adapter way) for users which exposes only 20% of functionality (that is used 80% of time) and is keeping everything pep8 compatible. this would gives us api like: plone.api.users.get('someuser').password = 'somepass'


and in case we need "raw" object to work on we could use "get_raw" accross all wrappers. eg: plone.api.users.get('someuser').get_raw().setProperties(…)

and then if you decided to get freaky with the user object you can do what ever you want after "get_raw" call.


this could provide us to create freaking sexy api which would solve most of the integrators problems.



There's actually a much more controversial 'design decision' in the

current plone.api docs, which I totally disagree with: "No positional

arguments. Only named (keyword) arguments [in all of plone.api]".


Basically, the authors want us to write "api.move(from_=obj,

to=folder)" every time explicitly, instead of being able to call the

function using positional arguments: "api.move(obj, folder)" -- that's

not very user friendly.


their argument for this is that its easier to keep backwards compatibility. i'm -1 on it since having nicer api and easier to use is priority for me. "looking sexy all the way" should be it motto :) 


even if it happens that we have to change something in the future we can do it by creating api2 in case its not easily solvable with depracating. in any case we'll document all changes of api and thats for me enough.



And then folks still have very different ideas about whether we should

use functions, or adapters/methods.  That's

"api.getTranslation(context, 'fr')" versus

"api.content(context).getTranslation('fr')".  I prefer adapters,

though I'm starting to care less about this.


Designing plone.api is hard, even more so because it's not just any

other package, but wants to be used by everyone.  I.e. we need some

experienced folks to look at it, and be critical.


+1 on adapters thingy or whatever is used for implementing it. what matters is how it looks and that is love on the first sight for every new developer. 




and +100 for all the plone.api guys i know its hard to please everybody and tnx you for trying anyway.




-- 

Rok Garbas

http://www.garbas.si


Daniel Nouri

unread,
Feb 29, 2012, 7:23:08 AM2/29/12
to Rok Garbas - gmail.com, plone-de...@lists.sourceforge.net
On Wed, Feb 29, 2012 at 12:42 PM, Rok Garbas - gmail.com
<rok.g...@gmail.com> wrote:
> their argument for this is that its easier to keep backwards compatibility.
> i'm -1 on it since having nicer api and easier to use is priority for me.
> "looking sexy all the way" should be it motto :)
>
> [...]

>
> +1 on adapters thingy or whatever is used for implementing it. what matters
> is how it looks and that is love on the first sight for every new
> developer.

Agreed.

> and +100 for all the plone.api guys i know its hard to please everybody and
> tnx you for trying anyway.

Yes!

Martin Aspeli

unread,
Feb 29, 2012, 9:40:41 AM2/29/12
to Rok Garbas - gmail.com, plone-de...@lists.sourceforge.net
Hi,

On 29 February 2012 11:42, Rok Garbas - gmail.com <rok.g...@gmail.com> wrote:
> On 2012-02-29 10:04:58 +0000, Daniel Nouri said:
>
>
> On Tue, Feb 28, 2012 at 8:39 PM, Martin Aspeli <optilud...@gmail.com>
> wrote:
>
> PEP8 itself actually explicitly says: "where an existing library has a
>
> different style, internal consistency is preferred."
>
>
> I agree that camel-case might be a good idea.
>
>
> i agree with optilude that:
> plone.api.users.get('someuser').setProperties(..) is not nice since mixing
> camelcase and pep8 style just feels weird. but there is a way to follow pep8
> style all the way.
>
>
> by creating a wrapper object (i think nouri called it adapter way) for users
> which exposes only 20% of functionality (that is used 80% of time) and is
> keeping everything pep8 compatible. this would gives us api like:
> plone.api.users.get('someuser').password = 'somepass'
>
> and in case we need "raw" object to work on we could use "get_raw" accross
> all wrappers. eg: plone.api.users.get('someuser').get_raw().setProperties(…)
>
> and then if you decided to get freaky with the user object you can do what
> ever you want after "get_raw" call.
>
>
> this could provide us to create freaking sexy api which would solve most of
> the integrators problems.

I think this is a recipe for disaster. Here are some reasons:

1) We now end up with two parallel object models for key entities. Yay
for simplicity.
2) We are now subscribed to doing more than just giving access to
objects, we also have to design APIs for what people may want to do
with those objects. This is a huge task.
3) Some methods/calls will expect the wrappers, other methods/calls
will expect raw objects. Integrators now need to be aware of both
class hierarchies. The choice will seem arbitrary to them.
4) Your 20% are different to my 20%. I guarantee you that you won't be
able to predict all the APIs that are used in all scenarios, so people
will always come up against the need for get_raw(). When they do, they
move up the complexity curve exponentially. The first question will be
"why is it like this".

This sounds like *exactly* the kind of over-engineered solution
plone.api is trying to avoid being. We're going from a simple set of
entry points with great documentation (which is really where the value
of this is, not in the code), to a complex engineering exercise where
we have to identify and evaluate what to wrap and how and trace the
use of those wrapped objects deep into the types of methods people may
want to call, which we then also have to replicate or mandate a
get_raw() call. It won't be long until someone suggests doing
pass-throughs with __getattr__ tricks or using C level wrapping with
zope.proxy. And why not let all the APIs be pluggable with adapters,
which you then have to bootstrap up with ZCML.

Disaster.

The rationale for doing this is that we want PEP8. In fact, it's that
we want the previously-optional underscore method name provision of
PEP8. Get real. Our end users don't give a crap. They want simple
things they can understand and pdb through. They want one set of
concepts, not three.

FWIW, I really hope the plone.api developers delete these mails unread
(including my rants about mixedCase) and continue with their efforts
rather than spend their time trying to get their heads around the
various well-intentioned technical proposals here. All of this is
really just noise by people who are not at all the target audience for
plone.api.

If we want cleaner APIs for the properties and methods that people get
back when calling a plone.api helper function, why don't we just go
and improve the classes we already have, rather than wrap a new level
of indirection on top of them? Start as the plone.api did, by writing
good documentation. Then add the new methods that implement those
docs. Then deprecate any duplicate/unnecessary ones. Yes, this is less
sexy than coming up an automated integration approach, but it is a lot
more valuable to the target audience here.

Martin

Daniel Nouri

unread,
Feb 29, 2012, 10:16:28 AM2/29/12
to Martin Aspeli, plone-de...@lists.sourceforge.net, Rok Garbas - gmail.com
On Wed, Feb 29, 2012 at 3:40 PM, Martin Aspeli <optilud...@gmail.com> wrote:
> [...]

> we have to identify and evaluate what to wrap and how and trace the
> use of those wrapped objects deep into the types of methods people may
> want to call, which we then also have to replicate or mandate a
> get_raw() call. It won't be long until someone suggests doing
> pass-throughs with __getattr__ tricks or using C level wrapping with
> zope.proxy. And why not let all the APIs be pluggable with adapters,
> which you then have to bootstrap up with ZCML.

I'd make them proxies indeed. I don't consider __getattr__ to be very tricky.

> Disaster.
>
> The rationale for doing this is that we want PEP8. In fact, it's that
> we want the previously-optional underscore method name provision of
> PEP8. Get real. Our end users don't give a crap. They want simple
> things they can understand and pdb through. They want one set of
> concepts, not three.

PEP8 is not the rationale. The rationale is to give people sane
object methods and attributes, where they can do:

user.password = 'foo'
user.email = 'foo'
user.thatattribute = 123

... regardless of whether these stored in acl_users or
portal_memberdata, properties or not. Can we realistically fix the
underlying implementations to do that -- in a reasonable time frame?

Or:

page.getTranslation('fr')
page.title = 'pub'

Consider that we can't even fix Archetypes to support attribute-style access.


Daniel

Wichert Akkerman

unread,
Feb 29, 2012, 10:22:41 AM2/29/12
to Daniel Nouri, plone-de...@lists.sourceforge.net, Rok Garbas - gmail.com
On 02/29/2012 04:16 PM, Daniel Nouri wrote:
> On Wed, Feb 29, 2012 at 3:40 PM, Martin Aspeli<optilud...@gmail.com> wrote:
>> [...]
>> we have to identify and evaluate what to wrap and how and trace the
>> use of those wrapped objects deep into the types of methods people may
>> want to call, which we then also have to replicate or mandate a
>> get_raw() call. It won't be long until someone suggests doing
>> pass-throughs with __getattr__ tricks or using C level wrapping with
>> zope.proxy. And why not let all the APIs be pluggable with adapters,
>> which you then have to bootstrap up with ZCML.
> I'd make them proxies indeed. I don't consider __getattr__ to be very tricky.
>
>> Disaster.
>>
>> The rationale for doing this is that we want PEP8. In fact, it's that
>> we want the previously-optional underscore method name provision of
>> PEP8. Get real. Our end users don't give a crap. They want simple
>> things they can understand and pdb through. They want one set of
>> concepts, not three.
> PEP8 is not the rationale. The rationale is to give people sane
> object methods and attributes, where they can do:
>
> user.password = 'foo'
> user.email = 'foo'
> user.thatattribute = 123

Now imagine someone has an attribute that is not a legal identifier in
python in their LDAP/SQL/whatever system. What do you do now?

Wichert.

Daniel Nouri

unread,
Feb 29, 2012, 10:27:17 AM2/29/12
to Wichert Akkerman, plone-de...@lists.sourceforge.net, Rok Garbas - gmail.com

Easy: getattr.

Anyway I should say again that I don't feel strongly about this.
Whatever. Might be a bad idea if it makes everyone nervous.

Daniel

jure

unread,
Feb 29, 2012, 12:28:44 PM2/29/12
to plone-de...@lists.sourceforge.net
Hi,


On 02/29/2012 11:04 AM, Daniel Nouri wrote:
I agree that camel-case might be a good idea.

I was also 100% sold on the pep8 style, though I'm not sure anymore. To me, the pep8 looks more "modern" and simple (though ironically, I hated it at first, coming from the Java world), but mixing the two might be confusing for the users.


On 02/29/2012 03:40 PM, Martin Aspeli wrote:
This sounds like *exactly* the kind of over-engineered solution
plone.api is trying to avoid being. 
+1

Let's keep it simple :)


There was also a discussion to simplify the registration of components. We already have Grok, I would like to know what are the reasons for all the hate it gets? Do you prefer to have all configuration in one place? Do you feel uneasy about adding "strange" grok.name, grok.context etc. directives to your classes? Would it be better if we used decorators, like in pyramid? Something like:

@view_config(name='my-fancy-view', context=ISomeContext, permission='zope2.View')
class MyView(BrowserView):
    ...

Parameter 'name' could default to the name of the class lower-cassed ('my-view') and context to '*' (having a default value for permission is probably a bad idea :) ). So registering a view would be simple as:

@view_config(permission='zope2.View')
class MyView(BrowserView):
    ...
 
Cheers,
Jure
-- 
Jure Cerjak

Termitnjak d.o.o.
Ulica Janeza Pavla II 007, 1000 Ljubljana
Web: http://www.termitnjak.com
Tel: +386 (0)40 162-740
Email: jce...@termitnjak.si 

Malthe Borch

unread,
Feb 29, 2012, 2:29:00 PM2/29/12
to Martin Aspeli, plone-de...@lists.sourceforge.net, Rok Garbas - gmail.com
On 29 February 2012 15:40, Martin Aspeli <optilud...@gmail.com> wrote:
> If we want cleaner APIs for the properties and methods that people get
> back when calling a plone.api helper function, why don't we just go
> and improve the classes we already have, rather than wrap a new level
> of indirection on top of them? Start as the plone.api did, by writing
> good documentation. Then add the new methods that implement those
> docs. Then deprecate any duplicate/unnecessary ones. Yes, this is less
> sexy than coming up an automated integration approach, but it is a lot
> more valuable to the target audience here.

And then why don't we just go and add these convenience methods to the
`PloneSite` object, which might then call into various components to
complete the calls (mostly already existing components).

The fact is that `PloneSite` has to know an awful lot about these
components already – at least that they exist and what symbolic name
they go by and/or utility lookup signature.

Wouldn't it be great to be able – anywhere where you have a context – to say:

>>> context.<api method>(<args>)

It would just do the right thing based on what context you've got, its
parent chain and so on. This is acquisition and unless abused and
misunderstand as it often is and has been, it works incredibly well
with a persistent object graph.

Let's work with what we've got and improve that, instead of trying to
"make it all go away". Zope 2 won't go away so why not just embrace it
and/or improve it –– or take active steps to do away with it, if
people think it doesn't work for them anymore.

+1 for a `PloneSite` that lets you do things easily.

Alternatively,

+1 for merging a number of tools and embrace them as APIs for their
respective domain.

Note that this fully integrates with components as long as the tools
delegate to utilities and adapters as needed.

\malthe

Martin Aspeli

unread,
Feb 29, 2012, 3:15:05 PM2/29/12
to Malthe Borch, plone-de...@lists.sourceforge.net, Rok Garbas - gmail.com
On 29 February 2012 19:29, Malthe Borch <mbo...@gmail.com> wrote:
> On 29 February 2012 15:40, Martin Aspeli <optilud...@gmail.com> wrote:
>> If we want cleaner APIs for the properties and methods that people get
>> back when calling a plone.api helper function, why don't we just go
>> and improve the classes we already have, rather than wrap a new level
>> of indirection on top of them? Start as the plone.api did, by writing
>> good documentation. Then add the new methods that implement those
>> docs. Then deprecate any duplicate/unnecessary ones. Yes, this is less
>> sexy than coming up an automated integration approach, but it is a lot
>> more valuable to the target audience here.
>
> And then why don't we just go and add these convenience methods to the
> `PloneSite` object, which might then call into various components to
> complete the calls (mostly already existing components).
>
> The fact is that `PloneSite` has to know an awful lot about these
> components already – at least that they exist and what symbolic name
> they go by and/or utility lookup signature.
>
> Wouldn't it be great to be able – anywhere where you have a context – to say:

This may not be a bad option, but I have two concerns:

1) You'd need to always get hold of a plone site root before doing
anything, even if what you wanted to do was mainly context-less (not
that there really is such a thing in Zope, but conceptually at least
things like members and tool lookup is not contextual)

2) In tests, you'd need a real or fake Plone site root. Maybe not an
issue since the thing the methods operate on is probably contextual in
most cases, but still.

3) The Plone site object has a ton of crufty properties/methods, so
help(siteroot), say, is not very useful/discoverable.

>  >>> context.<api method>(<args>)
>
> It would just do the right thing based on what context you've got, its
> parent chain and so on. This is acquisition and unless abused and
> misunderstand as it often is and has been, it works incredibly well
> with a persistent object graph.

What about the risk of name clashes with context-specific methods? Is
that a real risk?

> Let's work with what we've got and improve that, instead of trying to
> "make it all go away". Zope 2 won't go away so why not just embrace it
> and/or improve it –– or take active steps to do away with it, if
> people think it doesn't work for them anymore.

There is some merit in this argument, though I don't think there's a
huge difference whether we have a documented API that starts with a
site root lookup or a documented API that starts with a module import.
The docs and discoverability are more important.

Martin

Alex Clark

unread,
Mar 2, 2012, 9:30:24 AM3/2/12
to plone-de...@lists.sourceforge.net
Hi,


As a "hack" programmer, I wanted to add a few belated comments FWIW:

1. I haven't commented yet, because Martin has said everything for me
(in a strange, borg-like way, he wrote what I was about to write several
times; or, something very similar to it). In particular, I think:

a.) PEP8 ALL THE THINGS. I run flake8 on every module I touch
now-a-days, so for the things it catches I care about consistency. I
also, I think, follow the PEP8 underscore convention e.g. my_method()
instead of myMethod(). But AFAIK flake8 does not enforce this so I could
care less what legacy code is doing, as far as that syntax is concerned.
However any code I am writing is going to be "new style".

b.) DO NOT WORRY about inconsistencies in result sets. If I do:

from plone.api import do_something_useful

def my_method(request):
results = do_something_useful(request)
# XXX Do something with results
return results

I don't care if "results" is not iterable, or doesn't have a particular
method on it, or about any number of things other folks have argued
for/against. If it's well-documented what it returns, I'll figure it
out. One of the most important things is that I have an easy entrance to
start doing my work.

c.) OUR CHANCE to get it right came and went 13 years ago. If Plone
started in 1999, then that is when the chance to get the API design
right came and went. I don't buy any arguments about making sure we get
it right c.f. mcdonc and Pyramid. He was referring to his one chance
when he made those comments, about a new project he was starting it at
the time. None of that applies to Plone in 2012 IMHO. Two of the most
important things for plone.api to do is: I. exist and II. be well
supported, documented, and maintained.

d.) DO IT RIGHT, but not perfect. I do agree with the arguments that
we don't need another shitty way to do things in Plone. We have a half a
dozen or so ways to do common tasks. But "right" is subjective, and in
this case I am referring to approach not implementation. It's "right" to
design and build and document a new plone.api, if there are enough
talented people committed to seeing it through. C.f. Martin, Laurence,
et al who single (or double as it were)-handedly reinvented Plone
theming. Diazo works really, really, really well FWIW. It sucks that
plone.app.theming is tied so closely to the Z2 publisher, but life goes
on. Let's reinvent Plone programming the same way.

3.) START SMALL. As a consumer and possible contributor, I'm looking
for a few things. I don't particularly care about documenting everything
first, though that is nice. I want an alpha release that does 1 or 2 or
3 things ONLY. Let's get something out there and get it right and move on.


Kudos and thanks, plone.api folks! Please keep going.

Alex

>
> Martin
>
> ------------------------------------------------------------------------------
> Virtualization& Cloud Management Using Capacity Planning


> Cloud computing makes use of virtualization - but cloud computing
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/


--
Alex Clark · http://pythonpackages.com

Fulvio Casali

unread,
Mar 4, 2012, 6:54:05 PM3/4/12
to plone-de...@lists.sourceforge.net

As a "hack programmer" myself, and one of the Munich plone.api sprint participants I wanted to make a modest proposal.

It seems to me that there was a point in the Munich open space where the discussion definitely lifted off.  The lift-off happened when someone admitted to not knowing, or not being able to remember, how to write the code to do something that should be simple, such as copying a content object.  Everyone could relate to that frustration.  Everyone.  No doubt, it wasn't just about "copying an object".
I think we should not confuse the momentum behind plone.api with wanting to create a "great" API.  If we let the conversation go in that direction, everyone is going to produce a different wishlist, and there is no way we can make everyone happy.  I'm also not completely on board with the idea of solving 20% of the use cases that cause 80% of the problems.  That sounds too much like a common denominator approach, that could end up making everybody unhappy.
The momentum originates from the possibility that, someday, with this API I might actually be able to write (and remember) the simple method call required to do a very simple thing.  And so, while I'm proud of the sphinx docs we produced, and of our "document first" approach, perhaps to some extent this approach distracts us from where the energy is, and what we are trying to do.
Instead (or in addition) the energy comes from:  "I really hate that to do A I have to use this crappy xyz code!"

So, can we start a collection, a little gallery of horrors?  Here is a silly example of what I mean.  I'm going to paste a code snippet that I hate, and I'm going to explain what I would like to have instead.  After that, people can weigh in on what disadvantages my desired "API" would have, or why it would not work, or how it could be solved better.  Keep in mind, this is just an example of what an entry in the little gallery of horrors could look like, it could be anything else.  Don't fixate on the example.

My example
<tal:control_panel
tal:define="is_manager python:context.portal_membership.checkPermission('Manage portal', context);"
tal:condition="is_manager">
<dd>
<a href=""

tal:content="string:Site Setup"
tal:attributes="href string:${context/portal_url}/plone_control_panel;
title string:Site Setup">
Site Setup
</a>

</dd>
</tal:control_panel>

Why this sucks
The problem is not TAL, it's the double indirection to a method that I have to call with what looks like a set of positional arguments in an arbitrary order.  Could I please just have a global is_manager that I don't need to define?  If I set up my own set of custom permissions, I guess I'll be fine doing the python:context.portal_membership.checkPermission('Do something unusual', context).  Plone ships with a set of stock permissions, other than manager, so all of those should be available globally.  Actually, it would be nice if a global is_mycustomperm could be generated automatically when a new permission is defined.
Actually, this example contains two horribles in one.  What's with the ${context/portal_url}?  I can never remember when I can use portal_url and when I can't.  Why context?  Why would portal_url depend on it?  Subsites don't ship with Plone out of the box.

What would be better
Can I have this, please?
<tal:control_panel
tal:condition="is_manager">
<dd>
<a href=""
tal:content="string:Site Setup"
tal:attributes="href string:${portal_url}/plone_control_panel;

title string:Site Setup">
Site Setup
</a>
</dd>
</tal:control_panel>

Discussion, pros/cons
Is there a performance penalty to having all the permissions computed for the context at request time?


I don't have a strong preference on how this little gallery of horrors should be implemented.  Sphinx might work.  Google moderator, maybe.  [I'm a fan of wikis, I like how in MediaWiki (e.g. wikipedia) there is a separation between the content and the discussion about the content (they are on different tabs), and yet there is no barrier to either editing the content, or adding to the discussion, and full history is preserved (again with no barrier, no context switch).]

It's great that we started writing the documentation for plone.api, and even included examples for each element of it.  But somehow divorcing this documentation from the horrors we are trying to fix seems counterproductive.

Of course, the "little gallery of horrors" and the "official" documentation have to be integrated somehow, and this is another problem.

Finally, I think that while it's certainly better to start small than not at all, it should be possible to let plone.api grow over time to cover more than the 20/80 scenario that was proposed. 

-- 
Fulvio

Malthe Borch

unread,
Mar 5, 2012, 12:31:12 AM3/5/12
to Fulvio Casali, plone-de...@lists.sourceforge.net
On 5 March 2012 00:54, Fulvio Casali <ful...@solitonconsulting.com> wrote:
> The problem is not TAL, it's the double indirection to a method that I have
> to call with what looks like a set of positional arguments in an arbitrary
> order.  Could I please just have a global is_manager that I don't need to
> define?  If I set up my own set of custom permissions, I guess I'll be fine
> doing the python:context.portal_membership.checkPermission('Do something
> unusual', context).

What's wrong with:

>>> obj.checkPermission(name)

I don't think that method is on Archetypes' base class today, but it
could easily be.

> Actually, this example contains two horribles in one.  What's with the
> ${context/portal_url}?  I can never remember when I can use portal_url and
> when I can't.  Why context?  Why would portal_url depend on it?  Subsites
> don't ship with Plone out of the box.

It's a balance, isn't it. We tried the "globalize" method, but got rid
of it, because you always paid for having those globals available,
even if you didn't use them.

The problem is that you can't – in general – use
${context/portal_url}, because that method isn't on an Archetypes
object.

But it could be :-).

I guess my point is that you don't need to put these missing methods
into a separate package and call it "API" – just add it to the content
base classes (and make sure their implementation uses components to do
the heavy lifting).

\malthe

------------------------------------------------------------------------------
Try before you buy = See our experts in action!

The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!

http://p.sf.net/sfu/learndevnow-dev2

Daniel Nouri

unread,
Mar 5, 2012, 5:37:22 AM3/5/12
to Malthe Borch, plone-de...@lists.sourceforge.net
On Mon, Mar 5, 2012 at 6:31 AM, Malthe Borch <mbo...@gmail.com> wrote:
> The problem is that you can't – in general – use
> ${context/portal_url}, because that method isn't on an Archetypes
> object.

"in general"? When is that a problem? And when does putting
portal_url (or equivalent) onto ATBase solve this problem?

> But it could be :-).
>
> I guess my point is that you don't need to put these missing methods
> into a separate package and call it "API" – just add it to the content
> base classes (and make sure their implementation uses components to do
> the heavy lifting).

I thought in an earlier email you were suggesting using Acquisition
more (with portal_url being a prime example), now you're suggesting we
put portal methods into the AT base class? Seems like a bad idea.


Daniel

Malthe Borch

unread,
Mar 5, 2012, 7:56:52 AM3/5/12
to Daniel Nouri, plone-de...@lists.sourceforge.net
On 5 March 2012 11:37, Daniel Nouri <daniel...@gmail.com> wrote:
> I thought in an earlier email you were suggesting using Acquisition
> more (with portal_url being a prime example), now you're suggesting we
> put portal methods into the AT base class?  Seems like a bad idea.

You're right – I got it somewhat wrong.

What I meant to suggest is to put the necessary methods on the site
object when the site is the right context, but in the case of
``checkPermission`` it should clearly be ATBase – or some other base
class that knows about ``AccessControl``.

\malthe

Guido Stevens

unread,
Mar 5, 2012, 2:17:27 PM3/5/12
to plone-de...@lists.sourceforge.net
It seems to me that there's two different ways of framing the goals for
plone.api:

1. make it easy to do simple stuff
2. ease the learning curve

Whilst these goals are not that different, they lead to very different
design principles:

1. invisibility: hide the legacy internals

In this vision plone.api is a self-contained domain specific language
(DSL) that allows you do do common things without having to peek under
the hood.

I think the idea that you can provide a DSL sandbox without ever
exposing legacy internals has very limited reach.

2. transparency: script common best practices

In this vision plone.api *does* provide the DSL described above.
However, it *also* does this in a way that when a new developer pops up
the hood to look how this is being done, she can easily follow the code
path and learn about deeper layers of the system.

This provides a somewhat graceful learning transition from the core
plone.api API to the wider range of underlying APIs.

It is important to note that embracing transparency as a design
principle makes it possible to obtain both the goals mentioned above:
simplicity of the exposed API /and/ ease of learning through a readable
plone.api implementation that can function as example code exercising
the underlying APIs.

People who only want to use the API don't have to read the
implementation. People who want to go beyond the API can use the
implementation as a springboard.

Embracing ease of learning as a goal has implications for the way
plone.api should be implemented. In my mind, composing functionality
through the component architecture is vastly superior from a learning
perspective, to polymorphism and/or acquisition. That doesn't exclude
refactoring existing interfaces if that makes sense, but it does argue
against overloading the siteroot object or the AT base class.

Finally, choosing between DSL-only or DSL+learning goals has
implications for coding style. A preference for PEP8 implies an
assumption of a relatively isolated plone.api that is mostly consumed,
not used as a springboard. Using plone.api as a transitional environment
that facilitates learning about the deeper stack would be much smoother
with a mixedCase style, as Martin has argued.

:*CU#
--
*** Guido A.J. Stevens *** tel: +31.43.3618933 ***
*** guido....@cosent.nl *** Postbus 619 ***
*** http://www.cosent.nl *** 6200 AP Maastricht ***

s h a r i n g m a k e s s e n s e

RT @glynmoody: #ACTA - Time to Win! - http://t.co/np0rlzBc new
#avaaz epetition; time to go for 5 million?

http://twitter.com/GuidoStevens

Martin Aspeli

unread,
Mar 5, 2012, 3:23:05 PM3/5/12
to Guido Stevens, plone-de...@lists.sourceforge.net
Hi,

On 5 March 2012 19:17, Guido Stevens <guido....@cosent.net> wrote:
> It seems to me that there's two different ways of framing the goals for
> plone.api:
>
> 1. make it easy to do simple stuff
> 2. ease the learning curve
>
> Whilst these goals are not that different, they lead to very different
> design principles:
>
> 1. invisibility: hide the legacy internals
>
> In this vision plone.api is a self-contained domain specific language
> (DSL) that allows you do do common things without having to peek under
> the hood.
>
> I think the idea that you can provide a DSL sandbox without ever
> exposing legacy internals has very limited reach.

Agreed. It is the Z-shaped learning curve all over again.

> 2. transparency: script common best practices
>
> In this vision plone.api *does* provide the DSL described above.
> However, it *also* does this in a way that when a new developer pops up
> the hood to look how this is being done, she can easily follow the code
> path and learn about deeper layers of the system.
>
> This provides a somewhat graceful learning transition from the core
> plone.api API to the wider range of underlying APIs.
>
> It is important to note that embracing transparency as a design
> principle makes it possible to obtain both the goals mentioned above:
> simplicity of the exposed API /and/ ease of learning through a readable
> plone.api implementation that can function as example code exercising
> the underlying APIs.

+1

> People who only want to use the API don't have to read the
> implementation. People who want to go beyond the API can use the
> implementation as a springboard.
>
> Embracing ease of learning as a goal has implications for the way
> plone.api should be implemented. In my mind, composing functionality
> through the component architecture is vastly superior from a learning
> perspective, to polymorphism and/or acquisition. That doesn't exclude
> refactoring existing interfaces if that makes sense, but it does argue
> against overloading the siteroot object or the AT base class.

I certainly think this would make it harder to know what methods on
the "fat" interface of the site root or content base classes are
"useful" as an API. You can't see the wood for the trees, so to speak.

> Finally, choosing between DSL-only or DSL+learning goals has
> implications for coding style. A preference for PEP8 implies an
> assumption of a relatively isolated plone.api that is mostly consumed,
> not used as a springboard. Using plone.api as a transitional environment
> that facilitates learning about the deeper stack would be much smoother
> with a mixedCase style, as Martin has argued.

Well put ;-)

Martin

Gil Forcada

unread,
Mar 5, 2012, 6:37:26 PM3/5/12
to Guido Stevens, plone-de...@lists.sourceforge.net
El dl 05 de 03 de 2012 a les 20:17 +0100, en/na Guido Stevens va
escriure:

I would love to have that direction, just like the collective docs are
created, best on this one to document the best practices to do daily
Plone stuff.

Also, commenting the implementation and giving pointers all over the api
calls will make the api consumer feel more safe, as she already have the
api to do what it does, but it also comes with "if you need to do the
same but twisted see how it's implemented to to your own twist".

Cheers,

> Finally, choosing between DSL-only or DSL+learning goals has
> implications for coding style. A preference for PEP8 implies an
> assumption of a relatively isolated plone.api that is mostly consumed,
> not used as a springboard. Using plone.api as a transitional environment
> that facilitates learning about the deeper stack would be much smoother
> with a mixedCase style, as Martin has argued.
>
> :*CU#


--
Gil Forcada

[ca] guifi.net - una xarxa lliure que no para de créixer
[en] guifi.net - a non-stopping free network
bloc: http://gil.badall.net
planet: http://planet.guifi.net

David Glick

unread,
Mar 22, 2012, 1:56:17 PM3/22/12
to plone-de...@lists.sourceforge.net
On 3/21/12 8:30 PM, Ross Patterson wrote:
> David Glick<david...@groundwire.org> writes:
>
>> On 2/26/12 7:02 PM, Nejc Zupan wrote:
>> 1. Regarding the calls like get_site_url() and get_request(), it's a
>> bit misleading to say that these will *always* be available. It would
>> be good to warn that the API is intended for use within code published
>> by the ZPublisher (which sets up the necessary thread locals), as
>> opposed to external methods or code run via bin/instance run or
>> whatever. Uh...I'll let you guys translate that into
>> non-Zope-addled-brain language. :)
> Very late here and slightly OT, but I've always thought that I've seen
> very few real-world "bin/instance run" scripts that didn't do `app =
> Testing.makerequest.makerequest(app)` and/or
> `zope.component.hooks.setSite(portal)`. To that end I think we should
> just add that to `plone.recipe.zope2instance`.
>
>
Fine for makerequest(app), but how would it know which portal to set as
the site?


----------
David Glick
Web Developer
david...@groundwire.org
206.286.1235x32

The Engagement Party 2012. So much more fun than the wedding reception.

http://www.npoengagementparty.com

------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure

Sean Upton

unread,
Mar 22, 2012, 2:01:48 PM3/22/12
to Ross Patterson, plone-de...@lists.sourceforge.net
On Wed, Mar 21, 2012 at 9:30 PM, Ross Patterson <m...@rpatterson.net> wrote:
> Very late here and slightly OT, but I've always thought that I've seen
> very few real-world "bin/instance run" scripts that didn't do `app =
> Testing.makerequest.makerequest(app)` and/or
> `zope.component.hooks.setSite(portal)`.  To that end I think we should
> just add that to `plone.recipe.zope2instance`.

The other day, I was thinking something similar about setting up a
request and site manager in the context of bin/instance debug (command
option passes in the site id):

$ ./bin/instance debug mysite

This should set a request using something like Testing.makerequest
(but that request might need some marker interfaces added to it for
(a) all browser layers on the specific site, (b) z3c.form
compatibility.

...the only unintended consequence I could think of is folks
programmatically traversing to views that store annotations on the
request object -- it should be easy (and documented) way to get a
fresh request multiple times in a run script or debug session (and not
have to manually mark the layer and other interfaces necessary on the
fresh request).

Sean

Ross Patterson

unread,
Mar 22, 2012, 4:23:14 PM3/22/12
to plone-de...@lists.sourceforge.net
Fulvio Casali <ful...@solitonconsulting.com>
writes:

> ...I think we should not confuse the momentum behind plone.api with
> wanting to create a "great" API...
>
> ...So, can we start a collection, a little gallery of horrors?...
>
> ...I don't have a strong preference on how this little gallery of


> horrors should be implemented.  Sphinx might work.  Google moderator,
> maybe.  [I'm a fan of wikis, I like how in MediaWiki (e.g. wikipedia)
> there is a separation between the content and the discussion about the
> content (they are on different tabs), and yet there is no barrier to
> either editing the content, or adding to the discussion, and full

> history is preserved (again with no barrier, no context switch).]...

+10, makes a lot of sense to me as a way to define the scope and limit
stop-energy.

Ross


------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure

Ross Patterson

unread,
Mar 22, 2012, 4:29:12 PM3/22/12
to plone-de...@lists.sourceforge.net
Daniel Nouri <daniel...@gmail.com>
writes:

> On Wed, Feb 29, 2012 at 4:22 PM, Wichert Akkerman
> <wic...@wiggy.net> wrote:
>> On 02/29/2012 04:16 PM, Daniel Nouri wrote:
>>>
>>> On Wed, Feb 29, 2012 at 3:40 PM, Martin
>>> Aspeli<optilud...@gmail.com>
>>>  wrote:
>>>>

>>>   user.password = 'foo'
>>>   user.email = 'foo'
>>>   user.thatattribute = 123
>>
>>
>> Now imagine someone has an attribute that is not a legal identifier in
>> python in their LDAP/SQL/whatever system. What do you do now?
>
> Easy: getattr.

Just to be clear on this minor point, lets stay Pythonic with this kind
of stuff. Attributes should be more (searching for the right term)
"structural" or related to the code. When it's arbitrary properties, we
should definitely use the more Pythonic dictionary-like, key-value
access:

>>> user['password'] = 'foo'

Or maybe to be more explicit:

>>> user.properties['password'] = 'foo'

Ross


------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure

Sean Upton

unread,
Mar 22, 2012, 6:08:46 PM3/22/12
to Ross Patterson, plone-de...@lists.sourceforge.net
On Thu, Mar 22, 2012 at 3:04 PM, Ross Patterson <m...@rpatterson.net> wrote:
> On further thought, I think they should also do:
>
>    >>> newSecurityManger(None, SpecialUsers.system)
>
> Should this be a default, or only when an option is passed.

+1 to default, even better is ability to override the location of the
user-folder and username when a (generic) system user isn't
appropriate to a run-script.

>>   $ ./bin/instance debug mysite
>
> How about adding a `--portal-path` option to `bin/instance debug` and
> `bin/instance run` will add a `portal` name in addition to the `app`
> name with `zope.component.hooks.setSite(portal)` already run?

+1

>> This should set a request using something like Testing.makerequest
>> (but that request might need some marker interfaces added to it for
>> (a) all browser layers on the specific site, (b) z3c.form
>> compatibility.
>>
>> ...the only unintended consequence I could think of is folks
>> programmatically traversing to views that store annotations on the
>> request object -- it should be easy (and documented) way to get a
>> fresh request multiple times in a run script or debug session (and not
>> have to manually mark the layer and other interfaces necessary on the
>> fresh request).
>

> This would be a good caveat to document but I don't think it's necessary
> to solve this since `debug` and `run` already don't do anything about
> this.  Is this sufficient, with descriptive text:
>
>    >>> from Acquisition import aq_base
>    >>> from Testing import makerequest
>    >>> app = makerequest(aq_base(app))
>    >>> portal = app.unrestrictedTraverse(portal.getPhysicalPath())

Yeah, this seems reasonable.

What I think I want overall for (initial) common scaffolding for debug
sessions and run scripts looks something like
http://pastie.org/3650726

Reply all
Reply to author
Forward
0 new messages