Using a normalized database structure instead of json for assets

[Nils Rückmann]

Hi, is there any reason why we are not using a normalized database structure to store our asset rules ? Nobody has coded it yet is no reason ;)

I was thinking about it, because today it isn't possible to use our rules in sql statements, which is why we need an extra "access" column.
With a normalized structure we could avoid that column and archieve a new flexibility.

[Nils Rückmann]

Nobody ?

Are there any downsides of using a normalized structure instead of the json stuff i do not mentioned ?

[Herman Peeren]

For some more inspiration about the subject, you could have a look at the code and experiments of Klas Berliç from one-and-a-half years ago, when he tried to make improvements to Joomla's ACL (a.o. putting the rules in a db-table instead of in JSON). See: http://forkito.org/

My opinion as answer to your question:it is easier to just use Joomla's ACL (as bad) as it is now. Trying to improve that might be less efficient than rebuilding it completely. I personally see the current ACL as temporary and a good opportunity to learn from our mistakes.

- Herman

[Herman Peeren]

On Thursday, 25 October 2012 19:30:26 UTC+2, Nils Rückmann wrote:

Are there any downsides of using a normalized structure instead of the json stuff i do not mentioned ?

In the answers to the questions after his jab11-presentation Klas estimated a 10% decrease of performance using table joins instead of JSON, based on his experiments. On the other hand, he guessed the view-action was implemented seperately in View Access Levels because they would be inefficient if also done via JSON.

[Nils Rückmann]

Thanks Herman, at least i'm not the only one who is not lucky with the current acl.

Of course, the perfomance will decrease, but thats why we have caches ;)

[Andrew Eddie]

There was no "mistake" in the design. It was a deliberate decision to trade off complexity, maintainability and performance. Yes, it's an anti pattern and they have the reputation for causing more harm than good, but in this case it was a useful trade off to achieve a result after many, many iterations of trying different ways (remember, the original phpgacl was fully normalised - ever tried to debug those tables?). In addition, I believe no less than four different strategies were presented and discussed by the community at the time. The current system was selected as the best fit at the time and I still believe its the best fit for the CMS architecture as it stands today. 

I wasn't a big fan of Klas's approach because it meant that the poor old component developer had to set up too much data in the database before anything would work.

However, i do agree the access package is propably not the best way forward in terms of designing a new breed of platform apps. For what it's worth I have since worked on an Authorisation package that decouples the storage of the "rules" from testing the rules so if you want you can go back to storing complex access control rules in 3rd normal form if you really want to (but you do that at the very high cost of maintainability) or your could plug into some proprietary system. Hopefully I'll get a chance to push that out sometime soon and we can put the access package int he legacy tree. 

Regards,

Andrew Eddie 

--
Regards,
Andrew Eddie
http://learn.theartofjoomla.com - training videos for Joomla developers

[Elin Waring]

Andrew,

Is that a newer version of what it in the content branch? No one ever answered my question about whether you could use a custom authoriser to handle edit.own. Is that going to be possible?

Elin

[Nils Rückmann]

Thanks for your deep insights Andrew, i wasn't there when it was discussed. But i have to disagree with you that the current stand is the best choice for the cms (at least for the future), if it wants to be more than a playground for small sites. In this case i have another approach, which perhaps should be discussed seperatly: wait a minute ;)

[Andrew Eddie]

Oh that's right, it's in there (forgot).

The package is agnostic to what the permission keys are so asking
whether "edit.own" is possible carrying the meaning that the CMS
assigns to it isn't the right question to ask - it's how you design
the permissions. Edit own really translates to "the owner of the
content is explicitly given permission to edit" so when you save the
rules, you should assign an edit permission to that user and then you
just test for "edit". Or, you can write your own authoriser to take
account of ownership, or you can implement "edit.own" as a permission
and test against the owner id in the application.

Regards,
Andrew Eddie

[Elin Waring]

Yeah I wasn't asking in the current CMS sense (which is a kludge) but for the more general problem of saying you want to give the creator certain rights outside of whatever the basic permissions implementation you have is in the same way that root is.

Elin

[Andrew Eddie]

I did say for the state of the architecture of the CMS as it currently
stands. If you want to design Joomla-NextGen or something, that's a
different discussion. The problem with "large" sites (I'm not 100%
sure what you define as a small site) and let's define that as a site
with many thousands of user groups is that the CMS UI is not designed
to scale beyond a few hundred (because of the parent_id SELECT list).
That actually has nothing to do with the capabilities of the access
control system until you get into the territory of many thousands of
connections (tens of thousands of users mapping to tens of thousands
of user groups mapping to many hundreds of permissions). The point
there is you wouldn't be casually browsing Joomla.org for the
out-of-the-box download button with that scale of site. You are going
to use it as a platform or, better still, you are going to build
something from the ground up on our Platform. The ACL in the CMS is
"ok" for the market that the CMS is sort of into (in fact, it's
probably too complicated for the average web site and needs to be
scaled back).

Anyway, regardless of what the CMS is doing, I think we all agree that
beyond-the-CMS needs more thought. Here's my proposition:

https://github.com/eBaySF/joomla-platform/tree/content/libraries/joomla/authorisation

It needs a bit of cleaning up (the /meta folder can go now) but I'm
pretty happy with the concept. The main thing to look at is in

https://github.com/eBaySF/joomla-platform/tree/content/libraries/joomla/authorisation/authoriser

You can see there that I've design the way the CMS handles regular
permissions and a "root" user by composition (not some crazy
procedural business rule - if there's any mistake in the current
system, that's a big one I did make). For the root user,
JAuthorisationAuthoriserRoot->isAllowed will always return true, and
the application would inject this authoriser into the application from
the start. JAuthorisationAuthoriserDefault emulates the deny-wins
rule that the CMS works on. Finally, you've use the factory to get an
authoriser and then use JAuthorisationAuthoriser->setRules to inject
whatever you want from any source (the application does this).

Regards,
Andrew Eddie

[Andrew Eddie]

On 26 October 2012 09:09, Elin Waring <elin....@gmail.com> wrote:
> Yeah I wasn't asking in the current CMS sense (which is a kludge) but for
> the more general problem of saying you want to give the creator certain
> rights outside of whatever the basic permissions implementation you have is
> in the same way that root is.

The authorisation package assumes you have already worked out how to
do that. The application would inject (setRules) your base ruleset,
and then it would cascade down whatever content is being looked at.

So, I said something wrong. The authorisation package doesn't connect
to other data sources, the application does that (which is still an
improvement over the access package due to the decoupling). What it
does is allow you to inject different ways of interpreting the rules
as they cascade (if you don't like deny-always-wins, you can roll your
own authoriser to take into account quirky business rules). Well,
actually, I guess you could bypass setRules and let the adapter lazy
load a ruleset from a data source.

Anyway, the next step would be to build a Rule Factory and/or a Rule
Mapper for reading and writing and I think that's what Nils is
interested in (and so am I for that matter) - I'm still a fan of a
JSON hybrid solution for that but you'd build it to handle any data
source. That's where you could start crafting ways of handling
edit-own transparently.

Regards,
Andrew Eddie

[Nils Rückmann]

Thanks Andrew, it looks really structured ;) I'm kidding, it's cool to decouple the authoriser. But like you said, it's not my point.
The only thing i really don't like is that we haven't a central way to handle all permissions. Maybe that's something which confuses the users too.
The permission to access something and to edit something should be stored in the same way, and the only way to do this is by using a normalised structure, or i simply do not know another solution.

I don't know where you see the market of joomla, but i think the cms should be scalable from tiny little landing pages till med-large corporate sites or even smaller communities (i can only talk about my experience and my biggest community project with joomla has round about 15k active users).
Perhaps it is my mistake, but i see Joomla as a cms with high usability (about the new backend interface can be debated), but also with the ability to do something great.

[Andrew Eddie]

On 26 October 2012 09:43, Nils Rückmann <syb...@googlemail.com> wrote:
> Thanks Andrew, it looks really structured ;) I'm kidding, it's cool to
> decouple the authoriser. But like you said, it's not my point.
> The only thing i really don't like is that we haven't a central way to
> handle all permissions. Maybe that's something which confuses the users too.
> The permission to access something and to edit something should be stored in
> the same way, and the only way to do this is by using a normalised
> structure, or i simply do not know another solution.

That's a UI decision. There is absolutely no reason why you can't
present the UI that combines action permissions with viewing
permissions. The reason it was done that way in 1.6 is to ease people
into the new permission system, but still having a familiar setting
for adjusting the viewing access (which they had been doing for many
years).

However, under the hood though, bringing view permissions into the
data structure of the action permissions would be a nightmare. The
two are separated in the code because the most efficient way to handle
"what can I do" is the most inefficient way to handle "what can I see"
and visa-versa.

> I don't know where you see the market of joomla, but i think the cms should
> be scalable from tiny little landing pages till med-large corporate sites or
> even smaller communities (i can only talk about my experience and my biggest
> community project with joomla has round about 15k active users).
> Perhaps it is my mistake, but i see Joomla as a cms with high usability
> (about the new backend interface can be debated), but also with the ability
> to do something great.

Joomla "as a CMS" and the architecture of "the CMS" are two different
things. The architecture of "the CMS" (versions 1, 2 and 3) have
limitation because you are trying for a line of best fit. Enabling
people to go beyond "the CMS" by using the Platform to build smarter
and better mousetraps is definitely where the action is going to be in
the future. To be honest, I would be so bold as to suggest web
services for me are the "true CMS" (a system that manages content
regardless of the device connecting to it). Probably a controversial
statement :)

[Herman Peeren]

On Thursday, 25 October 2012, Herman Peeren wrote:

(...) I personally see the current ACL as temporary and a good opportunity to learn from our mistakes.

 

On Thursday, 25 October 2012 23:26:31 UTC+2, Andrew Eddie wrote:

There was no "mistake" in the design. (...)

 
With "mistakes" I mainly meant JAccess (with it's static methods and datacentric/procedural approach instead of proper OOP), the way it is implemented in the CMS (scattered over a myriad of files and thereby very inflexible), the use of the asset_id outside the access-package itself (the asset name and asset_id both point to the same asset; the asset_id should only be used internally), the separate ACL for view access (if the implementation needs a different approach doesn't imply the interface should be separate), the 3rd access control by needing a login to a separate administrator application (5 years ago I was using DNN just at the moment they had implemented an ACL and therefore got rid of their seperate administrator login) and the implementation of a "deny always wins" (when someone is added to an extra, not hierachical related user group, she should not loose access rights). Storing a blob of rules in JSON was just a minor concern of me (I even learned to see some advantages of it in some cases)

I don't mean it as critique, but just to sum up some points I want to learn from. I understand the difficulty you and other developers had in that time when many wanted to keep everything in Joomla! the same as much as possible. And also: access control is a very difficult subject, for it has impact on the code in many places, yet you want to seperate all concerns as much as possible.

Your authorisation package proposal certainly is a step in the good direction. I smiled when I saw it for the first time (with Louis' first UCM-code proposal); also because for me it confirmed the current ACL implementation needed some improvement. Thanks for spending time on it.

[Elin Waring]

Deny does not win when you are added to a group which does not have access  or permissions if you are in  another group with access or permissions. Deny always wins is only about inheritance on the group branch.

Elin

[Herman Peeren]

On Friday, 26 October 2012 16:57:34 UTC+2, Elin Waring wrote:

Deny does not win when you are added to a group which does not have access  or permissions if you are in  another group with access or permissions. Deny always wins is only about inheritance on the group branch.

Thank you for the correction: that is as it should be. I didn't test it (sorry). I had just looked at the code https://github.com/joomla/joomla-cms/blob/master/libraries/joomla/access/rule.php#L63 and following, mergIdentities() and mergIdentity() and there I see that for every identity of that user an explicit deny always wins. Did I misinterpret that code?

The same for https://github.com/eBaySF/joomla-platform/blob/content/libraries/joomla/authorisation/authoriser/default.php#L29 with isAllowed(). Maybe I should test first instead of jumping to conclusions from reading code. Still curious what I misinterpreted there; I'll figure it out after testing.

[Herman Peeren]

Maybe ther was some confusion when I said a "deny always wins" it was not clear I meant "an explicit deny always wins".
Another typo (maybe due to not being a native english speaker, if that is an excuse): I meant "lose" where I wrote "loose".

The ACL is a topic where you have to be very precise...

[Andrew Eddie]

On Friday, October 26, 2012, Herman Peeren wrote:

On Thursday, 25 October 2012, Herman Peeren wrote:

(...) I personally see the current ACL as temporary and a good opportunity to learn from our mistakes.

 

On Thursday, 25 October 2012 23:26:31 UTC+2, Andrew Eddie wrote:

There was no "mistake" in the design. (...)

 
With "mistakes" I mainly meant JAccess (with it's static methods and datacentric/procedural approach instead of proper OOP), the way it is implemented in the CMS (scattered over a myriad of files and thereby very inflexible), 

Ah, we'll there I'd agree with you. 

 

the use of the asset_id outside the access-package itself (the asset name and asset_id both point to the same asset; the asset_id should only be used internally), t

As explained many, many times, the best way to do what-can-you-see queries if different from a pure authorisation challenge. If you've found an efficient and easy way to do both the same way I'd love to hear it. 

 I don't see any real problem with that. It's not uncommon to use numerical foreign keys but also use string values for looking up.

he separate ACL for view access (if the implementation needs a different approach doesn't imply the interface should be separate),

 

the 3rd access control by needing a login to a separate administrator application (5 years ago I was using DNN just at the moment they had implemented an ACL and therefore got rid of their seperate administrator login)

What's a DNN?

 

 and the implementation of a "deny always wins" (when someone is added to an extra, not hierachical related user group, she should not loose access rights). 

That's how file systems work, as was explained at length at the time. 

 

Storing a blob of rules in JSON was just a minor concern of me (I even learned to see some advantages of it in some cases)

I don't mean it as critique, but just to sum up some points I want to learn from. I understand the difficulty you and other developers had in that time when many wanted to keep everything in Joomla! the same as much as possible. And also: access control is a very difficult subject, for it has impact on the code in many places, yet you want to seperate all concerns as much as possible.

As I said, the design was about trade offs and I think you are confusing personal preference with actual design flaws. I agree that the access package is, looking at it with older and wiser eyes, poor, but there were reasons for doing it that way, mostly centred around taking pity on component developers that wanted to opt into it. That's why we used static methods. If I was to do that today, I'd use factory patterns but I think you could appreciate that would have scared a lot of people off back in to change over to 1.6. 

 

Your authorisation package proposal certainly is a step in the good direction. I smiled when I saw it for the first time (with Louis' first UCM-code proposal); also because for me it confirmed the current ACL implementation needed some improvement. Thanks for spending time on it.

You're welcome. With the history of what we did in the access package, I'd certainly love to discuss a better way to store the rules. 

Regards

Andrew Eddie 

[Herman Peeren]

On Friday, 26 October 2012 22:21:57 UTC+2, Andrew Eddie wrote:

What's a DNN?

Oh, sorry: DNN = DotNetNuke, an open source (MIT licensed) CMS under ASP.NET. http://www.dotnetnuke.com/

I come from a mainframe and later desktop programming (Delphi) background. My webprogramming experience started with (classical) ASP. Later  ASP.NET (and using .NET for desktop programming too). I made my first self rolled CMS in ASP.NET, my second in PHP. In that time I switched to open source, first DNN, later Joomla!. That is about 3 and a half years ago. So, compared to the Mambo-veterans I'm quite new in Joomla!.

I didn't say view access control and other action access control should be implemented in the same way, but that I think they can have a similar interface. I'm experimenting (using Doctrine2 as ORM) and hope to show some outcome of that soon.

[klas berlič]

Hi,

just few comments:

"It was a deliberate decision to trade off complexity, maintainability and performance. "

JSON means current implementation is more complex (2 ACLs says enough), less maintainable (more API, 2 ACLs) and its performance is _____ (insert bad word here) - checking permissions using php is simply a magnitude slower that when doing it using relational database.

Please note performance claim applies to the real ACL part, not to its view surrogate part that was invented just because real ACL was so badly performing.

 "I wasn't a big fan of Klas's approach because it meant that the poor old component developer had to set up too much data in the database before anything would work."

Not really true, perhaps you got it confused with some other proposal.

There are zero database actions needed on the developer side, all permissions are stored in the permissions table and everything works trough API.

Even Joomla database structure would only need to be minimally changed and even more - all drop parts can be skipped for first version to maintain backwards compatibility :

• drop JSON rules in the assets table
  
• drop view permission columns
• drop viewlevels table
  
• add permissions table
• add asset_id columns to extensions,modules,menus
  

What I'm proposing with my (working) proof of concept is to drastically reduce API and simplify things to the point where it can't be simplified anymore. The whole API relevant to developer consists of 2 functions:

• one to check permissions for a single action (and this includes view action)
  isUserAuthorizedTo($userId, $action, $asset)
• and another one to auto insert query into multiple items queries
  insertFilterQuery($query, $joinfield, $permission);

so instead of having
->where('access IN ('.$levels.')')
in query, you would just do

MAccess::insertFilterQuery($query, 'e.asset_id', 'core.view');

And this is the really simplest comparison example on the joomla side, try checking multiple items for edit permission in single query in Joomla if you can.

As Herman mentioned performance - it wasn't just a guess, I did some test and (view) performance degradation was 10% and with further optimization it could be improved further, while performance on the other actions is highly increased, but I have no actual numbers (but again, try checking edit permission on 1000 items via joomla vs. simple query).

Latest version I have (not public, but I can make it so if there is interest) is compatible with Joomla 1.7., it would be fairly easy to update it to the 2.5, for 3.0 widget javascript would need some more work.


If anyone is interested take your time and check the video and slides from jab11 ( http://forkito.org ) 

Not that I expect this to happen as one or two people in charge would have to admit their mistakes, but ->
 
If either platform or CMS is seriously interested I can fairly quickly update and contribute this code for one of the next framework/CMS releases.

regards,
Klas

2012/10/25 Andrew Eddie <mamb...@gmail.com>

[Andrew Eddie]

On 27 October 2012 06:53, klas berlič <klas....@gmail.com> wrote:
> JSON means current implementation is more complex (2 ACLs says enough), less
> maintainable (more API, 2 ACLs) and its performance is _____ (insert bad
> word here) - checking permissions using php is simply a magnitude slower
> that when doing it using relational database.

I beg to differ, and so do a lot of people trying to solve the same problem.

> Please note performance claim applies to the real ACL part, not to its view
> surrogate part that was invented just because real ACL was so badly
> performing.

Bollocks. Talk to anyone who has design access control and they'll
tell you that the "challenge" vs "give me a list of what I'm allowed
to do" and they'll tell you they are two completely different
strategies. The are all under the umbrella of access control ("real
ACL" and "surrogate" are just things you are making up).

> "I wasn't a big fan of Klas's approach because it meant that the poor old
> component developer had to set up too much data in the database before
> anything would work."
>
> Not really true,

It is true that I'm not a big fan of your proposal :P

> perhaps you got it confused with some other proposal.
>
> There are zero database actions needed on the developer side, all
> permissions are stored in the permissions table and everything works trough
> API.

I'm struggling to work out how your system is designed without
database actions. Maybe you can explain how that works?

> Even Joomla database structure would only need to be minimally changed and
> even more - all drop parts can be skipped for first version to maintain
> backwards compatibility :
>
> drop JSON rules in the assets table
> drop view permission columns
> drop viewlevels table
> add permissions table
> add asset_id columns to extensions,modules,menus
>
> What I'm proposing with my (working) proof of concept is to drastically
> reduce API and simplify things to the point where it can't be simplified
> anymore.

phpGACL actually does that (but not as a flat matrix). But it's a
"challenge" system. It's not good at querying a list of what you can
do (even Mike agrees with that, and acknowledges that it's a difficult
problem to solve).

> The whole API relevant to developer consists of 2 functions:
>
> one to check permissions for a single action (and this includes view action)
> isUserAuthorizedTo($userId, $action, $asset)

That's the "challenge" - easy peasy and not overly expensive in either
JSON or 3rd normal form (inheriting in 3rd normal form though, that
gets fun).

> and another one to auto insert query into multiple items queries
> insertFilterQuery($query, $joinfield, $permission);

More or less the same "strategy" we are using now.

> so instead of having
> ->where('access IN ('.$levels.')')
> in query, you would just do
>
> MAccess::insertFilterQuery($query, 'e.asset_id', 'core.view');

Right, there's nothing wrong with that function, but the devil is in
the detail in what it's doing to the query. Without those details
it's hard to judge one way or the other.

> And this is the really simplest comparison example on the joomla side, try
> checking multiple items for edit permission in single query in Joomla if you
> can.

It would be a trivial exercise to wrap a method to be able to do that.
Whether it's efficient or not is another matter.

> As Herman mentioned performance - it wasn't just a guess, I did some test
> and (view) performance degradation was 10% and with further optimization it
> could be improved further, while performance on the other actions is highly
> increased, but I have no actual numbers (but again, try checking edit
> permission on 1000 items via joomla vs. simple query).

So, the thing is, I'm personally not too interested in saying there is
only one way to store your "rules" - I don't care if you want to use a
CSV, phpGACL, a flat matrix, Active Directory or LDAP. That's the
point of the authorisation package. It's there to establish an
interface for access control challenges and "you" work out how the
rules work. What it probably lacks is an interface to get a list of
things you can do - I'm enjoy talking about how to define that
interface.

Whatever the case, I would try to avoid a use case where I had to
check 1000 items against an "edit" permission. As to whether the
query is simple or not, again, I'd have to see how many joins you are
doing and how well you've indexed the data.

> Latest version I have (not public, but I can make it so if there is
> interest) is compatible with Joomla 1.7., it would be fairly easy to update
> it to the 2.5, for 3.0 widget javascript would need some more work.

Joomla CMS version X is out of the scope of this discussion. However,
I am interested in generic strategies. If a new strategy proves
useful, any Platform Application (include the Joomla CMS) can
potentially use it.

> If anyone is interested take your time and check the video and slides from
> jab11 ( http://forkito.org )

Ignoring the obvious antagonism (one hopes for theatric effect):

"Simple matrix of groups and actions" - there are numerous articles
about why matrix style access control is severely limiting. The
winnowing of those matrices is really not easy and inherently
restrictive (we don't do it that way any more). Mike's documentation
on phpGACL covers matrix theory pretty well, including its limits.

"Lower level always wins" - well, whatever. That's not the "right"
way, just a "different way" (doesn't mean the ACL is poor or
excellent). That's why I set up the authorisers in the authorisation
package so you could change the rules depending on your use case. The
same goes for "assigned beats inherited". It's not necessarily
better, though it may be in some cases, it's just different. The CMS
access control was based on folder permissions - something developers
would be familiar with (again, not necessarily "better", just
"familiar"). I will say though, that your rules are suited to the
limitations of matrix style access control, and there's nothing wrong
with that.

"No code repetition" - hard to know what you meant in the absence of
slide notes.

You don't like JSON encoding. So what? Who are you to judge it's a
crime :P It does disable database relations on challenges, but not
view access. However, you actually do something naughty here (slide
27) and equivocate "viewing" with "challenging". You wouldn't do
"view" in JSON form and there were never any plans to do so. Implying
that Joomla uses JSON to check the "view" permissions was rather
dishonest. Same with slide 28 - I absolutely agree with your
statements, that's we we didn't design it that way (but you imply we
did).

So, ok, we get to code examples in slide 35, 36 et al but all you show
is your API to use. There's no data theory to back up how you've
designed the system (schema, queries, etc). I'd be happy to compare
the current access package with yours if that information was
available.

> Not that I expect this to happen as one or two people in charge would have
> to admit their mistakes, but ->

If you can give me chapter and verse as to where it established that
your way is the only right way, I will happily concede ;) (said with
ample tongue in cheek) Let's be honest Klas, we are really talking
about different strategic approaches to a very broad, and common,
engineering problem and which one you prefer. There is no single
right way of achieving an acceptable result. I accept and honour the
fact you don't like the way it was implemented, and interpret what is
needed differently.

> If either platform or CMS is seriously interested I can fairly quickly
> update and contribute this code for one of the next framework/CMS releases.

Now we are talking. How about you first outline the strategy involved
(don't just tell us to replace this API with that - tell us how your
API works under the hood). I'd certainly be interested in unpacking
what you've done at the technical level, but you need to be the story
teller :)

Regards,
Andrew Eddie

[Andrew Eddie]

On 27 October 2012 07:03, Herman Peeren <herman...@gmail.com> wrote:
> On Friday, 26 October 2012 22:21:57 UTC+2, Andrew Eddie wrote:
>>
>> What's a DNN?
>
> Oh, sorry: DNN = DotNetNuke, an open source (MIT licensed) CMS under
> ASP.NET. http://www.dotnetnuke.com/

Of course. I should have known that :) Yeah, things would be simpler
if we didn't separate frontend from backend.

Regards,
Andrew Eddie
http://learn.theartofjoomla.com - training videos for Joomla developers

>

[Herman Peeren]

Hey, wow, some interesting things in this discussion. I had missed Klas' posting in my mail (probably my spam filter is too severe) and I thought he had given up this whole ACL discussion more than a year ago. But if we could all leave the past in the past, put aside that some of us are not best friends with some others and move forward together, I have good hopes we can get some good ideas to improve Joomla!'s ACL. I think it is very good what Andrew says: "There is no single right way of achieving an acceptable result".

I think it is good to make a clear distinction between interface and implementation in all discussions. I see some arguments starting from implementation details (JSON of not; performance issues) that have implications for the interface (view-actions as part of the ACL). It probably should be the other way around.

Some loose ends laying on my desk. More questions and remarks than solutions.

interface

• if someone has edit-access (s)he should also have view-access. So there can be a relation between the actions.
  
• in a hierarchical relation an explicit deny should always win, but not on separate branches of a tree. I'm playing with this in an AND (= parent, hierarchical) and OR (= sibling, branch) relation. There are some people who have worked on a mathematical ACL-notation to make this more clear (I'll look up the links), but much of that still has to be worked out.
• working on a better GUI I have resources (assets) in a "possibility space" of users. An explicit deny puts the resources out of the possibility space. That is a different thing to do than allowing actions: that is a connection between users and resources (only possible in the possibility space). Hope you are not anoyed by me just scratching the surface of this whole thing I'm still developing. I hope it stimulates new ideas just mentioning it.
  

implementation

• hierarchical relations are now implemented with nested sets in Joomla!. I see some transitive closure implementations at other frameworks instead (with performance arguments a.o.). In this Tree-implementation (using Doctrine2) you can choose between several implementations: https://github.com/l3pp4rd/DoctrineExtensions
• I also see some interesting things happening in the graph-implementation in http://cmf.symfony.com/ there is quite some theory behind it. Sorry I'm so vague, but it is really worth looking around a bit more.
  

Oh dear, dinner is ready... posting is not finished at all, but I'll push the button now and continue later.

More ideas later.

Herman

[Herman Peeren]

Some more loose ends from my desk about ACL:

interface

• MVC and ACL: we should avoid making all kinds of adjustments in our components. My ideal for ACL with MVC is: the model only works with the resources we have access to. No access = non-existent. The view should not have any access control: it is just a view of the data that are provided by the model.
• our ACL is about the relations between two hierarchies (trees): users/usergroups and resources (assets).
• All authorization should be in the Access Control. Login should only be authentication, thereby providing the user (or more general: the "principal") that is used for Access Control. Therefore we don't need 2 logins for an administrator application and a site application.
• An ACL (for instance who is allowed to do CRUD-actions) and the permission to manage that ACL are two different layers. Should those layers be discernable in the interface? What confusion comes from mingling those layers? It would be useful if usergroups can get the possibility to manage a certain possibility space; then a head of a department can manage her department's ACL, but not that of another department.
• if we don't think in data (like an Access Control LIST) but in objects, what kind of object is an access controller? It is like a "bouncer" at the door, who tells you to come in or go away. A filter. You have to go through it to come in. When retrieving data in a model to display in a view it could be the interface: you ask the access-object to ask the model to get or set the data.
• A tree is a DAG (directed acyclic graph) where every node has exactly one parent. Multiple inheritance can however always be splitted into several (partly identical) parallel trees. Parallel trees are an OR-relation.
• A calculus of ACL is probably based in graph-theory. A start with an ACL-calculus has been made by Abadi a.o.: http://www-sst.informatik.tu-cottbus.de/~db/doc/People/LNCS/papers/05760001.pdf , http://users.soe.ucsc.edu/~abadi/Papers/acldcc-entcs-with-additions.pdf , http://www.di.unito.it/~argo/papers/TR1272010.pdf , http://www.cs.ox.ac.uk/ralf.hinze/WG2.8/24/slides/steve.pdf

implementation

• Petri-nets (GSoC-project) and other graph-calculation can also be useful for ACL-implementation. In this EzPublish approval-workflow component there also is quite some ACL involved: http://www.mugo.ca/Blog/Complete-approval-workflow-solution-for-eZ-Publish

[Herman Peeren]

Some more practical remarks about the current ACL implementation. When you are now developing a component (in the CMS) using item level access control you'll have to do quite some things (see list below) in every component. Such repetitive tasks could be done by a framework (or "platform" or whatever you call it). That could be improved by:

• providing better defaults on which to fall back
• providing a simpler interface.

Example: when implementing item-level access control you now need to override the _getAssetParentId() method in your JTable:

• if that method checks for item-level access control by default we wouldn't have to write that same piece of code over and over again. Using my own JTable extension to provide that for my own components is only half a solution: then we would need an extension of JTable and one of JTableNested. Better just fix it in JTable (which I now do with an override of JTable...).
• I like the flexibility so that I can overwrite/extend methods like that, but in most cases a default parent asset could be provided by the system (which table does it belong to and does it have categories?).
• _getAssetParentId() provides the AssetParentId by retrieving the id of the asset's parent and you retrieve that by using the asset's parent name. If you would simply provide the asset's parent name to the access control interface, then that access control package can check the asset_id.The asset_id could be kept internally, behind the interface.

When developing a component with item-level access control you now have to

• provide settings for the component-level permissions in the config.xml (could be available by default)
• provide a list of actions for various levels (sections) in access.xml (could be available by default for the core actions)
• add an options toolbar in your view (could be available by default)
• restrict access to the component's backend in the entry file (could be available by default)
• Add an asset_id to the item's database table (we could make it so, that the asset_id is not necessary outside the access package)
• Bind and store the permissions in the assets table (could be done by default).
• Provide the GUI for editing the permissions at the item level if someone is entitled to manage that (could be done by default)
• add some language strings (some could be more general to fall back on by default).

Providing an overridable/extendible default could save a lot of work and complexity.

[Herman Peeren]

implementation
Still studying everything that has been said in this thread, for I think there are some usefull things in it.

One thing just to clear up: I assume when Andrew mentioned 3NF as opposed to JSON, he meant 1NF.  Oct. 25: "storing complex access control rules in 3rd normal form" and Oct. 27: "not overly expensive in either JSON or 3rd normal form (inheriting in 3rd normal form though, that gets fun)". For anyone wanting a simple explanation about database normalisation, here is an article I wrote 5 years ago: http://www.yepr.nl/PracticalDatabaseNormalisation.pdf

Storing a JSON blob instead of atomic values is in some ways comparable to what a document database (like MongoDb or CouchDb) does. There can certainly be cases where that works much faster. If this is the best way to store rules in our ACL-system mainly depends on how the data must be available: if you would need a list of what you can do, then it is not the best way. A way to solve that could be to use a graph to store those data (and optimising handling of that graph) or to store it redundantly (in 2 formats). As usual all methods have their advantages and disadvantages, but by quantifying the needs and costs we could find the optimal solution for us. Or implement several strategies, so the optimal way can be chosen depending on the situation.

[klas berlič]

I'm not trring to convince anyone - I just though I could offer something that is already done and could benefit all - facts (see my reply to Herman) should be good enough. Implementation and details certainly have a lot of room for improvement, but complex JSON data in relational database is not anti pattern, it is just plain wrong. To the level of basics wrong.

Anyway, this discussion is now pointless as I changed my mind after reading your reply.

2012/10/27 Andrew Eddie <mamb...@gmail.com>

[klas berlič]

10% decrease I mentioned earlier applies only to view action vs. vielewels performance where we are comparing oranges and apples ( could be workarounded by replacing viewlevels with precalculated groups that have view access on an item and refresh this whenever acl tree changes - but as caching already solves performance problem this would probably be redundant to do )

When we are talking about JSON in database vs. using normalized structure for Joomla ACL, there can be no question which one to use at all - experiments in real Joomla environment clearly show that when we are talking about single items there is not much difference (difference is smaller than error of measurement), but for multiple items - lists (e.g. blog view) normalized database works from 1000 to 100000% faster, depending on number of items and used php vs. mysql server performances.

2012/10/28 Herman Peeren <herman...@gmail.com>