A new Unified Content Model.

2,126 views
Skip to first unread message

Louis Landry

unread,
Dec 19, 2011, 6:21:36 PM12/19/11
to Joomla Platform List
We are very pleased to present our first draft of what we, at eBay, have coined the Unified Content Model, or UCM for short.  UCM is the basis for an intranet product at eBay that we call the DataHub which is currently modeled on Joomla 1.5.  It's a social analytics site that allows business, financial and data analysts a way to collaborate on our data and analytics assets.  You can sort of think of it as a GitHub for data/analytics.  The repository (https://github.com/eBaySF/joomla-platform) holds a fork of the Joomla Platform with the first step in making parts of the DataHub available to both the Joomla and wider Open Source communities.  There are many parts to what we have built which are in varying degrees of readiness to share, but we wanted to get the ball rolling with the foundational stuff with an eye to sharing all the other bits as soon as is feasible.  We'd love to know what you think and if there is interest in adding it to the core platform distribution.  With that here is a quick overview of what you will find in the content branch of our fork.

New Content Package

Under /libraries/joomla you'll find a new content package.  This package includes some new classes: JContent, JContentType, JContentHelper and JContentFactory.  JContent supports a base content object that automatically handles typical "content" data that will be familiar to Joomla and CMS users alike - things like a title, body, checkin and checkout, and so forth.  JContent is designed to be extended both in terms of API and in terms of database schema.  The base object already does this by providing related tables to capture hits and whether users "like" the content.  Any number of content "types" can be supported which use the base schema and API, or can be extended from JContent to add more API, and can be linked to one or more additional tables to provide additional "fields" for the content.

JDatabaseObject

There are a number of additions to support the content package and one of these is a new class called JDatabaseObject.  JDatabaseObject can be thought of as a re-imagined JTable class, and allows for content to be stored across multiple tables.  It's strength is that it is aware of the schema of declared tables so while you can bind as much data to the objects as you want, only the data in the underlying tables is stored (compare this to JTable where you can get into trouble if you add properties that aren't in the database table).

JCacheObject

Underpinning JDatabaseObject is JCacheObject and this provides as simple way for objects to cache instances of themselves either in memory or to a persistent store.

New Authorisation Package

The authorisation package is a reworking of Joomla's access package but done to decouple it from the CMS-ism's of that package.  This new package adds a number of interfaces to allow for better design practice.  Among these are a new concept called "authorisers" and you'll find two authorisers in the package: "default" and "root".  Currently the Joomla CMS hard codes the checks for what it calls a "root" or "super" user.  Separating the logic out into authorisation strategies allows for such checks to be done at a more abstract level, even allowing for an application to create new strategies.

Documentation

The API includes full documentation blocks and also a developers reference in docbook format for all the new code mentioned above.

Feedback

Hopefully the code will speak for itself, but there is certainly a lot to take in.  We'd appreciate feedback on the API and how we've handled various aspects of the UCM.  Following that feedback, we would like to offer this to the Joomla Platform if it's something that others will find useful.

Future Steps

Showing off code is great, but showing off how it works is even better.  We are developing a simple platform application to show off the main capabilities of the UCM architecture and that should be available in Q1 next year.  There are some coupling problems with the current platform MVC implementation which made it hard to put together a dead simple example application.  Since reworking that part of the platform is on the 12.1 roadmap we decided to put off the example application until then.  Those of you who are already building platform applications will be able to play with the UCM, and we certainly look forward to your thoughts.

Obviously we could not include everything that we've built in the DataHub all at once, but to list a few of the things we've already built on top of the UCM that we'd like to share in the near future include integrated tagging support, social groups, member profiles, activities (think activity stream), and much more.  

Cheers,
Louis

Mike Carson

unread,
Dec 19, 2011, 8:08:09 PM12/19/11
to joomla-de...@googlegroups.com, louis....@gmail.com
What is the speculation of when this will be added to the core Jplatform?

Elin Waring

unread,
Dec 19, 2011, 8:48:06 PM12/19/11
to joomla-de...@googlegroups.com, louis....@gmail.com
You're a week early for delivering new toys  :).

Elin

Jeremy Wilken

unread,
Dec 19, 2011, 9:03:26 PM12/19/11
to joomla-de...@googlegroups.com, louis....@gmail.com
Thanks Louis, it looks promising and after taking a look at the code it looks like I might have imagined an abstracted content model system might. There is a lot there as you say, but why not try to refactor one of the CMS extensions as an example?

Will there be a kind of content type manager, or will it be expected that only developers will hook into this as part of their extensions? I also wonder if the concept of 'like/unlike' really belongs in there? 

I suspect it will take all of the next 6-7 months for Joomla 3.0 and Platform 12.2 or 12.3 to fully realize this, but look forward to hearing more and seeing it in action.

Matt Thomas

unread,
Dec 19, 2011, 9:06:36 PM12/19/11
to joomla-de...@googlegroups.com, louis....@gmail.com
This video from Joomla Dev Day New York on the Unified Content Model may also be informative to some as well http://www.joomladaynyc.com/home/joomladevday-1023/81-unified-content-for-the-joomla-platform

Best,

Matt Thomas
Founder betweenbrain
Phone: 203.632.9322
Twitter: @betweenbrain

Andrew Eddie

unread,
Dec 19, 2011, 9:35:45 PM12/19/11
to joomla-de...@googlegroups.com
On 20 December 2011 11:08, Mike Carson <mi...@itdwebdesign.com> wrote:
> What is the speculation of when this will be added to the core Jplatform?

Mike, if developers are happy with the direction or various tweaks are
made to achieve that, we'd hope it would go into 12.1 (that's what
we've punted on for the @since tags anyway).

On 20 December 2011 12:03, Jeremy Wilken <gnom...@gnomeontherun.com> wrote:
> Thanks Louis, it looks promising and after taking a look at the code it
> looks like I might have imagined an abstracted content model system might.
> There is a lot there as you say, but why not try to refactor one of the CMS
> extensions as an example?

Jeremy I actually tried to make a simple, ground-up component but we
have some more work to do on mapping to custom tables names. The
current API is hardcoded to use #__content which, of course, is
already taken in the CMS. And, as Louis mentioned, there are some
roadblocks in the platform MVC at the moment that we need to sort out
to take it to the next level (the tight coupling to the CMS is a
problem).

> Will there be a kind of content type manager, or will it be expected that
> only developers will hook into this as part of their extensions?

You can imagine UCM as the API or engine for how an extension
developer would build a CCK, where content type managers, field
manipulation and all that jazz is definitely application or extension
specific. So, just hypothetically, K2 and Zoo could switch to using
UCM to power their content, but could implement totally different
strategies in how the backend (or even frontend) user might interact
with it. And obviously there's the whole new world of how standalone
platform apps would interact with it.

> I also
> wonder if the concept of 'like/unlike' really belongs in there?

It makes sense in terms of the other social features we've already
implemented at work, but aren't ready for release in this cut.
Probably best to look at it as the first integrated social feature.

> I suspect it will take all of the next 6-7 months for Joomla 3.0 and
> Platform 12.2 or 12.3 to fully realize this, but look forward to hearing
> more and seeing it in action.

That's what we'd expect too if the consensus is this is a good way forward :)

Regards,
Andrew Eddie
http://learn.theartofjoomla.com - training videos for Joomla 1.7 developers

Rafael Corral

unread,
Dec 20, 2011, 11:24:53 AM12/20/11
to joomla-de...@googlegroups.com, louis....@gmail.com
Louis, it looks great, good job to everyone that worked on this.
Is there a plan to add content versioning to JContent. I know this could be handled at a higher level in the stack, but it would be nice if this was handled automatically and there would be an API to revert to a previous version.

Rafael

Jeremy Wilken

unread,
Dec 20, 2011, 11:51:56 AM12/20/11
to joomla-de...@googlegroups.com, louis....@gmail.com
Another question came to mind this morning, and was wondering about how semantic data (such as Schema.org specs) would be associated with content? Obviously that can be extended, but I would think it could be important enough to incorporate as part of the package. I believe semantic data will be more and more important, so this looks like the ideal location to include it.

Louis Landry

unread,
Dec 20, 2011, 1:23:57 PM12/20/11
to joomla-de...@googlegroups.com
Hi Rafael,

You bring up a good question.  The topic of content versioning has come up quite a bit internally.  While we have a couple of use cases where it would be useful, it isn't a very important feature for us right now.  Before joining eBay it is also a topic that Rob, Andrew, Myself and many others have talkeld about for a long time.  One of the major stumbling points for implementing a sound content versioning solution has been that we did not have something like a unified content model.  The implementation would have had to been quite different for every different type of content due to the lack of uniformity in how the data is modeled, and how it is stored/retrieved.  This was certainly one of the goals behind implementing our UCM.

In the existing "content model" that Joomla uses, anything can be stored anywhere ... and all of the data are essentially on islands (or within black boxes).  There is no central place to get a unique identifier for a weblink and an article and a newsfeed for example.  There are unique identifiers for each type, but nothing that is unique for all types.  This causes problems when you want to build things that connect to all content stored in a Joomla application.  Let's take tags for example.  

You should be able to tag weblinks, articles and newsfeeds right?  Well, to actually implement that there is no single foreign key that can be used by the tags system to map tags to content.  There are a couple of ways this can be solved, but none of them are efficient and robust.  By creating a base content table which the other types inherit we create a singular unique id that can be referenced by tags, comments, ratings, other content, and also revision history.

That's a really long winded way of saying that I believe this model will make it much much easier to build a versioning system for content.  While it isn't a top priority for us, we would certainly welcome someone taking it on and would very likely help as much as time allows for.  Our intention is not to just replace things, but to work with others in the community to make the system better for everyone ... including ourselves.

@Jeremy,

The point of the UCM implementation is not to care about what content is stored within it, or how it is structured, but to provide an easy way for features to be built on top of the content, as well as a fluid way for developers to interact with it.  I've been looking at microformats and semantic structures for a long time, and I agree that it is important to be thinking and building web content with those things in mind.  I am unsure though how it would fit into what we've currently built though.  What we have as of now is very abstract... and that is on purpose.  We haven't defined even an article content type.

By definition, the semantic structures you reference will require some sort of structured storage.  That can be achieved either by normalizing those fields in a schema or by serializing that structured data into larger fields, but I don't really see a way of building that into the abstract structures that we've built.  I've always wanted to have helper methods to quickly build that semantic output given structured input.  It is one of the things I had hoped people would build into the JHtml libraries.  I'd welcome a discussion on the topic, but let's start a new thread if you want to do that.

To be clear, our team is sharing this with the platform project in hopes of people sharing our vision for how a unified content model can/will help developers build more innovative things faster.  If it does end up in the platform (and I hope that it does), that does not mean that the Joomla CMS must or will use it.  I think that would be a great idea obviously, and we would offer guidance and potentially some help to that end, but that isn't the goal of our team.  We are driving the DataHub application away from using the CMS as a foundation since there is very very little of the CMS that is useful for us.  We are doing some really cool things internally, and it is exciting to be able to start sharing some of them with the community... this is just step one.

Cheers,
Louis

garyamort

unread,
Jan 4, 2012, 12:21:17 PM1/4/12
to joomla-de...@googlegroups.com, louis....@gmail.com
Do you know which Git id from the platform that this repository is a root of?  [I'm probably missing an easy way to get it from github]..  I'm trying to compare it against the platform for differences and it's a bit of a moving target. :-)

garyamort

unread,
Jan 4, 2012, 12:34:01 PM1/4/12
to joomla-de...@googlegroups.com, louis....@gmail.com
Scratch that, checking the logs I see that you have been merging the changes periodically.  Creating a mirror of the platform repo and rolling it back to December 16th[merge 650] and the different files drops down to a manageable 18 files.

Louis Landry

unread,
Jan 5, 2012, 12:23:40 PM1/5/12
to joomla-de...@googlegroups.com
Yep, glad you figured it out.  I'll probably rebase it sometime soon just to keep it clean and up to date.

Cheers,
Louis

Elin Waring

unread,
Jan 21, 2012, 9:55:50 AM1/21/12
to joomla-de...@googlegroups.com, louis....@gmail.com
Could one of you explain a little more about the $temporary property?  Is the basic idea that there is a quick temporary save on create which lets us do things but then the flag is removed on the real/permanent save?

Elin

Elin Waring

unread,
Jan 21, 2012, 10:43:44 AM1/21/12
to joomla-de...@googlegroups.com, louis....@gmail.com
Also ... and I know I need to build instead of code browse and maybe this is just totally obvious but I just want to ask so I know how to build a simple example.

 If I did this in a hypothetical web application X, the rules are stored right there in the content item so we wouldn't need anything like the assets table that .. some other applications built on older versions of the platform, let's call that application Y, have  to hold the rules (setting aside inheritance/nesting for the moment). And presumably also if application X had the capacity to add ... things... that were built to extend it, the rules for those things could be stored right in the #__things table.  

And if Application Y wanted to move to this, in terms of content,  it would really just have to go get the rules from the assets tables and add them to a new column in the content table. And in terms of things move the rules to the things table.

 Is that about right?  


Elin

Sam Moffatt

unread,
Jan 21, 2012, 2:51:55 PM1/21/12
to joomla-de...@googlegroups.com, louis....@gmail.com

The idea is that you create temporary objects before you go in to an
edit screen to create the item. This solves the problem where you
might want to link to something you are creating on in an edit screen
however because it hasn't been saved yet it doesn't have an ID. By
giving it an ID before editing it you can then tie any of those things
to the temporary ID. Upon saving the item properly the item is not
marked as temporary any further. Upon cancelling the item is deleted
and you can code your content item to clean up any linked items. You
can also periodically scan for items that were being edited but
dropped their session to either provide some kind of recovery
functionality (perhaps you have autosave so the majority of the
content is still around) or automatically delete those items.


Sam Moffatt
http://pasamio.id.au

Elin Waring

unread,
Jan 21, 2012, 4:46:20 PM1/21/12
to joomla-de...@googlegroups.com, louis....@gmail.com
So this solves a number of practical issues, right? Like in the past in the CMS (sorry to be application specific) there were certain content plugins that you wanted to have access to when editing but you couldn't because there was no save yet. And certain other things not in plugins also where you really need an id. 

I was actually wondering autosave, but baby steps and I as thinking that what this does is potentially leave an empty checked out record marked temporary if there has never been a save, but then as you said you could run a cron job to clean that up and the temporary tag actually tells you which should get thrown out and which might just have been someone who wanted to save a seemingly empty body (I'm assuming that is the intended textarea/editarea field) but really has something else going on, like things saved in other fields.

Elin

Ofer Cohen

unread,
Jan 21, 2012, 7:10:27 PM1/21/12
to joomla-de...@googlegroups.com

Hey

What is the current status of the UCM? Do we have implementation plan to integrate it to the CMS (for 3.0)?

Best Regards,

Ofer Cohen
Joomlics Anonymous Group

Elin Waring

unread,
Jan 21, 2012, 8:15:45 PM1/21/12
to joomla-de...@googlegroups.com
Well I think the idea is people are supposed to be doing what I am which is looking at it and discussing.

The Maintainers will have to make a decision about whether it gets into the platform.
The CMS team will then have to make a decision about whether to use and if so, how. 
And then there will have to be building.

But ... I want to say personally just based on reading the code, hearing Rob's talk in New York and some playing :  I hope this gets merged ASAP and that as soon as 2.5 GA is out that we pull the updated platform including it over to the CMS repo in a 3.0  branch. 

To me it looks great, probably needs a few updates since December and some coming into practical usability the way we are now (but I don't want to compromise on anything that it offers just for the  sake of compromise or b/c with any specific application i.e. the CMS).  If I were a maintainer I'd be voting yes.

I do think there are some specifics that should be discussed more, and that is what some of the questions I am asking are about, but there are specifics of lots of classes that should probably be discussed.


Elin

Elin

Ofer Cohen

unread,
Jan 21, 2012, 8:48:42 PM1/21/12
to joomla-de...@googlegroups.com

IMHO, 3.0 should be include the UCM and multisite, cause it will make Joomla! more acceptable as enterprise product. Those 2 features are in the head of the ideas pool.

I've also hearing Rob's amazing lecture in NY and I was wonder how we didn't get the UCM till now (reminder: com_content was not changed since mambo's days).

Once the UCM would be in the platform we should call the community to help us with the UCM implementation into the CMS.

Ofer Cohen
Joomlics Anonymous Group

Rouven Weßling

unread,
Jan 21, 2012, 9:25:03 PM1/21/12
to joomla-de...@googlegroups.com
A very specific bit of feedback that got to me when I scanned trough it:

As far as I can tell the the time when something is checkout out isn't recorded. Personally I really like it when it shows me when something got checked out and since it applies to all content types I think that should be added.



Also I wonder how an implementation of categories would look like. Would we write something like JCategories that extends JContent?


I really can't wait for an example app to start playing with this :)


On 22.01.2012, at 02:15, Elin Waring wrote:

But ... I want to say personally just based on reading the code, hearing Rob's talk in New York and some playing :  I hope this gets merged ASAP and that as soon as 2.5 GA is out that we pull the updated platform including it over to the CMS repo in a 3.0  branch.

I started a branch called joomla30 in my CMS fork that I regularly update to the current platform. As time progresses I'll also add the changes required to support the new platform version (but only the very least that is needed).

Best regards
Rouven

Elin Waring

unread,
Jan 21, 2012, 11:04:15 PM1/21/12
to joomla-de...@googlegroups.com
Yeah, I noticed that we have checked out session instead of checked out time and I suppose that gives us the time potnetially or the time the session started, though I'm curious to know the thinking on that.

I also noticed that there is not a language property and it seems to me that is pretty essential and something I think most of us have assumed is a core property moving forward.

Categories from what I heard in NYC is just another content type that happens to be nested.

I'd also like to know the concept of the media field. I like it but I wonder now that we are actually using the images field in content and of course many of the other tables had an image field 

Another really pedestrian question is whether/when this table is going to replace the content table in the platform or if this is going to be called the jos_content table for real etc. 

Elin

Louis Landry

unread,
Jan 22, 2012, 12:30:40 AM1/22/12
to joomla-de...@googlegroups.com
With respect to the checked_out_time vs. checked_out_session, the thinking is as follows.  The purpose for checking things in and out is for solving concurrency problems.  It isn't OK for two clients to be editing the same item at the same time.  Doing so could mean that two clients could save the item where the second client to save never saw the version changed by the first client to save.  There are some other concurrency problem possibilities, but that is the classical example.

The reason we decided to tie the editing to the session instead of the user/time (like the current CMS does) is because that more accurately relates to the person editing the item.  When the session is expired, then the checked out lock is expired.  We can easily know whether or not an item is legitimately checked out using this information unlike now where a couple of days after someone edits and doesn't save or cancel an item it remains checked out.

The media field can be used however.  We've placed no specific meaning to it in this implementation because we didn't feel that the way we use it internally is really broad enough to be useful for the project.  Suffice to say that the current image field could be simply copied over.  

As for language, we don't have a use for that internally and figured it would be better to have that discussion within the community rather than creating an arbitrary implementation where we don't have any particular horse in the race.  It certainly would be easy to add it though, just depends on how you'd want to handle it.

- Louis

Elin Waring

unread,
Jan 22, 2012, 5:04:21 AM1/22/12
to joomla-de...@googlegroups.com, louis....@gmail.com
That makes sense about the check ins and then what that does is also help with perhaps implementing a way to clean up, maybe with a cron job similar to what Sam said about temporary. 

 I think language  is a thing we would expect to see in the core table and that it's  at least as central as a media field. I don't know how you are thinking about it but to me that core table should have all of the basics that should be (or potentially be since you wouldn't force an application to be language aware) in every content type.I would think as a platform it's important to make it just natural to build in language awareness as a baseline assumption. I don't think that is just CMS bias either, even though in the CMS every single content type table is language aware and of course internally search is language aware.  Of course externally Google is language aware and browsers are. And people are. So to me it seems to me that language belongs in the core table.  It's just a property all content has. What applications (in this case what you are building internally) do with that, if anything, is up to the application developer. 

Elin

  

Elin Waring

unread,
Jan 22, 2012, 6:36:54 AM1/22/12
to joomla-de...@googlegroups.com, louis....@gmail.com
I'm sorry to have so many questions, it's just excitable me, but two other things.

1. What's your concept of the config in that table? Is it like attribs/params in the CMS? Or is it something else? Or are you going to say it's up to the application again?
2. It seems to me also that on the web we have metadata and it matters a lot. Therefore I think that there should be a common metadata field (but not the crazy 3 that the CMS now has in each content type table). Really the JSON in the current cms metadata field works fine as far as I know.  So I'd be in favor of adding one decent sized metadata field that an application can work with as it wants (e.g. just do description as a string or use JSON to implement Dublin Core or anything in between). If more than that is needed it can go in the  type table.


Elin

Andrew Eddie

unread,
Jan 22, 2012, 7:12:29 AM1/22/12
to joomla-de...@googlegroups.com
Shooting from the hip, I would expect multi-language support to be handled as a content type with the language field in the extended table and queries adjusted slightly to account for filtering. Not all content needs to be ML but I could see, for example, a base ML content type being provided in the cms (ml content would be an extended class of jcontent). Soooo, if the cms has some sort of cck like backend editor, there would probably be a stock language field that you'd add to ml enable any sort of content type.

Regards
Andrew Eddie
--

Elin Waring

unread,
Jan 22, 2012, 7:32:59 AM1/22/12
to joomla-de...@googlegroups.com
I'm not sure I follow. First, what kind of content doesn't have language? It's not about being multilingual it's about being lingual. If we have to add it to the extended table for every single type why does that make sense?  What kind of content type are you thinking of where it would not be appropriate to have language?

It's not about filtering at all. Filtering like in the current multilngual plugin in the CMS which is essentially a multisite approach should most certainly be handled separately, but that is a highly specialized case. Many other kinds of sites have content that is in different languages and they mix it together rather than assuming that people don't ever want to see things in more than one language. 

Now, you could make the argument that by throwing meta data into the picture that we should just put language in as a type of metadata. The language of a piece of content is most certainly data about data.  But as a practical matter I would think that given its fundamental importance (more important than having a media field and equally important as a created_by_user field which you could also argue should go elsewhere since some platform applications won't even have users in that sense) it would be in the core table as its own field. Even just thinking about how search inside web applications on the platform is likely to look in the future I would think that language will be write in there with author and date. 



Elin





Elin Waring

unread,
Jan 22, 2012, 8:25:17 AM1/22/12
to joomla-de...@googlegroups.com
And I meant Right not write.

Just to be clear, I'm really saying that this is a chance to think fundamentally about what we mean by content and what the essential properties of content are (not just as in a list of object properties or database fields, but really fundamentally what those properties and fields represent) and I think that is worth a very deep and perhaps argumentative discussion. We're going to have this model for  a while, let's say at least as long as the current content model, and we should really get it right.
 
  So, for example, I'm in favor of a media field because I think content is not necessarily only words. The bias in favor of words is something we should leave behind. The visual and aural are just as important.  Some implementations may not use the body field and may only use the media field. Some will use both. That's fine.

I think this is a very good first pass at it but I think as with anything that is built first for a particular use case and then trying to become a general essential model there has to be some thinking about it. It's not a criticism in any way to point out that something that came from an internal application that maybe does only have one language and doesn't care about meta data doesn't have that. That's why many eyeballs and many perspectives and stories about many use cases are good and make better code. 

 The web has really moved far since the 1.5 content model that this is based on. Joomla has too, which is why for example the CMS incorporates meta data and language across all content types and has since 1.6. We've come to see that thinking that they only made sense for specific types of content really did not make sense. That is, that it is all content and shares essential features even while differing in details. That is why most of us want to move to a unified model because we see it as a core with variations on a theme.

 So now we have two applications, one that you've built internally where it seems language and meta data are not essential parts of content and one where the realization has been that they are. Can we put what has been learned together to make something even more powerful? I hope so.

Elin

Elin Waring

unread,
Jan 22, 2012, 5:52:55 PM1/22/12
to joomla-de...@googlegroups.com, louis....@gmail.com
So hypothetically, data might look like something along these lines for an application using this 

except the foreign keys to the user table would be working. :P 


I mean just as a basis for playing around with.

Elin

P.S. What in the world are some of those fields about ...

Andrew Eddie

unread,
Jan 22, 2012, 5:55:52 PM1/22/12
to joomla-de...@googlegroups.com
So, there's UCM and the CMS, and UCM as a package in the Platform. Is
there a content type in the CMS that makes sense without language?
Probably not. Does UCM as a package need ML support in the base
classes? I don't think so. My point is not every content type of
every platform application that will use UCM needs ML. My current
thinking is that it should be supported on a per-type basis.

Consider this. You have a content type fictitiously called ML. The
extended table would include language, ml_title and ml_body in the
most simple case. There could be "one" row in the master content
table for the content. However, with language in the extended data,
you can have many translations for the same content but still
maintaining the same content_id (translations are tightly coupled to
the content, which is not the case in the CMS at the moment). That's
a basic sketch - obviously there's a lot of devil in the detail but
that's the way I think I'd approach it.

Whatever the case, this topic is a whole sub-project on it's own and I
remember saying that we have the opportunity to start from scratch and
design ML from the ground up. I would start with trying to build on
top of UCM first and then see where bolstering of the base content
type and data is necessary. However that happens, we'll end up with
some sort of one (content_id) to many (translations) type of thing
which means, to me, a language field in the base content type is
redundant.

Regards,
Andrew Eddie

Andrew Eddie

unread,
Jan 22, 2012, 6:07:10 PM1/22/12
to joomla-de...@googlegroups.com
On 23 January 2012 08:52, Elin Waring <elin....@gmail.com> wrote:
> So hypothetically, data might look like something along these lines for an
> application using this
> https://github.com/elinw/Content--Play/blob/master/sql/mysql/sample_data_ucm.sql
>
> except the foreign keys to the user table would be working. :P
>
> I mean just as a basis for playing around with.

Sort of. There should be some foreign key constraints in there somewhere.

Just on the "rules" field, the UCM uses different names for the
actions compared to what the CMS uses. You'll notice there's a new
authorisation package in the UCM branch that deals with a slight
change in how that is handled. What I expect will happen is there
would be some sort of translation layer that could, on the fly,
convert "action" names from one system to another. This might sound
like overkill but if you also consider a custom platform app could be
getting default permissions from a LDAP server or similar, we'll
eventually need that facility anyway.

> P.S. What in the world are some of those fields about ...

Which fields?

Regards,
Andrew Eddie

Elin Waring

unread,
Jan 22, 2012, 6:39:00 PM1/22/12
to joomla-de...@googlegroups.com
Sorry, that ps about the fields was just a sigh in the direction of the CMS.

I'm not talking about building multilingual which is indeed a massive project that UCM will make practical if it is accepted. 

Hm that's an interesting way to think about language. I was thinking about that exact conversation. 
I am precisely not thinking about the CMS  at all.  I'm thinking about a very simple web application that perhaps doesn't need to extend the table in any way and what it needs.  

To me multilingual in an adjective not a noun so I don't conceive of it as a type.  Language describes content it is not content itself. I could see an application focused on one or another type of multilingual structure (and there are several)  but I don't see those applications managing many content types not just one. I see articles as a type and articles as having language and the same for weblinks, whether a whole database is in one language or in many. And just like always I think we should build things that allow for future enhancement. But who knows what people do, I will be interested to see what they come up with.

I'm not in any way talking about a property in this class, I'm talking about a field in the table. Not every field has a corresponding property in the class nor should it. The language libraries will take care of handling what is done with the field as will the actual applications. 

Aren't the names for actions derived from the application? I mean there are fundamental CRUD actions but I don't expect edit.own to be something built into the framework, for example. That doesn't make a lot of sense to me. I do really like the way the problem of the root user is being handled by the way. 


On the keys, I'm sure I'm doing something wrong but the sql file in the library has foreign keys to the user table but of course does not have a user table and I just used the existing cms one since I guess that corresponds to what is in that table in the platform, but maybe not.  The content table also references access so I just threw that and the related tables in there for kicks. Anyway now on to see if I can something render. 


Anyway the more I look, the more I see to like. :)

Elin

Rouven Weßling

unread,
Jan 22, 2012, 6:54:26 PM1/22/12
to joomla-de...@googlegroups.com
I started toying around with this and I think I found the first bug. It's happens when you use JApplication instead of JWeb (e.g. in the CMS)

I get this error:

Catchable fatal error: Argument 5 passed to JContent::__construct() must be an instance of JWeb, instance of JAdministrator given, called in /Applications/MAMP/htdocs/joomla-cms/libraries/joomla/content/factory.php on line 168 and defined in /Applications/MAMP/htdocs/joomla-cms/libraries/joomla/content/content.php on line 135

The issue appears to be that JContentFactory passes the JApplication object to the JContent constructor but it is type hinted for JWeb causing this error.

Appearently the JContent constructor is fetching the JApplication object from the factory again anyway so I suspect it shouldn't be passed to it in the first place, which means the issue is in JContentFactory.

Does JContentFactory even need to keep track of the app when we're using JApplication?

Rouven

Elin Waring

unread,
Jan 22, 2012, 9:56:27 PM1/22/12
to joomla-de...@googlegroups.com, louis....@gmail.com
Between Rouven and me we figured out that the CMS has user id of int (11) signed and you're expectedin int (10) unsigned. 

Why it's signed I don't know but now the FKs will work.

Elin

Niels Braczek

unread,
Jan 22, 2012, 11:36:55 PM1/22/12
to joomla-de...@googlegroups.com
Am 22.01.2012 11:04, schrieb Elin Waring:

> I think language is a thing we would expect to see in the core table and
> that it's at least as central as a media field. I don't know how you are
> thinking about it but to me that core table should have all of the basics
> that should be (or potentially be since you wouldn't force an application
> to be language aware) in every content type.

Translations do belong to a *separate* table, and definitely *not* to
the original record. Thus one can have several translations for the
*same* item, retaining the ID and any relation for that item without the
need to duplicate everything.

With UCM providing unique IDs through all kinds of content, only 1 extra
table for translations and 1 single join is needed to cover everything.

Regards,
Niels

--
| http://barcamp-wk.de · 1. Barcamp Westküste 2./3. März 2012 |
| http://www.bsds.de · BSDS Braczek Software- und DatenSysteme |
| Webdesign · Webhosting · e-Commerce · Joomla! Content Management |
------------------------------------------------------------------

Andrew Eddie

unread,
Jan 22, 2012, 11:40:52 PM1/22/12
to joomla-de...@googlegroups.com
That's my thinking too Niels.

Regards,
Andrew Eddie
http://learn.theartofjoomla.com - training videos for Joomla 1.7 developers

Nikolai Plath

unread,
Jan 22, 2012, 11:58:11 PM1/22/12
to joomla-de...@googlegroups.com
I believe that makes perfect sense, Hopefully this will bring us to a
real ML CMS - sooner or later ;)

Am 22.01.2012 23:40, schrieb Andrew Eddie:
> That's my thinking too Niels.
>
> Regards,
> Andrew Eddie
> http://learn.theartofjoomla.com - training videos for Joomla 1.7 developers
>
>
>
> On 23 January 2012 14:36, Niels Braczek <nbra...@bsds.de> wrote:
>> Am 22.01.2012 11:04, schrieb Elin Waring:
>>
>>> I think language is a thing we would expect to see in the core table and
>>> that it's at least as central as a media field. I don't know how you are
>>> thinking about it but to me that core table should have all of the basics
>>> that should be (or potentially be since you wouldn't force an application
>>> to be language aware) in every content type.
>> Translations do belong to a *separate* table, and definitely *not* to
>> the original record. Thus one can have several translations for the
>> *same* item, retaining the ID and any relation for that item without the
>> need to duplicate everything.
>>
>> With UCM providing unique IDs through all kinds of content, only 1 extra
>> table for translations and 1 single join is needed to cover everything.
>>
>> Regards,
>> Niels
>>
>> --

>> | http://barcamp-wk.de � 1. Barcamp Westk�ste 2./3. M�rz 2012 |

Elin Waring

unread,
Jan 23, 2012, 2:43:30 AM1/23/12
to joomla-de...@googlegroups.com, nbra...@bsds.de
No one is talking about translations though.

Elin

Elin Waring

unread,
Jan 23, 2012, 5:45:28 AM1/23/12
to joomla-de...@googlegroups.com, nbra...@bsds.de
I think that some how people have totally lost the thread raised by my original question which was not about translated content, multilingual sites or anything like that. It was about why there isn't a language field in the core content table. I'm sure that's because it is exciting to think about the possibilities that exist with UCM, but that's not the question I raised.

Here are just a couple of reasons why it would make sense.

1. It's baseline expected that the actual language of a page will be given in the header. Do we really need to build a whole separate table just to do this for a web application running sites that might just have a few pieces of content are in a language that is not the default language for the instance of the application? Do we even assume that all platform web applications have a default language? None of the example applications so far have that, even the ones that have configurations.

2. If and when we have a search package in the platform, wouldn't we routinely want to incorporate information about the user's language and the content language in order to provide the most useful results at the top of the results list? Not to mention won't we need to know if we need to deal with not having spaces between words or double byte spaces? Aren't we likely to just end up having to add language as a field later if we add search? And isn't search basically the way people interact with the web?

Those are just two of the reasons why I think it would make sense to just add a column. 

Here are two use cases where a site could have multiple languages but is not having translated content.
 
1. I run the language department in a school. We teach English, Mandarin, German, and Latin. I write a simple web application to allow my students to post essays they have written in the languages they are learning. Should I really need to create a whole new extras table just to get that one column?

2. I run an international NGO that helps victims of natural disasters find lost relatives. I never know from one day to the next if i will be dealing with an earthquake in Armenia or a tsunami in Sri Lanka. I want the people who I am serving to be able to write in their own languages and I want search engines to index my site effectively so that people searching for people impacted by a specific disaster are served as efficiently as possible which means sending out the correct information on the language of the content items. 

I'm not saying it shouldn't be a discussion (just like the need for 4 dates  or a media field or revisions or featured can be--and I'm in favor of all of them) but I'd appreciate it if this discussion could be about the question I actually raised and that what I have raised not be misrepresented (and I'm certain that was unintentional).

Thanks, and yes I know it is all because of the excitement of something potentially new and cool people are taking small, narrowly defined comments and running in all different interesting directions with them. 

Elin  

Rouven Weßling

unread,
Jan 23, 2012, 6:24:48 AM1/23/12
to joomla-de...@googlegroups.com
I'm quite far in getting com_content into the UCM (no categories, just articles) and I got a couple of points.

1. I found at least two issues because JContent is apparently written with JWeb in mind. Do we even need to track the app when using JApplication? We could just use JFactory like we do everywhere else.

2. I'm really unsure how the split model/controller should look like now. JContent does so many of the things the model used to do I wonder if it make sense to add separate Models for those extensions that use JContent.

3. In general I found it quite easy to work with JContent. I got everything but saving in com_content set up in about 4 hours with no prior knowledge of the class. On the other hand it took me about 2 hours before that to set up the tables correctly - as Elin pointed out the demo SQL doesn't quite match the CMS. (and I suck at database stuff). Overall the documentation is really great - thanks for that!

4. What is the intended process to get that stuff into the platform? Do you want to get the MVC stuff in before/at the same time or can the UCM land earlier?

5. It would be nice to get the UCM branch updated, I'd like to send back fixes for a few issues I found.

I'll probably have more later but that's it for now.

Rouven

Elin Waring

unread,
Jan 23, 2012, 10:33:51 AM1/23/12
to joomla-de...@googlegroups.com
Rouven is right, it just makes a lot of extra stuff that we've had in the CMS just drop  by the wayside. it makes the concept of different basic content extensions fall away into one extension/application with different  content types, with the core table and the the additional table specific to the type.  There's just one content application/extension with variations and that makes so many things simpler.  Yet there is no reason it can't all be transparent to legacy users if that is what is wanted. 

We can use this starting right now (even though there are things I would definitely tweak, but that is true for lots of things), from what I've been doing just I wrote up sql (not even using the api which would be the right way to do it) to migrate the sample data from the CMS 2.5 to be content types in a few hours a lot of which was spent trying to figure out the user id FK problem. When the MVC changes come that will be even better but in my opinion nothing stops us from putting this library in the platform and letting people run with it.


Can someone who has been working with this speak about permissions inheritance in this context? To me storing the rules with the item is great, and I assume since we are also storing rules with types that items would/could inherit from the type. Now that categories are just another kind of content and there is no requirement to use a category construct for an item, how are you thinking about (or actually using) inheritance that covers groups of items within types?  I know Rob had talked about the idea that the category type is just a content type with nesting, So with content types with nesting we can still have inheritance along the branches, right? But what about inheritance from one type (category) to another (let's call it page so we don't have legacy baggage)? Does that concept even make sense any more outside of legacy applications? Or would nested branches just end up with leaves that are of the same type?

Elin









Niels Braczek

unread,
Jan 23, 2012, 12:12:35 PM1/23/12
to joomla-de...@googlegroups.com
Am 23.01.2012 11:45, schrieb Elin Waring:

> I think that some how people have totally lost the thread raised by my
> original question which was not about translated content, multilingual
> sites or anything like that.

I assumed that, because the language field is used that way in 1.7.

> 1. It's baseline expected that the actual language of a page will be given

> in the header. [...]

> 2. If and when we have a search package in the platform, wouldn't we
> routinely want to incorporate information about the user's language and the
> content language in order to provide the most useful results at the top of

> the results list? [...]

Ok, I think, I got your point now.

In consequence, *every* table having textual content would need a
language field, which contains the code of the language the content
actually is *written* in (currently the language field is widely used to
declare the language the content is *targeting* at, i.e., for which
language it should be shown).

> Those are just two of the reasons why I think it would make sense to just
> add a column.

Seen from that perspective, it actually does make sense.

Language is a natural (and important) attribute of textual content that
usually can't be retrieved from the content itself, so it has to be
provided as a separate field.

> Thanks, and yes I know it is all because of the excitement of something
> potentially new and cool people are taking small, narrowly defined comments
> and running in all different interesting directions with them.

[OT] That's how the system is working. One leads to the other. That is
ok and intended; see Usenet culture. [/OT]

CirTap

unread,
Jan 23, 2012, 1:27:12 PM1/23/12
to joomla-de...@googlegroups.com
Hello,

I totally agree with Elin regarding the 'language' *property* to be a
relevant property for a content item.

I'm just curious about the rational why it shoudn't be in "the core"? I
hope some of those "voting" against can provide reasonable answers to
back up their thinking.

As Elin already noted several times, having a language field/property
attached to content is *not* about *translations* or multilingual
content, gentlemen, which is why I think some are/were entirely missing
the point Elin tried to make.

It's about storing a perfectly legitimate and relevant *attribute* with
the original content data. A attribute that has an inherent meaning for
the majory of content humans create and deal with.

She presented some interesting use cases and I'm glad to see Niels got
the drift, once he pulled off his CMS hat :)

> Language is a natural (and important) attribute of textual content
> that usually can't be retrieved from the content itself, so it
> has to be provided as a separate field.

If you throw language in the mix *you can* go multilingual with about
anything, and *you can* do translations, but this are just two possible
applications of language.
I'd argue that language allows to attribute much more than just
"textual" content.

If you think of language, stop thinking "CMS" and stop thinking
"webpage". Language carries much more than some formal grammer or script
that's applied to its textual representation.

We all write and/or speak some sort of language, right?
We all name things using a language, don't we?

So for those with some lack of imagination, here are a couple of more
examples and why `language` is an important and inherent property of
_content_.

If all of your application's content is nothing but sourcecode
snippets, the `language` field could perfectly well feature the name of
the *programming language* a snippet is written in, i.e. PHP, Ruby,
Java, C++, you name it.
You might think a category could do it, and you're right. It could, but
why bother with that (business) data, if the `language` attribute
already delivers that information and perfectly attributes the code?
You release the app, people love it, but they want the code comments to
be available in various forms, even other (spoken) languages, or more
specific, with and without examples etc.
Your snippet-app's data model _could_ then split the sourcecode from
it's comments, and use the code language for the former and a human
language for the latter.
Comments can be in English, Spanish, Klingon.
And since the language issue is out of the way, comments can be derived
into simple API "content types", more elaborate "content types" or turn
into tutorial "content types" ...
Open for translators to kickit.
Even the code itself could be "translated" from one programming language
to another.

Although a `language` property often represents a human language, it's
also useful as a locale to apply (or preserve) proper data formatting.

Any piece of communication detail, part of a more complex content type
(let's call 'em "compound types"), may have the need to be formatted
based on it's language (locale) which might very well be different from
the language assigned to the compound content type.

A customer (human) contact presumably speaks a particular language no
matter where s/he lives, so it's nice for customer care to have that
information at hand to contact "the spanish guy living in India".

My communication details in an address book might contain 'en'
despite it's not my first language.
My home address might feature 'de' so the app knows how to properly
format the address and validate the phone number.
My lastname (if you want to be that granular) would feature 'fr' so a
TTS engine can pronounce it correctly...

Assigned to text the application or consumer of that text can choose
the approriate spell-checker.

Screen readers can switch the language dictionary.

Assigned to audio one can feed the stream to the appropriate
transcription tool that knows how to handle it (i.e. Adobe Premiere).

So whereever I look I can see a language attached to it :)

For those who (still) disagree, here are some questions:
Why can't you "see" it? ;)

What makes `language` a less important *property* of some arbitrary
basic type of content such as text, image, audio, or video, than let's
say, fields such as `created_by[_alias]`, `hits`, various meta
fields, or a bunch of dates?

What makes those fields so utterly superior over `language` that they're
"granted" their own field, which apparently makes them inherently
relevant to the record, and a "must have" for ALL types of content?

They're certainly *nice to have* from a pure *technical* perspective and
pretty *convenient*, too, but they have nothing to do with the *content*
itself nor are they even close to be even considered *a native property*
for the any sort of *content* humans create and consume.

They're at best considered *business data* and as such *exclusively*
related to some inner workings of some aribitrary application.
They're exchangeable meta-data an application can use to make smarter
decisions according to its "business" model.

It's been suggested to put `language` in a separate table.
Fine, but why should anybody else then bother with or have any general
need for such meta-data fields or a purely statistical information such
as `hits`?
Hit when, by what, or why?
Even the updated, checked-out or whatever'ed dates are only required for
specific application logic; they're not inherent attributes of content.
We just got used to these (kind of) fields.
They're a habit, maybe a good practice, but that's about it.

What if content isn't supposed to be edited or modified once it has been
created/added (think of static archives)? Why would such content have a
need for a modified or checked-out date hanging around?

Why are any such fields considered necessary and more relevant to be
applied to ALL content items by default, but the most obvious property
of almost any content created *by* humans *for* humans is not?

They can easily be replaced with completely different values and it
wouldn't make any real difference.
Change the "owner" or the modified date. No problem.

Try that with `language`...

CirTap

Sam Moffatt

unread,
Jan 23, 2012, 1:58:30 PM1/23/12
to joomla-de...@googlegroups.com
So to quote Louis:
<quote>

As for language, we don't have a use for that internally and figured
it would be better to have that discussion within the community rather
than creating an arbitrary implementation where we don't have any
particular horse in the race. It certainly would be easy to add it
though, just depends on how you'd want to handle it.
</quote>

This is at least one production deployed situation where language
isn't a concern or even a part of what was offered. The UCM, as with
most systems, was designed with it's particular base requirements in
mind with a mind to making something generic available: so there isn't
a language field because it just wasn't required and wouldn't be
utilised.

So instead of long posts on the utility of such a field, let's instead
do as Louis' suggests: work out how it should be handled and what
would be required to make that happen instead.

Cheers,

Sam Moffatt
http://pasamio.id.au

CirTap

unread,
Jan 23, 2012, 2:45:56 PM1/23/12
to joomla-de...@googlegroups.com
> So instead of long posts on the utility of such a field, let's instead
> do as Louis' suggests: work out how it should be handled and what
> would be required to make that happen instead.

For a starter: how 'bout adding a generic field to said generic solution
and name it 'language'?
Then people have a "first class citizien" in the content table and
something to handle.

There are many ways to skin a cat, but you need a cat first.

CirTap

Niels Braczek

unread,
Jan 23, 2012, 2:51:46 PM1/23/12
to joomla-de...@googlegroups.com
Am 23.01.2012 19:27, schrieb CirTap:

> She presented some interesting use cases and I'm glad to see Niels got
> the drift, once he pulled off his CMS hat :)

;-)

>> Language is a natural (and important) attribute of textual content
>> that usually can't be retrieved from the content itself, so it
>> has to be provided as a separate field.

> I'd argue that language allows to attribute much more than just
> "textual" content.

Yes, you're right. "Textual" in this context should be replaced with
"containing text (written or spoken)".

> If all of your application's content is nothing but sourcecode
> snippets, the `language` field could perfectly well feature the name of
> the *programming language* a snippet is written in, i.e. PHP, Ruby,
> Java, C++, you name it.

Since programming languages usually need different renderers, they are
different content *types* (eg, as a subtype of "snippet"). Storing the
*type* information in a language property seems not to be the right
approach to me.

Nevertheless, I agree with the other points you raise.

Niels Braczek

unread,
Jan 23, 2012, 3:02:18 PM1/23/12
to joomla-de...@googlegroups.com
Am 23.01.2012 19:58, schrieb Sam Moffatt:

> This is at least one production deployed situation where language
> isn't a concern or even a part of what was offered. The UCM, as with
> most systems, was designed with it's particular base requirements in
> mind with a mind to making something generic available: so there isn't
> a language field because it just wasn't required and wouldn't be
> utilised.

Well, what we've seen in this discussion is that the language field *is*
needed. How it is handled, depends on the application. CMS will use it
to set the appropriate headers and meta tags. Other applications will
maybe do other things with it. That is not the platform's concern.

> So instead of long posts on the utility of such a field,

That long post might have made the need of the field obvious to many
peaople. So it were far from being superfluous.

> let's instead
> do as Louis' suggests: work out how it should be handled and what
> would be required to make that happen instead.

To start with: Just add that field. Document it as containing the
language the content is written (or spoken) in, and thus restricted to
ISO compliant codes (wiki, column comment). The further handling of that
field is delegated to the application.

Rob Schley

unread,
Jan 23, 2012, 5:17:26 PM1/23/12
to Joomla! Platform Development
Rouven,

This didn't go through the first time so I'll try again.

1) Yes, this is a known issue. The main problem is that the three
existing applications (JApplication, JWeb, and JCli) do not share a
common interface. This is the main reason that I've been pushing
JApplicationInterface. I'm going to try to get that all squared away
this week and submit a pull request so we can deal with all the
applications in a similar way. Not having that, we decided to just use
JWeb as the base point because that would be the most adaptable from
the platform perspective.

2) In our implementation, we usually let the controller interact
directly with content objects for CRUD operations and left the model
to fetching data for views. In a revamped MVC paradigm, this would
probably change but that separation worked pretty well given the state
of things.

3) I assume the biggest issue was the foreign keys to jos_users? Sorry
about that, it is something we meant to note somewhere but forgot.

4) I think the content stuff as is (aside from needing to be updated)
can go in independently of any MVC work. That's why we decided to
release this chunk of code the way it is instead of tying it to any
MVC work that we've done independently or that is going on in the
larger community.

5) We were going to try to do that last week but just got distracted.
I'd be happy to review any fixes you might have.

Cheers,
Rob

Rob Schley

unread,
Jan 21, 2012, 2:23:33 PM1/21/12
to joomla-de...@googlegroups.com
Yes, that sounds right. The content package is also using a new authorisation package that we created to clearly define which objects supported access controls by using interfaces. This allows the rules to be stored anywhere, the objects just have to implement the correct interface. 

On Sat, Jan 21, 2012 at 7:43 AM, Elin Waring <elin....@gmail.com> wrote:
Also ... and I know I need to build instead of code browse and maybe this is just totally obvious but I just want to ask so I know how to build a simple example.

 If I did this in a hypothetical web application X, the rules are stored right there in the content item so we wouldn't need anything like the assets table that .. some other applications built on older versions of the platform, let's call that application Y, have  to hold the rules (setting aside inheritance/nesting for the moment). And presumably also if application X had the capacity to add ... things... that were built to extend it, the rules for those things could be stored right in the #__things table.  

And if Application Y wanted to move to this, in terms of content,  it would really just have to go get the rules from the assets tables and add them to a new column in the content table. And in terms of things move the rules to the things table.

 Is that about right?  


Elin

Rob Schley

unread,
Jan 23, 2012, 12:44:33 PM1/23/12
to joomla-de...@googlegroups.com
As for the whole language debate, honestly, it doesn't matter. If you want an extra field, add a field. If not, don't. The point is that the code will handle it either way. If you want to talk about the best strategy for implementing multi-lingual support, I suggest you guys start a new thread so you can focus on that issue independently. 

Cheers,
Rob

On Mon, Jan 23, 2012 at 9:41 AM, Rob Schley <eclec...@gmail.com> wrote:
Rouven,

1) Yes, this is a known issue. The main problem is that the three existing applications (JApplication, JWeb, and JCli) do not share a common interface. This is the main reason that I've been pushing JApplicationInterface. I'm going to try to get that all squared away this week and submit a pull request so we can deal with all the applications in a similar way. Not having that, we decided to just use JWeb as the base point because that would be the most adaptable from the platform perspective.

2) In our implementation, we usually let the controller interact directly with content objects for CRUD operations and left the model to fetching data for views. In a revamped MVC paradigm, this would probably change but that separation worked pretty well given the state of things.

3) I assume the biggest issue was the foreign keys to jos_users? Sorry about that, it is something we meant to note somewhere but forgot. 

4) I think the content stuff as is (aside from needing to be updated) can go in independently of any MVC work. That's why we decided to release this chunk of code the way it is instead of tying it to any MVC work that we've done independently or that is going on in the larger community. 

5) We were going to try to do that last week but just got distracted. I'd be happy to review any fixes you might have. 

Cheers,
Rob

Rob Schley

unread,
Jan 21, 2012, 2:14:55 PM1/21/12
to joomla-de...@googlegroups.com
Yes. That is correct. Creating the object before the form is displayed is very useful for doing AJAX operations and multi-step forms. 

On Sat, Jan 21, 2012 at 6:55 AM, Elin Waring <elin....@gmail.com> wrote:
Could one of you explain a little more about the $temporary property?  Is the basic idea that there is a quick temporary save on create which lets us do things but then the flag is removed on the real/permanent save?

Elin

Rob Schley

unread,
Jan 23, 2012, 12:41:54 PM1/23/12
to joomla-de...@googlegroups.com
Rouven,

1) Yes, this is a known issue. The main problem is that the three existing applications (JApplication, JWeb, and JCli) do not share a common interface. This is the main reason that I've been pushing JApplicationInterface. I'm going to try to get that all squared away this week and submit a pull request so we can deal with all the applications in a similar way. Not having that, we decided to just use JWeb as the base point because that would be the most adaptable from the platform perspective.

2) In our implementation, we usually let the controller interact directly with content objects for CRUD operations and left the model to fetching data for views. In a revamped MVC paradigm, this would probably change but that separation worked pretty well given the state of things.

3) I assume the biggest issue was the foreign keys to jos_users? Sorry about that, it is something we meant to note somewhere but forgot. 

4) I think the content stuff as is (aside from needing to be updated) can go in independently of any MVC work. That's why we decided to release this chunk of code the way it is instead of tying it to any MVC work that we've done independently or that is going on in the larger community. 

5) We were going to try to do that last week but just got distracted. I'd be happy to review any fixes you might have. 

Cheers,
Rob

On Mon, Jan 23, 2012 at 3:24 AM, Rouven Weßling <m...@rouvenwessling.de> wrote:

Andrew Eddie

unread,
Jan 23, 2012, 6:23:47 PM1/23/12
to joomla-de...@googlegroups.com
On 23 January 2012 20:45, Elin Waring <elin....@gmail.com> wrote:
> I think that some how people have totally lost the thread raised by my
> original question which was not about translated content, multilingual sites
> or anything like that. It was about why there isn't a language field in the
> core content table. I'm sure that's because it is exciting to think about
> the possibilities that exist with UCM, but that's not the question I raised.

Well, I think it just raises the point that language is thought of in
many different ways.

>
> Here are just a couple of reasons why it would make sense.
>
> 1. It's baseline expected that the actual language of a page will be given
> in the header. Do we really need to build a whole separate table just to do
> this for a web application running sites that might just have a few pieces
> of content are in a language that is not the default language for the
> instance of the application?

There is nothing stopping an application from adding the field to the
base table. JDatabaseObject will work it out.

> Do we even assume that all platform web
> applications have a default language?

I would expect that if you need a default language you would include it.

> None of the example applications so
> far have that, even the ones that have configurations.

I'm sure one could be provided, just needs someone to write it.

> 2. If and when we have a search package in the platform, wouldn't we
> routinely want to incorporate information about the user's language and the
> content language in order to provide the most useful results at the top of
> the results list? Not to mention won't we need to know if we need to deal
> with not having spaces between words or double byte spaces? Aren't we likely
> to just end up having to add language as a field later if we add search? And
> isn't search basically the way people interact with the web?

Well, we've already established that language can be considered in
several forms, so I think the role for treating those forms rests with
the search "plugin".

> Those are just two of the reasons why I think it would make sense to just
> add a column.

You can add the column for your application. However, I would leave
handling for that column in a derived class, not JContent.

> Here are two use cases where a site could have multiple languages but is not
> having translated content.
>
> 1. I run the language department in a school. We teach English, Mandarin,
> German, and Latin. I write a simple web application to allow my students to
> post essays they have written in the languages they are learning. Should I
> really need to create a whole new extras table just to get that one column?

Well, it would be a reasonable assumption that you are probably going
to have other custom fields, so the join table will exist anyway, but
yes, I'd add the column in the extended data for this case (note: you
don't need one table for language, and then another table for custom
fields - the language column would be a "custom field" among zero to
many more).

> 2. I run an international NGO that helps victims of natural disasters find
> lost relatives. I never know from one day to the next if i will be dealing
> with an earthquake in Armenia or a tsunami in Sri Lanka. I want the people
> who I am serving to be able to write in their own languages and I want
> search engines to index my site effectively so that people searching for
> people impacted by a specific disaster are served as efficiently as possible
> which means sending out the correct information on the language of the
> content items.

Again, you are likely to have a great deal of extended data so there
is no additional overhead in having the language field in the join
table.

>
> I'm not saying it shouldn't be a discussion (just like the need for 4 dates
>  or a media field or revisions or featured can be--and I'm in favor of all
> of them) but I'd appreciate it if this discussion could be about the
> question I actually raised and that what I have raised not be misrepresented
> (and I'm certain that was unintentional).

The solution I gave to you covers a one-to-one or a one-to-many (aka
ML) situation. To be perfectly clear on the answer to your question:
I'd use a custom language field (it just happens to also cover ML as
well) *but* a custom application can still do whatever it likes to the
base jos_content table and then define it's own handling in a custom
class (there is a lot of scope to override with custom classes, see
JContentFactory).

Regards,
Andrew Eddie

Elin Waring

unread,
Jan 23, 2012, 8:35:25 PM1/23/12
to joomla-de...@googlegroups.com
So you're anticipating there being a set of several fundamentally different architectural solutions to language aware content management, and I'm thinking about search, meta data, and mark up.


I thought this from W3C was interesting for the part I'm thinking about

Specifying the language of content is useful for a wide number of applications, from linguistically-sensitive searching to applying language-specific display properties. In some cases the potential applications for language information are still waiting for implementations to catch up, whereas in others, such as detection of language by voice browsers, it is a necessity today. On the other hand, adding markup for language information to content is something that can and should be done today. Without it, it will not be possible to take advantage of any future developments.



Elin

Andrew Eddie

unread,
Jan 24, 2012, 2:02:19 AM1/24/12
to joomla-de...@googlegroups.com
There could be fundamentally different approaches but I haven't
thought about it much beyond what I've already mentioned. As Sam has
already pointed out, it hasn't been a mandate to provide that feature
at this stage, hence it's absence. You are most welcome to work on it
though.

Regards,
Andrew Eddie

Rouven Weßling

unread,
Jan 24, 2012, 10:08:41 PM1/24/12
to joomla-de...@googlegroups.com
I know we're still talking about the architecture but I think I may have found a bug but I'm not entirely sure.

If I have a type that uses a second table to store extended fields it doesn't get deleted when I call JConent::delete(). I suspect this is a bug.

I also got a more general question - how do I get the id of a type?
Intuitively I'd have said JContentFactory::getType() with the type name as an argument would get me an object from which I could read the id. However this just loads the class (if not a special one is created just JContent) with no data populated.

My next guess was to try JFactoryHelper::getTypes() which returns an array of all type objects (this time populated) but the id is hidden away in a protected property.

Best regards
Rouven


Andrew Eddie

unread,
Jan 24, 2012, 10:22:22 PM1/24/12
to joomla-de...@googlegroups.com
On 25 January 2012 13:08, Rouven Weßling <m...@rouvenwessling.de> wrote:
> I know we're still talking about the architecture but I think I may have found a bug but I'm not entirely sure.
>
> If I have a type that uses a second table to store extended fields it doesn't get deleted when I call JConent::delete(). I suspect this is a bug.

You need to have an FK restraint on the content_id and have it
cascade. That way the database does all the cleanup and you don't
need to code it explicitly.

> I also got a more general question - how do I get the id of a type?
> Intuitively I'd have said JContentFactory::getType() with the type name as an argument would get me an object from which I could read the id. However this just loads the class (if not a special one is created just JContent) with no data populated.
>
> My next guess was to try JFactoryHelper::getTypes() which returns an array of all type objects (this time populated) but the id is hidden away in a protected property.

We generally only have to work with the alias of the type. What are
you trying to do?

Regards,
Andrew Eddie

Rouven Weßling

unread,
Jan 24, 2012, 10:31:20 PM1/24/12
to joomla-de...@googlegroups.com

On 25.01.2012, at 04:22, Andrew Eddie wrote:

> On 25 January 2012 13:08, Rouven Weßling <m...@rouvenwessling.de> wrote:
>> I know we're still talking about the architecture but I think I may have found a bug but I'm not entirely sure.
>>
>> If I have a type that uses a second table to store extended fields it doesn't get deleted when I call JConent::delete(). I suspect this is a bug.
>
> You need to have an FK restraint on the content_id and have it
> cascade. That way the database does all the cleanup and you don't
> need to code it explicitly.

That's where I'm being punished for knowing next to nil about SQL. Will try it out.

>> I also got a more general question - how do I get the id of a type?
>> Intuitively I'd have said JContentFactory::getType() with the type name as an argument would get me an object from which I could read the id. However this just loads the class (if not a special one is created just JContent) with no data populated.
>>
>> My next guess was to try JFactoryHelper::getTypes() which returns an array of all type objects (this time populated) but the id is hidden away in a protected property.
>
> We generally only have to work with the alias of the type. What are
> you trying to do?

As a test bed I'm trying to change com_content to use JContent. I'm using a direct access to the database to fetch the data for the list view. Now if I add another content type they would all be listed (set aside that the innerjoin wouldn't work, let's assume to content types that only use the content table) but I only want those of one content type.

Thanks for your help
Rouven

Andrew Eddie

unread,
Jan 24, 2012, 10:37:52 PM1/24/12
to joomla-de...@googlegroups.com
Ok. Hang off on the list views for now. We are still working out a
better system for that ourselves.

Regards,
Andrew Eddie

Rob Schley

unread,
Jan 25, 2012, 12:05:59 AM1/25/12
to joomla-de...@googlegroups.com
Rouven,

You can access any of the properties in the JContentType object using the magic __get(), $type->type_id is the thing you're looking for. If you have a content object loaded, you can use $content->type->type_id

Elin Waring

unread,
Jan 26, 2012, 5:15:07 PM1/26/12
to joomla-de...@googlegroups.com
Just thinking conceptually, are we thinking of everything that renders as a "page"  in a browser as content?  I think that makes sense and potentially kind of eliminates some of the difficult-to-get-your-head-around ways (e.g. seeing not being part of doing in one application but s being part of it -i.e. core.manage being essentially a seeing permission--in another application) we have been thinking about ACL implementation for the past few years.  (Yes I get that we may not end up with two applications running side by side, but let's just say we did.)

One question I had about the handling of root users is whether any thought has been given to the other concept that is related to but is disruptive of the rest of the permissions system which is the concept of "own" e.g. edit.own, edit.state.own, delete.own, even create.own really if you think about it.   I mean it works to the extent it is implemented but it breaks the elegance and rules of the rest of the system (there's no way it makes sense in a system with strict, default to deny inheritance everywhere else to suddenly say inside a locked area oh but never mind it's fine to do whatever you want here. I would think that at scale and with social that the issue of what really is owned by the user would be really important and I wonder if you have thought about it? (Not saying I want anything done about it, just interested in if it has come up at all.)


Elin

Elin Waring

unread,
Feb 6, 2012, 1:59:06 AM2/6/12
to joomla-de...@googlegroups.com, louis....@gmail.com
Hi,

I decided that I wanted to understand the UCM better so I started playing around with building some pages using JContent and working with the sample data from the CMS. Obviously this is pretty crude (there are a bunch of things that are hard coded that shouldn't be and it's not using MVC at all), but it is a functioning set of pages and if you are trying to get your head around the concepts it may help to take a look. 

http://ucm.cloudaccess.net/platform/examples/web/content/index.php

If you want the code.

I am reasonably positive that this doesn't match what was expected by Louis, Rob et al.,  but I guess when you have a platform that's kind of how it goes. You get people doing the unexpected.  It definitely is very empowering in that way. So for example there didn't seem to be a rule that said each type had to have a separate table just that each type had to have a table, so I created some new types that use the same tables as others. (You'll see this in the photo gallery where I  took a category and made it a gallery type and then took the articles the images were in and made them type=image instead of type=article).

Overall I found it easy to work with especially when I actually read the documentation. There are a few things I'd like to see handled better (there are way too many fatal errors coming out of getting an empty result from a query--a mistyped URL should not be generating a fatal error and we need a way to handle making lists based on fields beside content_id and type. As I said previously I think that language and meta data have a place in the core content table) but overall I'm really pleased with how easy it is, even getting the parameters out of the old data is easier than it is now. 

I'm not sure what the deal is with process, but I haven't seen anyone arguing against merging these libraries. Totally separate from the CMS schedule I'd like to see that happen soon because I think they are going to create an explosion in people building platform applications for specific purposes.  


Elin

Elin Waring

unread,
Feb 10, 2012, 4:55:31 PM2/10/12
to joomla-de...@googlegroups.com, louis....@gmail.com
So about the media field ... I notice that the documentation specifies that it is JSON and that's great because application X uses JSON to store a field of images it as does the application that I have been working on. I think it's pretty useful but it led me to a couple of questions. First, do we just assume that the application is going to the decoding of that? Or would the library be aware of it the way it is for config? 
Also do you have some general thinking about best practices for using JSON in JContent? For example, it might make sense to assume that you  store some or all of meta data in JSON. I know it's not in the core table at the moment though it's hard for me to imagine an argument for web content best practices that doesn't include some meta data.

Elin

Beat

unread,
Feb 11, 2012, 5:40:29 AM2/11/12
to Joomla! Platform Development
Just a short reminder:

While JSON is great, JSON should only be used for field contents that
cannot be searchable in database queries. ;-)

Beat
http://www.joomlapolis.com/

Elin Waring

unread,
Feb 17, 2012, 8:17:22 PM2/17/12
to joomla-de...@googlegroups.com, louis....@gmail.com
I have a question about the config field.
Is the idea here that this would store settings that relate to the common fields (such as show_title, show_media, show_body, allow_like and so on? 
And in that case if you were explaining how to use ucm in development, would you say that, although it would not be required, often there would be a second config field that would set the options for fields in the secondary table? 



Elin

Rob Schley

unread,
Feb 17, 2012, 8:34:30 PM2/17/12
to joomla-de...@googlegroups.com
Yes, the config field will be used to store various configuration attributes (whatever they may be).

I think there should only be one config field if they are being used for (more or less) the same purpose. The fields are in different tables because that is what the structure requires, the config field has no structural requirements to split the fields. So, no, there should only be one config field in my opinion.

Cheers,
Rob

Elin Waring

unread,
Feb 17, 2012, 11:26:39 PM2/17/12
to joomla-de...@googlegroups.com
That's what I was thinking originally and I made my content table with config field structure that was type dependent,  but then when I went to make the forms I realized that if  content.xml holds the config field then it's going to have to be the same for all types or at least that's how I read it. On the other hand maybe the config field should just always be in type.xml so that they are always type specific, but that is going to make a lot of redundant xml since a lot of config options will be the same (show_title, show_body etc). I don't see a structural reason why the division between primary and secondary table has to match the division between primary and secondary xml files, when they are in use they are like one big table and one big form. Is that right?

Elin


Rob Schley

unread,
Feb 18, 2012, 12:15:58 AM2/18/12
to joomla-de...@googlegroups.com
The forms support extending/overriding with extra fields just like the content objects. We use a generic content.xml and then a type specific xml file if necessary. 

Elin Waring

unread,
Feb 18, 2012, 7:19:27 AM2/18/12
to joomla-de...@googlegroups.com
So what would make sense to me based on what you are saying is if there is a common field called config in content.xml but that could be replaced if the type.xml wanted to override it, but the loadFile for type.xml has the replace flag set to false so if I understand it correctly JForm is actually going to ignore the second field with the same name? 

 
Maybe it would make sense to default reset to true so, for example, a field could be made required or have a different 
filter for different types and also for the case I'm interested at the moment in you could override the config.

I guess I should also change my original question to: Would a type often have a field called say config_type that would contain a second group of configuration fields that are specific to the type. 

Elin

Elin Waring

unread,
Feb 19, 2012, 2:42:45 PM2/19/12
to joomla-de...@googlegroups.com
Actually I think I'm changing my mind on the replace issue, I think it will be more flexible on a practical level not to replace (not only but partly because of the issue below), I think it will probably i but I want to raise another issue.

The JContent package includes an sql file with a number of tables with user_id as a key and the user_id in the sql file is specified as var(10) unsigned. This is actually standard in CMS data tables, but those tables do not currently use those IDs as foreign keys the way JContent does. 
 CONSTRAINT `#__content_ibfk_2` FOREIGN KEY (`modified_user_id`) REFERENCES `#__users` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
  CONSTRAINT `#__content_ibfk_3` FOREIGN KEY (`checked_out_user_id`) REFERENCES `#__users` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
  CONSTRAINT `#__content_ibfk_4` FOREIGN KEY (`created_user_id`) REFERENCES `#__users` (`id`) ON UPDATE CASCADE


This actually means that if you have a user table created by any prior verison of the CMS the SQL can't really be imported at least   in MySQL because you will get an error 150.  This is the issue that Rouven and I discussed in this thread a few weeks ago.
So I think that if that there needs to be some communication between platform and CMS and maybe the platform team could suggest to the CMS team to do an alter table but given that it is likely that there are many many applications/extensions that use the user table and possibly some of them have built in FKs the time to give some notice on this is, basically, now since there could be many ripple effects.(You need a content_id field in the content table too but that's a simpler thing since it doesn't change something existing).

This makes me wonder what we should do. I really like having the sql in the package and in this case it is essential precisely because you do need to have the keys set up and a specific set of fields. Would the idea be to do similar for other packages? I think it is a good idea. Although it might cause some pain in the short term I think it will reduce conflicts in the longer term especially if people are assuming that the CMS joomla.sql file is going to be definitive, which it clearly is not in this case and I'm sure won't be in others going forward. Still we could start by using the ddl in that file as a baseline for the ddls for platform tables . Then there is also the question of where this should go, in the table package or in say the user package? 

 The other issue is that there are at least 5 tables that those tables or JContent have dependencies on (user, usergroups, user_usergroup_maps, session (unless you aren't storing in the database? But I think even if you use file it still stores in the database), viewlevels.  It seems to me that this should probably be in the documentation for the package and really for any package. Does that make sense? Just to have a standard part of the manual be a list of the tables needed?    And maybe a list of other dependencies? That's all going to be necessary if people want to start taking just specific pieces.

Even though I'm raising this in the UCM thread because that's where it became clear to me that we have a potential problem, I think it is a more general issue. And i think there is actually a very immediate issue on the user id that has to be taken care of if this package is going to be merged.

Elin

Elin Waring

unread,
Feb 20, 2012, 8:49:14 AM2/20/12
to joomla-de...@googlegroups.com
Another dependency/question I have is about the assets table. It seems to me that type are assets since they have permissions and only assets have permissions, right? And so the parent for an item would be the type, and would type  have a parent of content or would we just skip the component level?  So if gallery is a type and i have a gallery with content_id of 592 ... that would be gallery.592 in my application since I don't have a concept of component? I haven't really go my mind completely around how the assets table works outside of the cms. Or if UCM were implemented in the CMS as say com_content_types it would be com_content_types.gallery.592? 

And then if you wanted to implement a nested content type (which is something Rob mentioned in NYC) and connect gallery to that for purposes of inheritance of permissions (and maybe state) then you could conceivably parent to that in the assets table?  If I wanted to make gallery as a type itself nested from my reading thus far (and admittedly I'm finding new all the time) I don't see a simple way to use another gallery as the parent asset.

Related to that, the current api for categories is dependent on the cms concept of extension but in moving to types we would escape a major limitation of  that and be able put different types in the same category because they are all in the com_content_types extension. Is that right?  Which is totally great. I know it's a cross over question, but this is really about using the API. 

Are we going to do something about the dependency in categories or is categories going to move to the cms?


Elin

Elin Waring

unread,
Feb 22, 2012, 10:56:36 AM2/22/12
to joomla-de...@googlegroups.com
Actually rereading this thread I realized Rob answered most of my question about assets weeks ago. I just have that table on my mind for other reasons.

Elin


Elin Waring

unread,
Feb 23, 2012, 7:10:30 PM2/23/12
to joomla-de...@googlegroups.com, louis....@gmail.com
When you have a moment, would someone explain the difference between a cursory and full permissions check and possible use cases for each? And also why not all not the can   methods have the option?

Elin

Rob Schley

unread,
Feb 24, 2012, 2:52:14 AM2/24/12
to joomla-de...@googlegroups.com
Sure. It is simple really, "full" checks are available for actions that require checking out the content item. The full check will call the canCheckout() method to provide the most accurate response. That is why not all of the can* methods have a full flag, not all of those actions require checking out the content. For example, canView() does not require checking out the content. 

I'm currently rethinking some of those methods in conjunction with some other stuff I'm working on. We'll see if they survive the next iteration. 

Herman Peeren

unread,
Apr 5, 2012, 3:09:30 PM4/5/12
to joomla-de...@googlegroups.com, louis....@gmail.com
A thought I want to share concerning the UCM. Exploring Seblod CCK, I saw that categories and user(group)s are all considered content types in there. So, a category can just as easily get extra fields as an article or a gallery or whatever. As I'm often working with hierarchical data (hence my interest in ORM, especially Doctrine 2), the possibilities of using categories as any other content type struck me.

One of the things that bothered me with Joomla (and remains unchanged for the UCM) is what I call the  "flat structure": a one-on one relationship between an object and a table-row, with the restriction of the first normal form imposed by relational databases. A field in an object cannot have multiple values, let alone multiple (possibly complex) objects. But: if I make a category content type in the UCM, then I can put other objects "under" that category. For instance: if an article would be derived from a category content type, then comments can be put under it easily. The same for orderlines and orders. Here I still see some other advantages of a "full blown" ORM (like cascading deletes or lazy loading), but I'm just playing around with some concepts.

One of the things I think about too, is that some properties and behaviour that are now fixed in UCM's base content class and inherited by the subtypes, could better be realised by composition. I'm trying to discern different behaviours that can be used in different situations; like making a difference between data, meta-data and "containment data" (in my definition, content = information that is contained and information = data + meta-data). Maybe hierarchy is also just a behaviour, that could be added to a content type.

I waited posting these half-baked thoughts until I had fully worked it out in code, but now I thought: let's just put it here, for it might be food for someone else's inspiration too.

Elin Waring

unread,
Apr 5, 2012, 9:45:19 PM4/5/12
to joomla-de...@googlegroups.com, louis....@gmail.com
I think that ucm lets the developer decide whether there should be categories and if  they exist what they should be.

Based somewhat on Rob's talk at JDay NYC and also on just how  I think about content, I've been treating categories as a content type that happens to have nesting built into its supplemental table.  This makes sense to me ontologically (a category is a thing in itself--just like a building is a thing in itself even though it  contains rooms and then things and people within rooms).  This was a big discussion in the CMS in terms of  whether the nesting i.e. should there even be such a thing as a category description or should empty categories ever be allowed to be displayed if categories are merely empy vessels for nesting or should we have the ability to configure at the category level.  This view is not wrong at all, in some ways ... the types concept solves one kind of issue which is that in the old model webmasters sometimes used categories as kind of a proxy for type and had all kinds of work arounds to implement that.  Now we can just say this is a "type" (we don't even need a new supplemental table) and give it its own layouts, parameters, ACL and so on.

Another thing that UCM potentially lets developers do is separate out the idea of nested content from the idea of nested sets for issues like ACL.  In some contexts that's going to make sense, in others maybe not especially if ownership can be handled nicely (a previous question I asked that got lost in other discussions).

The whole matter becomes a question of what you do with the supplemental table for the type, whether it is the only table or it in turn has, for example, many to one relationships or for that matter goes out to a different databases ... that's probably one way to approach comments or multilingual for example, or any time you need that.  It's still really up to a developer to decide that, and the supplemental table can be really minimal. I don't see why there would be any inherent limitation on what you can do once you leave the supplemental table (any more than there is a limitation to what happens in the supplemental table beyond the limits of a single record ).  

Those are just some thoughts, there is definitely a lot to think about and I suspect developers will push UCM in all kinds of interesting and unexpected ways.

Elin

Rob Schley

unread,
Apr 6, 2012, 2:39:12 AM4/6/12
to joomla-de...@googlegroups.com, louis....@gmail.com
Hey Herman,

There are many different points in your post so I'm not sure I'll be able to reply to them all but here are a few things that come to mind:

1) The published version of UCM is actually quite old and needs to be updated. As we've continue to work on it, we've done a lot of improvements and enhancements but, as a result, we've added a lot of dependencies. We're in the process of getting all of the dependencies sorted out and contributed (starting with MVC). 

2) In my opinion, categories are content. I have a pretty broad view of content and would and have included categories, tags, and several other organizational entities in my definition of content. The way I tend to define things as content is very simple: content is things that should be searchable and have a permanent URL. Anything that meets that criteria, could be considered content.

3) Without getting into all the details here, in newer versions of UCM, we use a data mapper pattern which allows us to support more complex properties, lazy loading, complex relationships and a number of other very useful features. If you're going to be at JAB, I'll get into it pretty deeply there.

I hope that helps. Keep an eye on the next few pull requests coming from eBay SF. They should be pretty interesting ;)

Cheers,
Rob

Herman Peeren

unread,
Apr 6, 2012, 3:01:13 AM4/6/12
to joomla-de...@googlegroups.com, louis....@gmail.com
Thanks for the answers. Unfortunately I cannot be at JAB this time (sob-sob). But I'm looking forward to what is published on the web (and publish some of my own experiments, and my experiences with Doctrine too).

Ofer Cohen

unread,
Apr 7, 2012, 6:23:00 PM4/7/12
to joomla-de...@googlegroups.com

Hey Rob,

Is there any documents in the wiki about all of those content & UCM definitions?

When can we see the last improvements of the UCM?

Thanks

Ofer Cohen

Ian

unread,
Apr 7, 2012, 9:15:50 PM4/7/12
to joomla-de...@googlegroups.com
Do you mean aside from the documentation that is in his source tree?

Ian

Rob Schley

unread,
Apr 8, 2012, 3:51:27 PM4/8/12
to joomla-de...@googlegroups.com
Ofer,


As for when you can see the new code, we're trying to get all of the content package dependencies sorted out and committed. The MVC changes have caused a bit of a debate though. So, it just depends on how long that takes.

Cheers,
Rob

Elin Waring

unread,
May 11, 2012, 6:00:21 PM5/11/12
to joomla-de...@googlegroups.com, louis....@gmail.com
I have a question about the authorisation package. I know it's all still a work in progress and you've also changed some names, but in many web applications there is a concept of individual level acl/ownership in addition to group acl. For example, a content item has a creator and that is FK to the user table.  If we only want that creator (and root) to be able to do anything to that item I think as of now we would have to create a group for each creator.  That seems, to put it mildly, potentially unwieldy on sites with thousands of creators.  So the option of being able to create a custom authoriser seems like a way to handle that.  Is that right? 

Of course in the CMS we're frustrated by the fact that we have only edit.own and not edit.state.own and delete.own so if this did give us a way to overcome that it would be great.

Elin

On Monday, December 19, 2011 6:21:36 PM UTC-5, Louis Landry wrote: 
We are very pleased to present our first draft of what we, at eBay, have coined the Unified Content Model, or UCM for short.  UCM is the basis for an intranet product at eBay that we call the DataHub which is currently modeled on Joomla 1.5.  It's a social analytics site that allows business, financial and data analysts a way to collaborate on our data and analytics assets.  You can sort of think of it as a GitHub for data/analytics.  The repository (https://github.com/eBaySF/joomla-platform) holds a fork of the Joomla Platform with the first step in making parts of the DataHub available to both the Joomla and wider Open Source communities.  There are many parts to what we have built which are in varying degrees of readiness to share, but we wanted to get the ball rolling with the foundational stuff with an eye to sharing all the other bits as soon as is feasible.  We'd love to know what you think and if there is interest in adding it to the core platform distribution.  With that here is a quick overview of what you will find in the content branch of our fork.

New Content Package

Under /libraries/joomla you'll find a new content package.  This package includes some new classes: JContent, JContentType, JContentHelper and JContentFactory.  JContent supports a base content object that automatically handles typical "content" data that will be familiar to Joomla and CMS users alike - things like a title, body, checkin and checkout, and so forth.  JContent is designed to be extended both in terms of API and in terms of database schema.  The base object already does this by providing related tables to capture hits and whether users "like" the content.  Any number of content "types" can be supported which use the base schema and API, or can be extended from JContent to add more API, and can be linked to one or more additional tables to provide additional "fields" for the content.

JDatabaseObject

There are a number of additions to support the content package and one of these is a new class called JDatabaseObject.  JDatabaseObject can be thought of as a re-imagined JTable class, and allows for content to be stored across multiple tables.  It's strength is that it is aware of the schema of declared tables so while you can bind as much data to the objects as you want, only the data in the underlying tables is stored (compare this to JTable where you can get into trouble if you add properties that aren't in the database table).

JCacheObject

Underpinning JDatabaseObject is JCacheObject and this provides as simple way for objects to cache instances of themselves either in memory or to a persistent store.

New Authorisation Package

The authorisation package is a reworking of Joomla's access package but done to decouple it from the CMS-ism's of that package.  This new package adds a number of interfaces to allow for better design practice.  Among these are a new concept called "authorisers" and you'll find two authorisers in the package: "default" and "root".  Currently the Joomla CMS hard codes the checks for what it calls a "root" or "super" user.  Separating the logic out into authorisation strategies allows for such checks to be done at a more abstract level, even allowing for an application to create new strategies.

Documentation

The API includes full documentation blocks and also a developers reference in docbook format for all the new code mentioned above.

Feedback

Hopefully the code will speak for itself, but there is certainly a lot to take in.  We'd appreciate feedback on the API and how we've handled various aspects of the UCM.  Following that feedback, we would like to offer this to the Joomla Platform if it's something that others will find useful.

Future Steps

Showing off code is great, but showing off how it works is even better.  We are developing a simple platform application to show off the main capabilities of the UCM architecture and that should be available in Q1 next year.  There are some coupling problems with the current platform MVC implementation which made it hard to put together a dead simple example application.  Since reworking that part of the platform is on the 12.1 roadmap we decided to put off the example application until then.  Those of you who are already building platform applications will be able to play with the UCM, and we certainly look forward to your thoughts.

Obviously we could not include everything that we've built in the DataHub all at once, but to list a few of the things we've already built on top of the UCM that we'd like to share in the near future include integrated tagging support, social groups, member profiles, activities (think activity stream), and much more.  

Cheers,
Louis

Andrew Eddie

unread,
May 11, 2012, 6:58:05 PM5/11/12
to joomla-de...@googlegroups.com, louis....@gmail.com
On 12 May 2012 08:00, Elin Waring <elin....@gmail.com> wrote:
> Of course in the CMS we're frustrated by the fact that we have only edit.own
> and not edit.state.own and delete.own so if this did give us a way to
> overcome that it would be great.

That's due to the fact that I cheated to implement edit.own for
expediency. That permission should not actually not exist. Rather,
the document should save an "edit" permission against the owner of the
document (-$userId in the case of the CMS), probably via a plugin. It
should be treated as a business rule, not a permission per se.

Regards,
Andrew Eddie
http://learn.theartofjoomla.com - training videos for Joomla developers

Elin Waring

unread,
May 11, 2012, 11:42:37 PM5/11/12
to joomla-de...@googlegroups.com, louis....@gmail.com
Yeah, it's really outside of the ACL as implemented and I remember talking to you about that a lot at that time, but could you explain what exactly you mean by a business rule in terms of implementation and why you think in particular in say setting up a multi blogger site  we would use business rules rather than ACL for managing what actions people can take on what assets.  

Elin

Andrew Eddie

unread,
May 12, 2012, 6:17:04 PM5/12/12
to joomla-de...@googlegroups.com
On 12 May 2012 13:42, Elin Waring <elin....@gmail.com> wrote:
> Yeah, it's really outside of the ACL as implemented and I remember talking
> to you about that a lot at that time, but could you explain what exactly you
> mean by a business rule in terms of implementation and why you think in
> particular in say setting up a multi blogger site  we would use business
> rules rather than ACL for managing what actions people can take on what
> assets.

You still use the ACL classes, it's just that you manipulate it with
business rules. For example, an "owner" content plugin could be added
that automatically adds the required permissions to achieve edit.own,
edit.state.own, etc through business rules set in the plugin. The
"business rules" are used to tell the plugin when to apply them, but
you still end up with a JSON string representing that User 123 (the
owner) can explicitly edit article id 456.

Nick Weavers

unread,
May 13, 2012, 5:23:54 AM5/13/12
to joomla-de...@googlegroups.com
In an business/organisation their is commonly a hierarchy of responsibility (eg departments/groups), with "managers" at each level able to delegate authority. Lets say each dept has it's own content which is managed an a "need to know" justified basis. Since the staffing in businesses/organisations is never static; managers and  come and go, and so do delegates who need to create new assets/rules for the dept. Would delegation be considered a business rule? Is there scope in the suggested plugin scheme to be able to model what we see as fundamental organisational constructs in business in a practical way. Should we encode business rules in programming code as a plugin? Or do we need something like a "friendly" ACL editor that facilitates ACL housekeeping and management for ACL rules in all (currently known) contexts (ie individual, group, business/organisation) storing the rules in the current JSON format? Such an editor could have all of the knowledge about which parts a user could edit/modify in the various contexts available embedded within it, and would be able to understand, use and obey the ACL itself recursively. 

I'm really thinking of the analogy of log table to slide rules to calculators. It feels a little to me like te ACL as it stands is akin to solving maths problems while only being allowed to use first principles.  Since Joomla appears to be finding its way into more and more organisations these days the delegation problem may become larger (although I don't have any stats to back this up I know it caused difficulties for me when I worked at IBM and used it), so an ACL editor/manager could be a solution.

Another point I often run up against is that it's often not just articles or chunks of content we need to control access to. For components that are less CMS like, with complex tables/relationships of their own, the data in those tables themselves isn't necessarily what you need to define access to, it is the "views" (as in selects with joins, not Joomla views) that make the data meaningful. I have often thought it would be useful to be able to define rules that use controller/task or sub-controller/task as the "assets", since I often align them more with "views" and the operations performed on a "view".

Does anyone else have similar thoughts or organise their components is a similar way, or am I alone in my own hopelessly lost "doing it all the wrong way"  little bubble ? :)   





  

Elin Waring

unread,
May 13, 2012, 5:24:07 PM5/13/12
to joomla-de...@googlegroups.com
I started a reply to Andrew that asked a similar question. I think the answer is probably that the plugin can have whatever rules it wants with the level of detail needed for the use case. You could also have multiple plugins. But this is really a separate issue from what I really asked which is whether it would make sense to have an authoriser called owner help manage some of this when UCM is implemented.  We also have a GSoC project that is building a workflow engine which is ultimately probably the way to go for a lot of the business logic.  Adding that kind of infrastructure to the platform is the route to make the platform attractive to new developers.

I didn't mean for my side comment to hijack this thread into a discussion of the limitations of CMS 2.5 ACL because the whole point is that we're going to have the potential to do things differently and potentially more powerfully when UCM is implemented which I think it must be for the CMS to remain viable for the long term. 
Elin

Elin Waring

unread,
Jun 21, 2012, 7:38:19 AM6/21/12
to joomla-de...@googlegroups.com, louis....@gmail.com
We've been having a discussion about how UCM might be implemented in the CMS on the CMS list https://groups.google.com/forum/?fromgroups#!topic/joomla-dev-cms/fwS3NkGwsdo and the question has been raised of taking the likes field that is in the core content table and instead putting it in a separate table the way hits is.   Sam pointed out that this would be easy to do.  Is that something that you're open to for the platform packages?  

Also, on kind of the same theme, at the JAB discussion of UCM of course one of the things I talked about was how inexpensive buy in is for extensions since really they can just define a type or type and put their existing table(s) as secondary tables and not even map any of their other fields except really alias to the core content table.  Some of the extension developers (and these were extensions like Kunena which will likely be platform applications so this is a valid platform question) asked whether it would make sense to take out the fields  that they see as CMS-specific fields like body, media and config and put them in a  separate table. The whole issue of config raised another question, since there is some interest in reducing the number of places where redundant configuration options are set and really removing the presentation elements completely from the storing of the content data (and make them the business of whatever ends up controlling the construction of pages in the application plus defaults for the type). But that is why I asked earlier in the thread what kind of config items you are thinking go in that field. 

Ofer Cohen

unread,
Jun 28, 2012, 5:42:49 PM6/28/12
to joomla-de...@googlegroups.com
Hey Elin,
IMHO, the likes field should not be part of the core fields.
We should decide and declare which fields are mandatory.

Just my 2 cents

Ofer Cohen
Reply all
Reply to author
Forward
0 new messages