Mommy, where does the code come from?
Lukas Kahwe Smith
Jordi just merged https://github.com/doctrine/phpcr-odm/pull/2
With this commit PHPCR ODM follows the Doctrine ODM Interfaces (see http://www.doctrine-project.org/jira/browse/DCOM-28)
But there are a couple issues with this commit:
- there are no tests
- there is no documentation
- setting a different id generator's isnt supported in any of the support classmeta formats
- we still do not have a decision on renaming id's to path (aka https://github.com/doctrine/phpcr-odm/wiki/Path-handling)
- we still do not have mapping support for references and embedded documents
On that note I should say that I am very thankful to Jordi (and his crew) for having created PHPCR ODM, for David's (and his crew) work on Jackalope and also on pushing discussions on this list and working on the sandbox, which was originally created by Florian and used as a public test-bed by the ideato guys. I am of course also very thankful to Benjamin for his feedback and Uwe for jump starting the file handling stuff.
But hey, we have 142 members on this list! I think we could do much better in terms of progress!
Now getting into Jackalope, PHPCR ODM is not trivial. It will take a couple of days. The benefit is that once that is done you will be ahead of the curve and helping out on something that will make a big splash. I know it will.
Also you will let me look like less of a douche for having told Liip that its worthwhile to spend the extra time to build our CMS strategy for the years going forward on the community.
So seriously if you want to be part of this step up sooner rather than later. Ask questions, poke around in the code, submit a few crappy PR's, learn from the feedback and sooner rather than later you will find yourself feeling all around awesome for being a critical piece in this community :)
regards,
Lukas Kahwe Smith
m...@pooteeweet.org
Uwe Jäger
Tom Boutell
of them), I think something that would really help motivate people
would be a quick blog post on why and how using the CMF gives you
things that just using the MongoDB ODM doesn't. (I'm guessing the
advantages over MySQL ORM are more obvious, like not having to make a
big deal of "schema" changes.)
--
Tom Boutell
P'unk Avenue
215 755 1330
punkave.com
window.punkave.com
Lukas Kahwe Smith
On 18.03.2011, at 13:38, Tom Boutell wrote:
> For those who understand the CMF and the JCR (myself not yet being one
> of them), I think something that would really help motivate people
> would be a quick blog post on why and how using the CMF gives you
> things that just using the MongoDB ODM doesn't. (I'm guessing the
> advantages over MySQL ORM are more obvious, like not having to make a
> big deal of "schema" changes.)
this should answer a fair number of questions:
http://cmf.symfony.com/2011-03-03-Symfony2-CMF.html
simply MongoDB doesnt provide native facilities for tree let alone graphs, versioning etc.
regards,
Lukas Kahwe Smith
sm...@pooteeweet.org
Tom Boutell
(Our end users are horrified by their Drupal experiences, but that's
not the point I was reading the presentation for (: )
--
Lukas Kahwe Smith
On 18.03.2011, at 13:46, Tom Boutell wrote:
> That is helpful, thanks.
but is it sufficient?
do we need more information? do we need to make those slides more prominent? add more content, provide a summarized version for quicker review?
Lukas Kahwe Smith
On 18.03.2011, at 13:30, Uwe Jäger wrote:
> Hi,
>
> currently it might look like we are still far away from our go aim to create the cmf, but after playing around with what already exists (not for more than a day ...) I realized that just with the repository and the ODM you can really start building something. Some topics with the ODM look complicated and some are really complicated. From my point of view the best strategy is to start using the sandbox to create a content driven application. That will allow us to fix the missing pieces as we go and also give others a starting point to play. Luckily it's just software so none of our decisions is final.
>
> Yes we need to implement the references and the children stuff ODM and we need to sort out the basic repository structure. There more people are involved the better the result should be.
indeed!
> I hope there will be soon one or two projects at our company that I would love to build on Symfony2 and Symfony-CMF. Then we might have the change to contribute more.
awesome
> PS: Feedback on path, id, references and children coming soon ...
awesome :)
regards,
Lukas Kahwe Smith
sm...@pooteeweet.org
Lukas Kahwe Smith
On 18.03.2011, at 13:30, Uwe Jäger wrote:
> Hi,
>
> currently it might look like we are still far away from our go aim to create the cmf, but after playing around with what already exists (not for more than a day ...) I realized that just with the repository and the ODM you can really start building something. Some topics with the ODM look complicated and some are really complicated. From my point of view the best strategy is to start using the sandbox to create a content driven application. That will allow us to fix the missing pieces as we go and also give others a starting point to play. Luckily it's just software so none of our decisions is final.
btw i added a super simple example for phpcr odm to the liip hello world bundle:
https://github.com/liip/HelloBundle/blob/master/Controller/HelloController.php#L44
regards
Lukas Kahwe Smith
sm...@pooteeweet.org
Keri Henare
I'm trying to find time to spend on SymfonyCMF but at the moment I've been too busy to even have a proper fiddle with Symfony2.
I expected myself to be a lot more active on this project than has been the case so far, if not with code submission then at least with feedback and planning. I also expected people like Ryan Weaver, Tom Boutell, Jonathan Wage, etc to be more active. That's not having a go at them as I think their reasons are potentially similar to mine.
I haven't been very involved not just because I'm not very familiar with JCR and Jackalope but also because they aren't very important to me currently.
I was always interested in JCR compatibility because it was a nice-to-have in the long term but my main interest in SymfonyCMF has always been as a replacement for Sympal, Diem, Apostrophe and our own in-house CMS.
JCR compatibility is very important to SymfonyCMF but it can't be a hindrance, otherwise it will undermine the whole project.
We need to find a way to develop the pure PHP backend alongside the Jackalope backend otherwise people are going to loose interest and create their own solutions which will defeat the whole purpose of this initiative.
Just my 2 cents worth.
Kind regards,
Keri Henare
---------------------------------------------------
Chief Technical Officer
Pixel Fusion
[e] ke...@pixelfusion.co.nz
[w] pixelfusion.co.nz
[m] (+64) 021 874 552
[p] (+64) 09 550 3084
Zend Certified Engineer - http://www.zend.com/en/yellow-pages#show-ClientCandidateID=ZEND013450
PLEASE NOTE: I check my email 3 times per day and will respond at these intervals. For anything urgent please ring me.
---------------------------------------------------
Lukas Kahwe Smith
On 21.03.2011, at 01:53, Keri Henare wrote:
> I was always interested in JCR compatibility because it was a nice-to-have in the long term but my main interest in SymfonyCMF has always been as a replacement for Sympal, Diem, Apostrophe and our own in-house CMS.
>
> JCR compatibility is very important to SymfonyCMF but it can't be a hindrance, otherwise it will undermine the whole project.
>
> We need to find a way to develop the pure PHP backend alongside the Jackalope backend otherwise people are going to loose interest and create their own solutions which will defeat the whole purpose of this initiative.
Ok, but this is a bit of a chicken and egg problem, which will hopefully not end up derailing the entire project. Right now I do not see Liip starting on the pure PHP backend any time soon, especially since so few contributions are done else where that we can justified doing this for the sake of the community after having done so much else already.
This is a very real problem.
That being said, form all I have heard people are not all that happy with Zend_Search_Lucene based fulltext search, so I wonder if whatever we come up with in the end will not require a proper Lucene based search anyway, meaning there isn't anyway around installing Java for anything half way serious anyway. Not saying this to imply that we do not need a pure PHP backend, just that I expected fewer people to see Java as a no go for their projects. Or is it not Java thats the no go, just Jackrabbit ..?
Uwe Jäger
Keri Henare
I don't think that there's any expectation for Liip to do it all. I'm sure that the rest of us can work on the pure PHP backend. I guess it's more a question of how we can do it alongside the JCR backend.
Kind regards,
Keri Henare
---------------------------------------------------
Chief Technical Officer
Pixel Fusion
Zend Certified Engineer - http://www.zend.com/en/yellow-pages#show-ClientCandidateID=ZEND013450
PLEASE NOTE: I check my email 3 times per day and will respond at these intervals. For anything urgent please ring me.
---------------------------------------------------
Lukas Kahwe Smith
On 21.03.2011, at 12:35, Keri Henare wrote:
> For us it's an issue of scale. Ideally, we could use the CMF on large sites where we could use Jackrabbit and Solr but we also need it to work on smaller sites with a pure PHP backend and Zend_Lucene.
Ok, that makes sense. Now the question is in what order things need to happen for it to be worthwhile for others to join in. If other people start working on PHPCR ODM and Bundles on top, then that would free up resource at Liip to help jump start a pure PHP backend.
> I don't think that there's any expectation for Liip to do it all. I'm sure that the rest of us can work on the pure PHP backend. I guess it's more a question of how we can do it alongside the JCR backend.
Well thx to Jackrabbit and the fact that we have a more or less working PHPCR ODM we can work in parallel on all parts. Aka we can work on a backend, improve the ODM or work on frontend Bundles all at the same time.
If someone starts work on a php backend, Liip will surely assists with knowhow, but until the community also chips in on the things that we need for our clients (aka PHPCR ODM and frontend Bundles) we cannot allocate significant development resources into the backend. At least not for the next months.
In terms of installating Jackrabbit shouldnt be a problem btw:
Just download and start with java -jar [downloaded jar name].
So the question remains:
How do we break this vicious circle that we seem to facing here, aka lack of a pure PHP backend turning people off from helping on the PHPCR ODM and frontend Bundles, which in turn prevent Liip from being able to help on the PHP backend.
Benjamin Eberlei
write a backend for Jackalope that is not Jackrabbit. There is a backend
interface (Is that up to date David?) which you can implement. It should be
fairly easy to get a MySQL or CouchDB backend working for this. It musn't
support all JCR features, but jackalope itself doesnt even do that right
now.
The problem here is that people seem to think this is complicated or needs
special knowledge, but you can make it a weekend hackathon to implement a
first version that should run for all the simple read and write scenarios.
There are even tons of functional tests that support you in this
implementation.
With a backend working it is easy to get it working with Doctrine PHPCR
ODM and then with the Symfony CMF Bundles.
greetings,
Benjamin
David Buchmann
Hash: SHA1
> I don't think that there's any expectation for Liip to do it all. I'm sure that the rest of us can work on the pure PHP backend. I guess it's more a question of how we can do it alongside the JCR backend.
just a note on this one: most of jackalope is not tied to any specific
backend. look for "transport" in jackalope :-) its still WIP though, but
the plan is to settle on an interface for transport.
/basically/ you can just plug in a different transport
implementation and persist the data elsewhere. most work will be to do
all the validation that has to happen on save. we currently delegate it
to the jackrabbit backend.
we have tons of functional tests in the github project
jackalope-api-tests that can be used to check if the implementation
conforms to the spec.
if anybody wants to work on that, contact me, i will gladly try to help.
cheers,david
- --
Liip AG // Agile Web Development // T +41 26 422 25 11
CH-1700 Fribourg // PGP 0xA581808B // www.liip.ch
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk2HSt4ACgkQqBnXnqWBgIvkbQCgguvQVXHUH2TcD4n5tRJ+Fh7E
jmoAn2i39hRrqoYsykVQltYRuAX+ZHPY
=qCA7
-----END PGP SIGNATURE-----
Tom Boutell
like. Whether one can plan for an efficient join and so on.
--
Lukas Kahwe Smith
On 21.03.2011, at 14:07, Tom Boutell wrote:
> I wonder what the performance of a MySQL back end for this would be
> like. Whether one can plan for an efficient join and so on.
Actually in most cases you will not need any join's since you are storing Documents which are not normalized. The main thing we should look into is making it possible to fetch multiple Documents (and their children) efficiently and thats all.
David Buchmann
Hash: SHA1
hi tom,
jackrabbit loads everything from the backend and does the tree
operations in-memory afaik, and uses lucene internally for search anyway.
for a php-only implementation, we will have to come up with some ideas
to get a decent performance. even more so if it has to work on cheap
hosting without memcache...
cheers,david
Am 21.03.2011 14:07, schrieb Tom Boutell:
> I wonder what the performance of a MySQL back end for this would be
> like. Whether one can plan for an efficient join and so on.
>
> On Mon, Mar 21, 2011 at 8:55 AM, David Buchmann <david.b...@liip.ch> wrote:
>>>> I don't think that there's any expectation for Liip to do it all. I'm sure that the rest of us can work on the pure PHP backend. I guess it's more a question of how we can do it alongside the JCR backend.
>
> just a note on this one: most of jackalope is not tied to any specific
> backend. look for "transport" in jackalope :-) its still WIP though, but
> the plan is to settle on an interface for transport.
>
> /basically/ you can just plug in a different transport
> implementation and persist the data elsewhere. most work will be to do
> all the validation that has to happen on save. we currently delegate it
> to the jackrabbit backend.
>
> we have tons of functional tests in the github project
> jackalope-api-tests that can be used to check if the implementation
> conforms to the spec.
>
> if anybody wants to work on that, contact me, i will gladly try to help.
>
> cheers,david
>>
- --
Liip AG // Agile Web Development // T +41 26 422 25 11
CH-1700 Fribourg // PGP 0xA581808B // www.liip.ch
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk2HUGYACgkQqBnXnqWBgIvZjwCdFP0+mbyv9k29PGVzjh25Zu6h
JVoAn13JoGCr1hXmSI+E/UtlT/L6OpDJ
=UW+y
-----END PGP SIGNATURE-----
Tom Boutell
backend seems a natural choice. We know it is not as fast, but those
who want truly epic performance will go for Jackrabbit presumably
--
Christian Stocker
On 21.03.11 14:07, Tom Boutell wrote:
> I wonder what the performance of a MySQL back end for this would be
> like. Whether one can plan for an efficient join and so on.
the "joins" are done with lucene in jackrabbit. So one of the hard
things will be to translate the JCR-SQL2 query language to something a
pure PHP solution can understand (even if it's Zend_Lucene). Just
storing and getting nodes are pretty simple DOM-like operations
chregu
Lukas Kahwe Smith
On 21.03.2011, at 14:25, Tom Boutell wrote:
> If Jackrabbit uses Lucene search, using Zend Lucene search in a PHP
> backend seems a natural choice. We know it is not as fast, but those
> who want truly epic performance will go for Jackrabbit presumably
Right. My point was just more that from all I gathered the "breaking point" for Zend_Search_Lucene is reached for fairly simple projects, which in turn let me to believe the acceptance that a non PHP Lucene implementation is needed for quite a lot of projects and that the PHP backend is likely more for less experienced developers scared of Java or for ultra simple projects.
I am just trying to understand what is scaring people off atm. If its Java or if its Jackrabbit.
Tom Boutell
to grapple with, I think it's good to question that first.
The Zend_Search_Lucene documentation warns not to use it as your data store.
Christian, you don't mean that (Java) Lucene is actually storing all
the CMS content, do you? Or is it simply that it has all the metadata?
Does that make sense - is it efficient to update things? I understand
that the JCR has been around a while and there are presumably major
sites running on it successfully...?
--
Christian Stocker
On 21.03.11 14:56, Tom Boutell wrote:
> Maybe Jackrabbit is something we have to ask our self-hosting clients
> to grapple with, I think it's good to question that first.
>
> The Zend_Search_Lucene documentation warns not to use it as your data store.
>
> Christian, you don't mean that (Java) Lucene is actually storing all
> the CMS content, do you? Or is it simply that it has all the metadata?
> Does that make sense - is it efficient to update things? I understand
> that the JCR has been around a while and there are presumably major
> sites running on it successfully...?
It just holds the index. The actual data is stored somewhere else
(mysql, filesystem, postgresql, there are many implementations for
that). The index can be restored from the datastore (if you delete the
index files, it just recreates it.
Does that answer your question?
chregu
Lukas Kahwe Smith
On 21.03.2011, at 14:56, Tom Boutell wrote:
> Maybe Jackrabbit is something we have to ask our self-hosting clients
> to grapple with, I think it's good to question that first.
>
> The Zend_Search_Lucene documentation warns not to use it as your data store.
>
> Christian, you don't mean that (Java) Lucene is actually storing all
> the CMS content, do you? Or is it simply that it has all the metadata?
> Does that make sense - is it efficient to update things? I understand
> that the JCR has been around a while and there are presumably major
> sites running on it successfully...?
No, well sort of.
JCR uses a data store like the filesystem or an RDBMS to store nodes essentially as serialized blob's indexed by their path. So accessing node's by path requires no joins.
Now for "searches" there is a separate API, which in the case of Jackrabbit is implemented on top of Lucene, so all data that should be searchable also needs to live inside the Lucene index.
Lukas Kahwe Smith
I should elaborate on this point, because its really one of the keys of why JCR is so elegant imho.
The point is that CMS content is quite different than normal data stored in an RDBMS in that one doesnt need to aggregate, rarely does bulk updates, usually also doesnt do equality searches and also no joins are needed. Instead one traverses nodes or does full text searches.
Only with these assumptions does it make sense how JCR is setup and this provides a lot of flexibility when it comes to content storage and scaling. The requirements for content storage then really just boil down to being able to support quick PK lookups (optionally also transactions) and for search to provide powerful full text capabilities. The later is perfectly provided by Lucene based solutions, while the former can be pretty much any database (including the file system).
Of course this also makes it clear why such a content repository isnt the right place for all data (like inventory or orders for example).
Tom Boutell
tag or category (as you might often otherwise do with a join), rather
than by fulltext search? I've found that to be much more of a hassle
in Zend Lucene than the documentation suggests.
--
Lukas Kahwe Smith
On 21.03.2011, at 15:14, Tom Boutell wrote:
> I understand. How well does this work for finding things cleanly by
> tag or category (as you might often otherwise do with a join), rather
> than by fulltext search? I've found that to be much more of a hassle
> in Zend Lucene than the documentation suggests.
Never used Zend Lucene, but it should be super trivial really:
Aka simply "tag: foo" states that the Lucene Document needs to match "foo" on the tag field. Note that a field can be configured to be multivalued and of course you can also query that multiple specific tags and categories are mandatory.
Benjamin Eberlei
for a MySQL (Doctrine DBAL) or CouchDB backend and ignore the search
question for now. The fcous on searching should be in a second, third, ..
milestone not in the first milestone.
greetings,
Benjamin
Lukas Kahwe Smith
On 21.03.2011, at 15:58, Benjamin Eberlei wrote:
> Why is Zend Lucene necessary for a first implementation? I would just go
> for a MySQL (Doctrine DBAL) or CouchDB backend and ignore the search
> question for now. The fcous on searching should be in a second, third, ..
> milestone not in the first milestone.
agreed.
Lukas Kahwe Smith
BTW, I really appreciate the feedback on this thread. Its critical for the success of the project. So anyone who has considered contributing but hasnt for some reason, please let us know so that we can look into solving these "impediments". let me be your scrum master :)
Tom Boutell
categories, etc. are strict matches, need to be easily updated if tags
are merged or categories renamed by an admin, etc. CMS content is just
as often organized in an exceptional way (as blog posts, as employee
bios...) as in a conventional way (as pages in a tree).
--
Lukas Kahwe Smith
On 21.03.2011, at 17:45, Tom Boutell wrote:
> Unfortunately we do equality searches all the time in CMS land. Tags,
> categories, etc. are strict matches, need to be easily updated if tags
> are merged or categories renamed by an admin, etc. CMS content is just
my previous comment wasnt clear .. equality searches are also possible. simply do not apply any stemming at both index and query time on a specific field (or even copy the field to index in different ways).
> as often organized in an exceptional way (as blog posts, as employee
> bios...) as in a conventional way (as pages in a tree).
thats still just a tree, just maybe a bit more flat.
furthermore JCR even supports graphs, so you are not even forced into a tree structure.
Tom Boutell
Zend Lucene to do what I wanted in these areas, but the real Lucene
APIs might be better at it. In general we certainly need this kind of
thing to not be *harder* to code correctly than it is with SQL, which
would be a step backwards.
--
Lukas Kahwe Smith
On 21.03.2011, at 17:54, Tom Boutell wrote:
> Good points. I spent more time than expected wrestling to try to get
> Zend Lucene to do what I wanted in these areas, but the real Lucene
> APIs might be better at it. In general we certainly need this kind of
> thing to not be *harder* to code correctly than it is with SQL, which
> would be a step backwards.
Hmm this is really strange. Especially stuff like searching if a post matches one or N tags should actually be simpler in the Lucene syntax, since you dont need to fiddle with JOINs.
Quickly searched on google for an example for tag search which is a bit more complex:
http://stackoverflow.com/questions/2438000/how-do-i-implement-tag-searching-with-lucene/2495659#2495659
Benjamin Eberlei
should be possible with the first version. What i meant is the features
that are unique to Lucene (be it Zend or Solr).
Lukas Kahwe Smith
Just FYI, I am on vacation this week which basically translates to even more OSS time (although I should say that Liip is already quite gracious with letting me do OSS stuff on company time). Anyway, I will work on a serious of blog posts that hopefully address the concerns raised in this thread in a Q&A fashion. So if you have more questions or concerns about what is stopping you from contributing code please let me know.
We might in end then take these blog posts and put them on the wiki as sort of an FAQ and/or work them into the slides I have uploaded to the website.
Lukas Kahwe Smith
Just published the last post:
Part I
http://pooteeweet.org/blog/1897
Part II
http://pooteeweet.org/blog/1898
Part III
http://pooteeweet.org/blog/1903