I want to keep the current interface as-simple-as-possible.
I just have a need for multiple text-blocks on a single page.
So not very big texts, just different text blocks.
Some might be a single line, others complete stories.
The idea is to be able to split the single big body element in smaller parts, which makes the design of the template more flexible.
My idea (for now) is to add an extra "[ insert section ]" button underneath the body.
Then you can click and add a section, like the proposal Joost made.
Sections could then be (and not limited to):
- rich text
- title
- media (picture, video)
- an aside
- a reference to another article (can be displayed like an aside)
When you look at the BBC site you see a bit of this kind of structure (for example http://www.bbc.co.uk/news/world-asia-17509349 )
- Marc
On 22 mrt 2012, at 14:08, Arjan Scherpenisse wrote:
> (cc'ing the developers list)
>
> Interesting idea.
>
> I like the idea but I also like the simplicity of the current interface.
> For a lot of use cases such a structured page editor is quite overkill.
> But it also might simplify our rich text editor, which is nice.
> (noticing https://github.com/xing/wysihtml5 lately)
>
> Anyway, I'm +1 but as optional page content, e.g. keeping the old "body"
> structure intact. So maybe as a special page category.
>
> (extra HTML bodies are indeed possible by just prefixing the field names
> with "_html"; that way Zotonic does not escape them)
>
> Arjan
>
>
> On 03/22/2012 01:19 PM, Marc Worrell wrote:
>> I see it more as that it is nice to be able to enter a structured "story".
>> I don't think that the story's sections should be separate entities.
>>
>> Those "internal names" is just stuff that comes from the original site.
>> I don't see any use here, except maybe for styling reasons.
>>
>> Of course, we can lump everything together in one big body.
>> But then we already have customers where we support two bodies.
>> Typically displayed with some extra content in between.
>>
>> Maybe we can abstract that into 'sections'.
>> And then we can add extras like an 'aside', 'quote' etc.
>>
>> Joost and I discussed that in the context of maxclass some time ago and
>> he made the following (rough) design for an editor:
>>
>>
>>
>> - Marc
>>
>>
>> On 22 mrt 2012, at 12:51, Arjan Scherpenisse wrote:
>>
>>> That has my preference, actually... Keeping the rscs simple
>>>
>>> On Mar 22, 2012 12:49 PM, "Andreas Stenius" <andreas...@astekk.se
>>> <mailto:andreas...@astekk.se>> wrote:
>>>
>>> Can't those sections be expressed with edges to other "section" rsc's?
>>>
>>> Perhaps I don't fully grasp what you are trying to do...
>>>
>>> //Andreas
>>>
>>> 2012/3/22 Marc Worrell <ma...@worrell.nl <mailto:ma...@worrell.nl>>:
>>>> Hi,
>>>>
>>>> Joost and I are converting a (Anymeta) web site to Zotonic.
>>>> In the process I am converting all data from this web site to
>>> Zotonic, using the API of the other site.
>>>> This goes very well.
>>>>
>>>> I have a question regarding the structure of some textual data.
>>>> In some items there are multiple sections of text:
>>>>
>>>> body
>>>>
>>>> extra-part 1
>>>> internal-name (kind of type: COMPLETE,
>>> TESTIMONIES, etc)
>>>> title
>>>> body
>>>>
>>>> extra-part 2
>>>> internal-name
>>>> title
>>>> body
>>>>
>>>> etc.
>>>>
>>>> These items look very similar in structure to the normal
>>> title/body we have.
>>>>
>>>> I was thinking of adding a 'section' part to zotonic for this.
>>>> Then a page can consist of multiple sections.
>>>> In due time we can add more sections (if/when we want).
>>>>
>>>> I don't see how I can elegantly combine it with the existing
>>> body, so that it might become part of the sections.
>>>>
>>>> Do you guys have any idea?
>>>>
>>>> Best,
>>>>
>>>> Marc
>>>>
>>>>
>>>
>>
>
> --
> MiracleThings * / Interactive projects / creative solutions
> http://miraclethings.nl / ar...@miraclethings.nl / +31641322599
I'm sure you'll be able to accomplish this with minimal impact. :)
--Andreas
2012/3/26 Marc Worrell <ma...@worrell.nl>:
That looks awesome!
The only thing I'm wondering is how much overhead that dom parsing is
causing. Apart from that it looks really neat :)
2012/3/27 M-MZ <mmze...@xs4all.nl>:
Alternative is to do it client side.
- Marc
I've long been thinking about a problem similar to yours, but from a different perspective.
If you look at an html page from the perspective of a graphic designer you see an abstract grid that organizes disparate content and emphasizes the various content elements through position, size, typography, and color.
If you look at many sites today you see that the enclosed spaces in the grid, let's call them boxes, may contain text, images, animations, video, and interactive "applets." Some of these content elements are only distantly related to others on the page; some, such as headline, summery, body copy, call-out boxes, pull quotes, etc. may well be parts of a single document, but are arranged in different boxes, e.g., their own distinct grid, for emphasis.
So I've been working toward a work flow that emphasizes the grid first, then provides an interface that makes it easy to assign elements of content to the respective boxes in the grid.
If I'm understanding your questions, you're seeing the need for a layer between all the content included on the page and a single element of content. Something like:
Set of all content elements on the page
Subset of elements related to document X
Single element of document X
This, then suggests something like:
grid
--box 1 -> subgrid A -> document A
box A1 -> element A1
box A2 -> element A2
...
box An -> Element An
--box 2 -> document B
...
--box n -> document N
This approach, it seems to me, would make it easy to mix and match various "documents' on a page; reuse documents across pages. E.g. Ideally, I should be able to pick up subgrid A and drop it into Box XYZ on a totally different page and it will maintain its visual integrity.
I'm not sure this is totally clear, since I'm still thinking it through myself. But it may help to think of it in terms of roles.
-- Site designers care about the functionality of pages and page navigation
-- Graphic designers care about visual consistency and coherence as well as eye paths through the page
-- Content developers think about documents, headlines, subheads, images, etc.
-- Web developer assigns content and interactive functionality to boxes in the grid
The problem we're trying to solve here is how to make the work-flow within and across these roles as efficient as possible.
Internal representation of grids, boxes, documents, and content elements, then, in my view, should be designed to support tools and user interfaces that maximize these efficiencies.
All the best,
Lloyd
Snippet:
>>>> 2012/3/22 Marc Worrell <ma...@worrell.nl <mailto:ma...@worrell.nl>>:
>>>>> [...] I have a question regarding the structure of some textual data.
>>>>> In some items there are multiple sections of text:
>>>>>
>>>>> body
>>>>>
>>>>> extra-part 1
>>>>> internal-name (kind of type: COMPLETE,
>>>> TESTIMONIES, etc)
>>>>> title
>>>>> body
>>>>>
>>>>> extra-part 2
>>>>> internal-name
>>>>> title
>>>>> body
>>>>>
>>>>> etc.
>>>>>
>>>>> These items look very similar in structure to the normal
>>>> title/body we have.
>>>>>
>>>>> I was thinking of adding a 'section' part to zotonic for this.
>>>>> Then a page can consist of multiple sections.
>>>>> In due time we can add more sections (if/when we want).
>>>>>
>>>>> I don't see how I can elegantly combine it with the existing
>>>> body, so that it might become part of the sections.
>>>>> [...]
How about having an alternative representation for body?
Now the body is stored as a list. With some added checks to
m_rsc:p_no_acl(Id, body, Context), we could allow a secondary format
for body,
being a #section record:
-record(section, {id::atom(), title::string(),
body::string()|integer(), children::[#section()]}).
this would allow for nested sections of arbitrary depth (if that would
tickle any one).
Also, in the case were body is not a string, but a int (or perhaps an
atom) the idea is that it includes another rsc.
So, m.rsc[foo].body will always yield the root body, for either format.
To get at the new sections structure, one would have to use some new
api, like m.section[rsc].body (.id, .title, .children etc). N.b.
m.section[rsc].body would give the same as m.rsc[id].body
to go over the children of a rsc with many sections:
{% for s in m.section[id].children %}
m.section[s].title
m.section[s].body
{ for ss in m.section[s].children }... for deeper nesting... :)
or, the s.body is not a string
m.rsc[m.section[s].body].media
{% endfor %}
Could undoubtedly be refined and improved. And does perhaps not lend
it self to reuse among different rscs as lloyd suggested would be a
good idea.
Cheers,
Andreas
What Lloyd tell's about the different elements on the page is indeed the reason I want to be able to cut the main content in smaller fractions.
Then it is possible to place them differently or give them markup that depends on the page/template.
Which is difficult when all is combined in a single big text blob (the current body).
Though I am a strong proponent to keep data separate from the view.
In that way it is possible to reuse content between sites or views.
I am still thinking about the hierarchical list version.
Might actually a good idea, might also be too complicated for editors (the people, not the software).
Though we might just use the hierarchical menu-editor for making the editor for this structure.
With a "drag this to add a section" thing, where we now drag a new page onto the menu.
Regarding the datastructure, I was thinking of a list of proplists.
This because it is more flexible than records, experience learns that you always need to add yet another field, and then you still need to be able to handle all the old record definitions that are stored in the database.
I am still pondering about how to define the possible sections.
They need:
- editing template
- pre-storage checking/modifier/cleanup code
- viewer template
- ?? more
All ideas are welcome!
- Marc
I delighted you're thinking about this problem. It's been on my mind for many a moon, and I've actually done some work in this direction.
My thought is to break the html page design/editing problem down into discreet tasks.
Basically, we're talking about:
1) -- defining a set of named boxes, a simple list
2) -- defining what content goes into the boxes, e.g. B1 contains C1, which could certainly be represented with with prop lists
3) -- defining how the boxes are styled, e.g., position, size, borders; could be represented as a prop list or record
4) -- defining how the content is styled, e.g., typography, color, etc.; could be represented as a prop list or record
For sake of argument, let's call everything related to a discreet unit of content a document, that is, all the elements that relate to a single "story." .
A document might be as simple as a single word or paragraph, or as complex as a feature story with headlines, subheads, illustrations, pull quotes, call outs, sidebars, illustrations, etc. If we generalize, a form with intro and prompts could be considered a document, as might a video or animation.
The idea is that each box in the page grid, unless deliberately empty, gets filled with a document.
I would imagine a separate tool for each of the tasks noted above, most likely under a unifying site development tools menu. Each tool is specialized for the task and role player likely to perform the task (Note, these roles may be performed by different web-development specialists or by a lone-wolf developer wearing different hats):
1) Site Editor: Site developer defines pages and page navigation with notes about the communication goals and content of the site
2) Grid Designer: Graphic designer lays out master grids for the site; e.g. the small set of design grids that will be used for different pages in the site. Note that the goal here is to maintain visual unity across the site. Each grid would be named and each box within the grid would named. Clearly, this tool would focus on development of CSS, but only those CSS tags that relate to size and position of boxes; e.g. divs or sections.
3) Document Editor: Much like the current Zotonic page editor, but with the possibility of tagging sub elements like pull quotes, call-outs and side bars.
4) Document Designer: As above so below--- a graphic designer would design the grid and styling for the various sub-elements of the document. This design would mostly be constrained to a single container box that can easily be assigned to a box in the encompassing page grid. In rare cases, it may be desirable to sign sub-elements of the document to two or more boxes in the encompassing page grid.
5) Page Editor: Assign content elements to boxes.
In all of the above, it's important to maintain separation of content and styling. But this is simple enough, I believe, by simply naming grids, boxes, documents, and the sub-elements of documents. Since CSS cascades, the grid designer could provide styling for content of each box in the grid. When a document that has been independently styled is assigned to a box, it could either override the default box styling or accept it.
Well designed, I believe, a set of tools as described above would greatly boost efficiencies in a large web development shop while serving the lone-wolf developer equally well. In both cases, it would encourage more effective presentation and design.
As a first step, tools could be presented as standard html forms. As a second step, they could be enhanced through Ajax and JavaScript.
Best,
Also, I do think that linking to media items and other rscs will work.
It all comes down to what properties is stuffed into the proplist.
We may have a section like: [{title, <<"Catchy title">>}, {body,
<<"Some interesting body...">>}]
and another section: [{title, <<"Pic">>}, {media, 345}]
and so on... (may even put ... hrrm harkle cough... records ... in
there [.... {custom_tag, {my_record, with, some, data}}, ...] )
On the other hand, I do like your idea about having a specialized
table. That would make it easier to share sections between resources.
Depends on how complicated/powerful we want this section thingy to be...
The spec table approach may give a cleaner implementation, though. (my guess).
I like both approaches for different reasons and can't really make up
my mind about it at the moment.
About using the existing rsc-edge tables with a new category has
already been dismissed by Marc.
Marc:
> I see it more as that it is nice to be able to enter a structured "story".
> I don't think that the story's sections should be separate entities.
In response to me:
> Can't those sections be expressed with edges to other "section" rsc's?
Cheers,
Andreas
2012/3/29 M-MZ <mmze...@xs4all.nl>:
Why wouldn't it work for many and/or large sections?
Also, I do think that linking to media items and other rscs will work.
It all comes down to what properties is stuffed into the proplist.We may have a section like: [{title, <<"Catchy title">>}, {body,
<<"Some interesting body...">>}]
and another section: [{title, <<"Pic">>}, {media, 345}]
and so on... (may even put ... hrrm harkle cough... records ... in
there [.... {custom_tag, {my_record, with, some, data}}, ...] )
On the other hand, I do like your idea about having a specialized
table. That would make it easier to share sections between resources.
Depends on how complicated/powerful we want this section thingy to be...
> I don't think that the story's sections should be separate entities.
> The body value is stored in the props field in the database, which is a bytea type.
>
> Would your data model looks something like this, just for my understanding
>
> props = <<[{body,escaped_html},{sub_section1,"some text"}, .., ..., ...]>>
I was thinking along this line.
Especially after discussion with Arjan to keep the 'body' property as-is.
(Of course the 'body' property could also be simulated by rendering all sections as html...)
- Marc
> Why wouldn't it work for many and/or large sections?
>
> Also, I do think that linking to media items and other rscs will work.
> It all comes down to what properties is stuffed into the proplist.
>
> We may have a section like: [{title, <<"Catchy title">>}, {body,
> <<"Some interesting body...">>}]
> and another section: [{title, <<"Pic">>}, {media, 345}]
> and so on... (may even put ... hrrm harkle cough... records ... in
> there [.... {custom_tag, {my_record, with, some, data}}, ...] )
I was thinking about something like that.
Of course the problem with having ids in the body text is that:
- the id doesn't make sense after pubsub to another site (needs mapping/translation)
- the id might be deleted (needs intelligence in the template)
The latter problem is similar to the id not being published or accessible by the current user, so it should be solved in the template anyway.
- Marc
Why wouldn't it work for many and/or large sections?Then your resource will become a somewhat big blob inside the db. This can lead to problems with db data transfer and binary_to_term parse times. This can lead to a clogged up connection pool. I've tested this with MB sized resources.
Also, I do think that linking to media items and other rscs will work.
It all comes down to what properties is stuffed into the proplist.We may have a section like: [{title, <<"Catchy title">>}, {body,
<<"Some interesting body...">>}]
and another section: [{title, <<"Pic">>}, {media, 345}]
and so on... (may even put ... hrrm harkle cough... records ... in
there [.... {custom_tag, {my_record, with, some, data}}, ...] )Yeah, but what will happen if the media item is removed?
On the other hand, I do like your idea about having a specialized
table. That would make it easier to share sections between resources.
Depends on how complicated/powerful we want this section thingy to be...Indeed, currently I don't really have a need for this. The shared sections between resources solution is somewhat fancy and requires quite some efford from the writers to get consistent publications. You have to adapt your writing style for that. Maybe that is too fancy for now.
A separate table would keep the resources small, which is nice for all sorts of reasons. My guess is that Marc is also thinking about what will happen with elastic zotonic. Having separate entities will introduce all sorts of combinations with versions when eventual consisitency is in full swing :-) But maybe small resources is also nice for elastic? I can imagine it is easier to manage and throttle the synchronization of small resources.
> I see it more as that it is nice to be able to enter a structured "story".> I don't think that the story's sections should be separate entities.
Ok. It all depends on the amount of fancyness you need I guess.
I personally would like to see a resource's body as a list of predicate links. Then we could do something like this:{% for item in id.body.items %}{% if item | is_a:"text" %}render text{% elseif item | is_a:"image" %}render image{% elseif item | is_a:"aside" %}render an aside insert/remark{% end if %}{% endfor %}etc.This way we can be as wild as we want, combine as many blocks as we want. And create sites that are otherwise impossible/hard to do with Zotonic.An example of one such site (sorry, Russian only, and not in Zotonic :)): http://stockholmania.ru/article/syndrome/blondes/ or http://stockholmania.ru/article/syndrome/roofs/ It has at least 4 or 5 elements mixed together and not easy to replicate in Zotonic.
Let me be a long-winded heretic here. I do hope I'm not drummed out of the corps as a result.
All that follows is intended to provoke introspection and, perhaps, creative insight, rather than a flame war.
First, I believe great and good work and thought have gone into Zotonic. And the Zotonic community is terrific. When Marc first invited me to look into Zotonic I looked and developed great enthusiasm. I have specific needs for an effective CMS. I program, but I'm not a professional programmer. Any of you on your worst day can out program me at my best.
On the other hand, I do have more than a quarter of a century of software development experience, having owned a software development company, managed development of more than 100 shrink wrapped products and a major web application.
When I came to Zotonic I'd worked with Forth (loved it, but it's not appropriate for the web) and Cold Fusion (ugh); I'd flirted with Python and Joomla as possible platforms for my needs. Erlang's promise of fault tolerance and scalability, plus its extensive documentation, looked like the ideal platform on which to build the sites I have in mind. And the more I work with Erlang, the more convinced I am of the correctness of this judgment.
So, for some 18 months or more, I've been active on the Zotonic list and have tried to contribute what little I can--- mostly through suggestions for better documentation. But for reasons that may become clear in what follows, I've been turning to other solutions for my own needs.
That said, I may well turn back to Zotonic at some point because I do feel that it has tremendous potential. So, in all that follows, keep in mind that I am an end-user with real needs first, a developer second, a programmer third, and one who enjoys technology and admires technical elegance fourth.
So here's my critique for the eyes of this group only:
Summary:
Zotonic is a work-in-progress. From my perspective, Zotonic at its current stage of development, gives the ILLUSION of comprehensive ease of use. It may be quite appropriate for skilled Erlang programmers, but not for end-users with sites to develop, work to do, and deadlines to meet unless well supported by one or more Erlang programmers.
Details:
Admin: One of the best features of Zotonic, but as the current discussion attests, it is not as comprehensive as we might desire.
ErlyDTL: Great for techies, but requires considerable fluency with html, css, and Django syntax. This is OK, but moving back and forth between so many semantic/syntax interfaces greatly impedes productivity and, I'd think, discourages all but the technically inclined.
Postgresql: Postgresql is a terrific relational database. But I'm not convinced that it's strictly necessary for most CMS needs. Yes, where the CMS draws on extensive data over and above the text and images of the typical web page it may be invaluable. But as currently integrated into Zotonic, I'm becoming increasingly convinced that it's more hindrance than help. For one thing, it makes installation and maintenance considerably more difficult than it might be. And, unless I'm missing something, it prevents Zotonic from fully exploiting the fault tolerance features of Erlang.
On the last point, I'm particularly sensitive, since it was a disastrous crash of a site that I was building, and the loss of data, that provoked me to start thinking about these issues.
Documentation: Much good stuff here. But I've commented often on how and where I think it can be improved--- tried my best to help.
Food for thought and improvement:
I still don't have a deep enough technical understanding to submit code and documentation that addresses the issues that concern me. But here are a few things that might be worth thinking about and discussing.
-- Can we step back and look at Zotonic through the eyes of the constituencies who might benefit from adopting it? If we want to reserve Zotonic for an in-group of tech-savvy users, then we may need to do nothing more, but if we want a more vital community of adopters and developers, then I'd venture that this is vital.
To spark discussion, I'd suggest that these constituencies include:
o Those who commission web development for business purposes; e.g. your clients
o Professional web developers
o Graphic designers developing and maintaining websites for clients
o Erlang consultants who take on projects requiring web development
o "Army of one" entrepreneurs like me who have minimal to no budget to hire professional web developers or consultants
-- Can we step back and look at Zotonic as a tool explicitly designed to solve COMMUNICATION problems through efficient development and maintenance of websites and web applications? What does this mean, fluency, flexibility, and efficiency of expression. When I build a website, I want to say something to someone and I want to say it as quickly and inexpensively as possible.
-- Is it worthwhile to step back, reconsider, and better define the PROBLEMS we're trying to solve than to jump in prematurely with solutions?
o In my last post I suggested an approach to the problem that Marc presented from my perspective, that is as I understand, how to provide richer, more flexible, representation of Zotonic pages.
Let me generalize to define the problem as I see it:
1) A website is set of linked web pages.
2) Most websites are created to achieve explicit or implicit communication goals; e.g. to inform, persuade, entertain, or some combination of all three. These goals are expressed as a set of site specifications and achieved through audience (end-user) interest and attention.
3) To the end-user, the web page presents text, images, and interactive functionality. End-user attention and time-on-page are necessary to achieve the communication goals of the site.
4) To the graphic designer, the web page is an abstract grid that provides containers or "boxes" for "content." Boxes and content are styled to attract the eye, emphasize some content items over other, and guide the eye through the content. For the graphic designer, time is money.
5) The web developer is concerned with how to put all the pieces together, text, imagery, links, forms, queries, etc., in the most efficient possible way. For the web developer, time is money.
6) The software tool builder's perspective is how to build a technically robust suite of software tools that meets all the needs above. An important goal for open source developers is to attract and foster a talented community of committers, testers, and documentation builders. And, no doubt, to profit through employment of a more powerful and reliable tool.
7) Marc has presented a use case that illustrates one of the shortcomings of Zotonic.
The problem from 10,000 feet as I see it:
-- A grid, G, is a set of boxes (divs or sections), B
-- For each B, f(R) -> C + S
where:
R is a request
C is content (words, numbers, images, links, forms, etc)
S is styling (CSS)
And C -> f(D)
where:
D is persistent storage
Our problem, then, is to find a language, user interface, data representations, and set of request and database functions, f(R) and f(D), that best meet all the needs and constraints of the constituencies noted above.
Zotonic as now constituted goes a long way toward achieving this. But can it do better? Given the talents and motivations of this group, I trust that it can. As thought goes into finding enhanced representation of Zotonic "pages" and evolution of Zotonic toward Elastic Zotonic, perhaps looking afresh at fundamental technical assumptions may be fruitful.
Do be gentle with your slings and arrows, but I don't mind at all correction where I'm wrong.
All the best,
Lloyd
I use zotonic as a web/erlang programmer. My colleague as a front-end developer. From this point I see that the boundary given by zotonic works. He can make interactive things with templates and by using pre-cooked actions. With the help of relations between db rsc's it is possible to build quite complex data models without writing a single line of db code. This is awesome.
As an erlang programmer you have awesome powers inside zotonic. Interfacing with external systems is usually a piece of cake. Writing new actions, custom models, etc. It all fits nicely within the current framework.
Some of the stumbling blocks are: postgres sometimes acts up. If seen it killed by the oom killer on linux. No auto restart. I've seen the connection pool being exhausted by a single (big) bad resource. Installing configuring can be stumbling block. It is usually the first thing newcomers have to deal with. I'm currently working on integrating SQLite. This db removes an external dep. This weekend i planned to make the one missing component, a connection pool.
Dealing with heart is sometimes maddening. Erlang just doesn't behave like a normal unix daemon.
The notification mechanism is nice, but sometimes it detaches an observer for one single bad message. This can suddenly cause stop some of the functionality of your site. (currently searching for nice solution for this)
Could embedding another script like language help remove a hurdle for non erlang programmers? Or some more powerful erlydtml additions? There are a couple of nice script engines available for erlang.
Maas
Good questions and perceptive points; much to think about and discuss.
> I use zotonic as a web/erlang programmer. My colleague as a front-end developer. > From this point I see that the boundary given by zotonic works.
This is a fine collaborative partnership provided each partner is well skilled with the required technologies. And I would hate to see any changes to Zotonic lock out work at the Erlang/html/css level or make it more difficult in any way.
But I face two other use cases:
1) I need to do all development on my own. My html and css skills are adequate, but coming into Zotonic I did not have Erlang skills. This was a serious barrier toward productive application of Erlang toward helping me solve the problems I faced. And I found the documentation to be a mixed bag; excellent in some areas and deficient in others. I've tried to make some contributions toward that. And, on the Erlang front, I've been studying, learning, and making progress. But as I've learned more, I've found deeper reasons to reconsider Zotonic for my projects, some of which you've touched on in your post.
But I do, indeed, believe deeply enough in Zotonic, it's developers, and community to want to do what I can to help Zotonic evolve to its full potential as an efficient, flexible, and productive tool suite. Zotonic has terrific foundations. As it stands, it clearly works well for folks like you with the skills to use it.
The question, then, is this: can the barriers to entry for less technically inclined noobies be lowered to make it more accessible to a wider base of users? And is there motivation and will to work in this direction?
The second problem for me is an issue of productivity. Content entry is fairly efficient and one of the best features of Zotonic from my standpoint. But page design and layout offer little to no advantage over straight html and css. In this regard, Zotonic, as a tool, requires moving across many semantic/syntax/conceptual levels to bring up even a fairly simple website. This greatly impedes productivity and adoption in my view.
2) Very much related to the first point, I work with generally non-technical writers, editors, and graphic designers. They need a tool suite that offers the flexibility and power of Zotonic, but at it's current level of development, Zotonic is not appropriate for them.
A website is a medium of communication. Zotonic positions itself as a Content Management System. But I'd argue that Zotonic as it stands is a programmer/professional web developer's tool and a good one; but not yet an appropriate tool for the folks who create content. So can it really be called a Content Management System?
> With the help of relations between db rsc's it is possible to build quite
> complex data models without writing a single line of db code. This is awesome.
Despite 18 months of study, I still don't understand how this works. I have in the past done a fair amount of work with SQL, but it doesn't translate for me. Maybe I'm just dense. I have sensed from the beginning, however, that there's an elegant conceptual model underlying Zotonic design. I've asked Marc a few times to try to articulate it. But so far, this aspect of Zotonic eludes me.
> As an erlang programmer you have awesome powers inside zotonic. Interfacing with > external systems is usually a piece of cake. Writing new actions, custom models, > etc. It all fits nicely within the current framework.
I can see this and believe it's one of the great strengths of Zotonic. But my own skills are still insufficient to benefit. And I've found the documentation of some of the stock actions lacking. More examples, use cases, and tutorials would go a long way toward addressing this.
> Some of the stumbling blocks are: postgres sometimes acts up.
I've commented on some of my problems with postgres. Evan Miller has addressed this issue in Chicago Boss by providing a db interface layer that makes CB fairly db agnostic. And you don't have to leave Erlang to use it. I haven't been able to evaluate this feature of CB for functionality or performance, but it might be something to look into. I'm certainly in favor of anything that makes Zotonic as modular as possible; e.g. encourages mix and match of functional components.
Incidentally, Miller has also done a superb job on the documentation front.
> Could embedding another script like language help remove a hurdle for non erlang > programmers? Or some more powerful erlydtml additions? There are a couple of
> nice script engines available for erlang.
I'm not so sure the problem for non-Erlang programmers is language as much as user interface.
What's the difference? Language assumes that the user knows enough to tell the computer what to do. Erlang, html, css all do what they do quite well within their realms. And, certainly html and css are simple enough. Indeed, html was designed to be a "language" for everyman. But even at that, many content creators rebel at the thought of learning it. And both html and css are tedious and sometimes frustrating to work with.
User interface, on the other hand, exploits the power of the computer to instruct and help the user accomplish the task at hand. The challenge, of course, is designing interfaces that are as intuitive and flexible as possible. Now this, I believe, is where the talents of our very best Erlang programmers and front-end designers could really make a difference-- perhaps a suite of modular tools that sit on top of Zotonic.
I hope I haven't come across as too critical. But I see Zotonic as a work-in-progress with the potential of becoming truly best of breed in the CMS category. Nothing would make me happier. And I'm more than willing to contribute whatever I can within my modest skills to help make that happen.
My long and checkered career has been working with content as a writer, magazine editor and publisher, corporate trainer, academic, educational and consumer software developer and, now, novelist. This experience is, perhaps, what I can best contribute.
The vision of what I believe Zotonic could be keeps me up into the wee hours, as it has this morning. But I think it's well worth the thought and effort.
All the best,
Lloyd
-----Original Message-----
From: "M-MZ" <mmze...@xs4all.nl>
Sent: Saturday, March 31, 2012 6:03am
To: zotonic-d...@googlegroups.com
Subject: Re: [Zotonic-Dev] Re: Some data representation question
I fear that I've been expressing myself poorly. That's the challenge of email and the limitations of human language.
The questions I've been trying to get at go beyond simple technical solutions. And they go beyond my individual needs. Perhaps they're not worth thinking about. But I've found that it's occasionally well worth trying to understand and challenge the IMPLICIT assumptions of whatever project is underway.
Here are a few questions we might ask:
1) Are we satisfied that Zotonic is as good as it can be? If so, then there's no reason to continue the discussion.
2) What do we mean by "good?" Easy to use? Reliable? Scalable? Productive?
3) For whom? The original developers? Erlang programmers? Professional web developers? Content creators? Graphic designers? Students? Small business owners? Large enterprise users?
4) Do we want to expand the user community? Attract a broader range of users?
5) If so, how and where should be focus our efforts?
6) Do we care at all about new user and their learning curve up to Zotonic fluency?
7) Where can we most effectively focus our efforts if we truly want Zotonic to excel in any or all of the areas noted above?
Until we have clarity and consensus on questions such as these, then "improvements" to Zotonic may or may not be worth the thought and effort. They may amount to little more than putting paint on a car when what it really needs is a valve job.
When you say "not the language," if you are talking about the Django-derived language in the templates, I don't have a problem with that in and of itself. My question has to do with the efficiency and productivity of solving the page layout problem in that particular way. It certainly works. But I don't find it particularly time-efficient and have faith that it could be made more so.
And, at least in my experience, documentation of the Zotonic-extended functions is not as clear and effective as it could be.
When you talk about "resources and edges," you bring up a related issue. These are technical terms that, presumably, have specific meaning in Zotonic and for Zotonic developers. Perhaps I've missed or misunderstood something in the Zotonic documentation, but in my reading they are not explicitly nor clearly defined. I think I understand what you mean by a resource, but not confident. To this day, I confess, I'm totally in the dark about what you mean by an "edge."
As I mentioned early, my dim interpretation of the conceptual module underlying Zotonic suggests that it's based on graph-theory and, likely, quite elegant. But for the life of me, I can't connect the dots, partially because some of the terms are not well defined-- at least for me.
This again, I'd suggest, is a documentation problem. What is the conceptual model? What are the major components? What are the functional metrics that tell us whether or whether not they're each performing optimally? How do they relate to one another? Are the interfaces between them well defined so they can be swapped out if we wish to substitute a "better" component for whatever reason?
With regard to "easy way to store," I've commented on my reservations regarding Postgresql as a core component of Zotonic. And, in posts in months gone by, I've also reported my travails. Indeed, it was a data-storage problem that provoked my deeper thoughts and gathering concerns about Zotonic.
I was developing two sites-- one a relatively straight-forward blog that you can see here: fromtheglen.com. The other a much more ambitious social site.
One day, while working on the social site, Zotonic totally froze up while I was doing an absolutely routine content-entry task in admin. I could not get it started again. I hired Alain to see what he could do. It took him some time and out-of-pocket expense on my part to bring it back. When it came back, critical content, including the entire main menu tree, was gone-- content that had been a part of the site for some time. Marc, Arjan, and Michael Conner all took a look, but could not see a problem.
I suppose this certainly could have happened for reasons that have nothing to do with Zotonic-- a hardware or network glitch perhaps. But as a result, bottom line, I cannot at the point trust Zotonic as a production-ready system for my applications for two reasons: 1) lingering fears that there may be a subtle bug in the admin/db interface that could again bite me or one of my clients; 2) The fact that I can still not understand Zotonic and how it's constructed sufficient well to fix such problems were they again to arise. This, despite many hours of study and examination of Zotonic source.
I bring this up not to rehash old business, but to note how I came to think about the larger issues and why I risk ruffling features and kicking up dust by suggesting that we examine, reflect, and discuss them.
I'm really talking about our VISION of what Zotonic is and what it can be and how we get from here to there. Premature focus on technical solutions, I fear, may divert us far off course.
All the best,
Lloyd
-----Original Message-----
From: "M-MZ" <mmze...@xs4all.nl>
Sent: Monday, April 2, 2012 3:46am
To: zotonic-d...@googlegroups.com
Subject: Re: [Zotonic-Dev] Re: Some data representation question
You can be assured that everybody from the core team is involved in the discussion.
And there are also some good documentation ideas being floated!
Please keep the discussion going!
Best,
Marc
I welcome the discussion and feel privileged to be involved.
I'm by no means advocating fundamental changes to Zotonic. I am suggesting that we'd be well served by clarifying mission and goals so the evolutionary changes lead in the right direction.
The assumptions we start with, often unspoken, determine the questions we ask.
The questions we ask, in turn, determine the technical solutions we implement.
Without clarity of mission and goals, the solutions we come up with may well not lead us to where we really want to go.
And, I'd certainly stress that Zotonic cannot be all things to all people. It will serve us all well if it has a tightly defined value proposition and fulfills that promise with technical excellence.
That is a good question: what is the mission and goal of Zotonic?
Let's narrow that down.
Zotonic was started from a frustration with the standard MySQL/PHP solution.
Problems with PHP/MySQL where:
1. inefficient & slow, especially when the code base grows
2. incapable of using websockets and comet connections
3. difficult to integrate with XMPP
4. SMTP receive (& send) only possible with external software (integration problems)
5. not good use of many-core systems
6. need external cron-like software for long running tasks
I first checked Django, Rails and other frameworks.
They shared many problems with PHP/MySQL (and they solved many others).
Example: with Rails you can easily build a site, but it will be slow.
Erlang solved most of the problems, due to its lightweight processes and OTP.
So the first description of Zotonic was:
"A CMS framework with which we can build (moderately) high traffic sites on simple hardware."
A couple of goals where formulated:
1. very friendly for the front end developer
2. front end and back end developers can work independently
3. easily withstand the slashdot/fireball effect.
4. basic data model for the 80% of the content management needs
5. easy to extend with independent modules and extension mechanisms
6. easy to overrule any pre-packaged template
7. run multiple sites on a single Zotonic process (vhosts)
Recently, as a result of the projects we do, we added another goal:
8. make it fault-tolerant against server failure (aka distributed)
And we keep in mind:
9. We would like to be a "spider in the web of information"
Which implies that it should be easy to put all kinds of information in (feeds, photos, etc etc) and get it out again.
In that respect Joe Armstrong suggested that it might be a solution to a problem he has.
His problem is that he has many many thousands of Erlang modules which needs to become accessible.
Next: what other goals to other people see/have?
And then: drill it down to a single mission statement.
- Marc
I very much agree with Lloyd, that we need clear goals and objectives
to help focus our combined efforts in a common direction.
2012/4/3 Marc Worrell <mwor...@me.com>:
> [...]
> A couple of goals where formulated:
> 1. very friendly for the front end developer
I feel more work can be done for this point. As is, it's very friendly
to produce and publish content. But design wise I think the template
system should be more accessible. Like a template designer tool, or
something like that.
> 2. front end and back end developers can work independently
Fulfilled as I see it.
> 3. easily withstand the slashdot/fireball effect.
I have no experience with this yet, but I'm confident zotonic will
perform well under pressure.
> 4. basic data model for the 80% of the content management needs
Love the data model. Lloyd, I'll write a little PS for you at the end
about resources and edges. It's fundamental to zotonic to know how
they work.
> 5. easy to extend with independent modules and extension mechanisms
Again, (almost) brilliant design. Altough lacking in documentation. I
still remember my first look at zotonic, and how confused I was with
all modules and directories seemingly everywhere. So much so that I
left it as messy. I can't really recall my second visit (many months
later) but I must have taken it for a test ride and got a better feel
for it and simply dived in without knowing what to expect. As I learnt
about modules, actions, templates, edges etc I realized the
flexibility that was built in.
It is this flexibility that may seem messy at first without proper
introduction (documentation). I highly value remembering my initial
reaction to zotonic, in realizing what areas to improve for new
comers.
> 6. easy to overrule any pre-packaged template
What I miss here is a way to know what templates exists, intended use,
and what arguments they expect.
Especially for API style templates that are meant to be used from
other modules. And what to do when such a template is included for a
disabled module...
> 7. run multiple sites on a single Zotonic process (vhosts)
Works well.
> Recently, as a result of the projects we do, we added another goal:
> 8. make it fault-tolerant against server failure (aka distributed)
Looking forward to this :)
> And we keep in mind:
> 9. We would like to be a "spider in the web of information"
>
> [...]
>
> Next: what other goals do other people see/have?
> And then: drill it down to a single mission statement.
My goals are pretty much in line with the above. I want to have a
responsive, stable application framework with good monitoring support
and that is easy to extend.
I have observed that the learning curve is currently quite steep for
people new to zotonic (and erlang in general, which gives quite a bit
of new stuff to get their head around). But it isn't unmanageable with
a bit of hand-holding.
I think it is primarily on the documentation side that would ease the
learning curve. And perhaps some CLI tools to create skeletons for new
actions, modules, dispatch rules etc. could also help new devs get up
to speed.
//Andreas
PS (for Lloyd, and any one else that feels a bit lost on the whole
resources and edges business).
* Resources
A resource (also known as a page, or rsc for short) is the main
content building block. It has properties, such as title, summary and
body. The body is full html, with the possibility to embed images
(need to apply the show_media [1] filter in the template when
rendering the body to get the images on the resulting page, though).
It is possible to add your own custom properties to any resource.
Using custom searches it is also possible to search on those
properties. I won't go into details about custom searches here, more
than noting that they exist.
A resource may also be a media file (or any other data file) that you
can store on the server.
Each resource is of a specific category (like Article, or Person, or
Media, etc) to indicate what kind of resource it is representing. New
categories are easily added in the admin (or from a site/module in
erlang code).
* Edges
Now, you can link resources together, and those links are called edges
in zotonic. There are different kinds of edges (like there are
different kinds of resources), called predicates. The predicate
describes what kind of link (edge) we are talking about. For instance,
if we have a page (resource) describing a book, we may have edges from
this book page to the author of the book. The author is then another
resource, possibly in the Person category. The predicate would be
"author" to say that the link from the book page to the Bill Boston
page indicates that Bill Boston is the author of the book "Boston On
Fire" (just to make up a name).
To make another example (instead of making the previous example
comprehensible) we can look at the default blog skeleton from zotonic.
There we have (blog) articles (the pages created under the article
category). To further categorize the articles there are another
category named keyword. And a few resources added to that category is
"technical", "support" and "announcement". Take a look at the default
data installed for the blog to perhaps get a better feel for how the
pages, categories and edges link together [2].
Now, by adding links (edges) from a article to one or more of the
resources in the keyword category, the article is further categorized.
So the article may now be found (when searched/browsed for) and
grouped or listed in more ways using these links. Depending on what
predicate is used to link the article to the keyword, it may convey
different meanings.
In the blog example, the subject predicate is used. But one could just
as well add a new predicate called keyword.
The predicate controls what type of pages may be linked (both on the
subject and object end of the link).
Hope this wasn't adding to the confusion too much. I would've edited
some, but I've got to run and don't feel like holding this up
indefinitely.
Perhaps a cleaned up version could be put up on zotonic.com, detailing
what resources and edges are...
[1] http://zotonic.com/filter-show-media
[2] https://github.com/zotonic/zotonic/blob/master/src/install/z_install_defaultdata.erl

Excellent! The goals and direction of Zotonic right on from my perspective. I'd like to think about this for a day or two. But this is a terrific foundation for the discussion I'd hope we'd have.
And, once again, I'd like to thank you and other Zotonic developers for the the fine work to date.
Best wishes,
Many many thanks for your explication of resources and edges.
I have a niggling feeling that this way of conceptualizing content and the relations between content may be at the root of the problem that Marc posed at the very beginning of this thread. My still-to-be refined thoughts may be wacky and deficient, but once I can see them more clearly myself, I'll toss them out for discussion.
Thanks again,
Lloyd
-----Original Message-----
From: "Andreas Stenius" <andreas...@astekk.se>
Sent: Tuesday, April 3, 2012 7:50am
To: zotonic-d...@googlegroups.com
Subject: Re: [Zotonic-Dev] Re: Some data representation question
This, too, I think, should have prominent pride of place in the documentation.
Thanks,
Lloyd
-----Original Message-----
From: "Michael Connors" <conn...@gmail.com>
Sent: Tuesday, April 3, 2012 9:08am
To: zotonic-d...@googlegroups.com
Subject: Re: [Zotonic-Dev] Re: Some data representation question