Remove <p> wrappers around WYSIWYG shortcodes

1,371 views
Skip to first unread message

akbortoli

unread,
Oct 2, 2012, 10:58:40 PM10/2/12
to silverst...@googlegroups.com
Hello guys, today i had a good discussion with @sminnee about <p>s around WYSIWYG shortcodes, you can see my pull request


and my changes here, the code ensures that shortcodes are not wrapped in a <p>


The problem is that we have a website full of cool and useful shortcodes for our client to use, but Tinymce adds <p>s arround shortcodes and if you have a block element in it, it's not semantic at all and the code wont be valid.

Researching i found the following solutions:
  • Make your .ss file one line long (doesn't work)
  • User javascript to hide empty <p> tags (what?)
  • Use margin-bottom:0; on <p> (why?)

Do you agree or disagree with my solution, any other solution? Keen to hear from you guys.

Message has been deleted

Sam Minnée

unread,
Oct 2, 2012, 11:04:00 PM10/2/12
to silverst...@googlegroups.com
Thanks for posting here.  To summarise my concern, the problem stems from the fact that <p> tags are the correct behaviour for the WYSIYWG editor: the user ends up laying out their code within a WYSIWYG view but the code is interpreted as an HTML stream.

So a user types this:

[shortcode]
something inside the shortcode
[/shortcode]

But the code is operating on something like this:

<p>[shortcode]</p>
<p>something inside the shortcode</p>
<p>[/shortcode]</p>

I felt that trying to make this situation work ends up relying on top much magic, and that instead we should leave shortcodes designed to be coded up inside the HTML view and not allow for entry of code into the WYSIWYG view.  This does severely limit shortcodes' utility, but I thought any alternative would diminish the overall reliability of the system.

akbortoli

unread,
Oct 2, 2012, 11:04:16 PM10/2/12
to silverst...@googlegroups.com

Hamish Friedlander

unread,
Oct 2, 2012, 11:23:44 PM10/2/12
to silverst...@googlegroups.com
Although I don't necessarily agree with Sam that the answer is just not using shortcodes inside the WYSIWYG editor, I don't really like the "every p tag gets swallowed magically" method - I agree with Sam that's it's edge-case-specific magic that's likely to cause confusion.

A couple of things I think might be OK:

- Have shortcodes indicate that they swallow their containing tag. Then when you do [swallowing_tag] it will always swallow the immediately surrounding parent.

- Have a HTML equivalent for shortcodes, so you can do <div data-shortcode="YouTube XXXX">....</div> which TinyMCE wouldn't surround with <p> tags. Would probably need a plugin for TinyMCE to actually enter via the WYSIWYG form.

Hamish Friedlander

akbortoli

unread,
Oct 2, 2012, 11:24:15 PM10/2/12
to silverst...@googlegroups.com
That's not magic at all, it's a solution to a problem that exists and maybe it's not just me, could be a struggle of others developers too, i know a website that implements javascript to hide empty <p> tags to overcome the problem in SS.

Shortcode is a powerfull feature and seems really backwards to disable it for WYSIWYG content... They can be used for so many good things such as:
  • Column Layouts
  • Images
  • Galleries
  • Buttons
  • Message Box
  • Pullquote and Blockquote
  • Content Toggle and Tabs
  • List styles and Dropcaps
  • Tables
  • Recent Posts
  • Dividers
And many others...

By default that's what WYSIWYG editors do, and i think it's fine, just trying to find a solution to be able to add shortcode using WYSIWYG content and have your HTML markup clean, valid and semantic.

akbortoli

unread,
Oct 2, 2012, 11:31:17 PM10/2/12
to silverst...@googlegroups.com
It's not every <p> tag, but just <p> tags around the shortcode.

Like

<p>[shortcode]Content[/shortcode]</p>

TO

[shortcode]Content[shortcode]

And

<p>[shortcode]</p>
<p>Content</p>
<p>[/shortcode]</p>

TO

[shortcode]
<p>Content</p>
[/shortcode]

So any shortcode get valid HTML.

akbortoli

unread,
Oct 2, 2012, 11:40:24 PM10/2/12
to silverst...@googlegroups.com
My fix implementation might not be the best one, that's why i created this post, to hear from core team and other developers.

Hamish Friedlander

unread,
Oct 2, 2012, 11:56:12 PM10/2/12
to silverst...@googlegroups.com
It's not every <p> tag, but just <p> tags around the shortcode.

Like

<p>[shortcode]Content[/shortcode]</p>

TO

[shortcode]Content[shortcode]

Some shortcodes might need to be in <p> tags though - some produce inline HTML elements, or text, or <div> tags that expect to be within <p> tags for styling to work.

Anything is magic if it happens without any apparent cause. Assume that the above is true, and you did want a <p> tag - how would you debug why they just seem to disappear?
 
And

<p>[shortcode]</p>
<p>Content</p>
<p>[/shortcode]</p>

TO

[shortcode]
<p>Content</p>
[/shortcode]

That second example is interesting, because you could quite easily get TinyMCE to produce this:

<p>[shortcode]</p>
<p>Content</p>
<h1>[/shortcode]</h1>

or even

<p>[shortcode]</p>
<h1>Content[/shortcode]</h1>

or

<p>[shortcode]</p>
<h1>Con<b>tent<br/>[/shortcode]Look!</b></h1>

Detangling that last example to do what the content editor intended is very difficult.

The more I think about it, the more I think the only option is to use HTML elements for adding shortcodes if you want them to work in the WYSIWYG editor.

Hamish Friedlander 

Sam Minnée

unread,
Oct 3, 2012, 12:02:47 AM10/3/12
to silverst...@googlegroups.com

that's not magic at all, it's a solution to a problem that exists

The "magic" comes if we want to get this code to handle all the cases that are going to come up:

<p>[shortcode]</p>
<p>something inside the shortcode</p>
<p>[/shortcode]</p>

<p>[shortcode]<br>
something inside the shortcode<br>
<p>[/shortcode]</p>

<p><br>
[shortcode]<br>
something inside the shortcode<br>
<p>[/shortcode]</p>

<p><br>
[shortcode]</p>
<p>something inside the shortcode</p>
<p>[/shortcode]</p>

<div>[shortcode]</div>
<p>something inside the shortcode</p>
<p>[/shortcode]</p>

<span style="display: block">[shortcode]</span>
<p>something inside the shortcode</p>
<p>[/shortcode]</p>

I'm sure there are more, this was just off the top of my head.

If we don't get it to handle all of these situations, we'll just have another solution that breaks confusingly half the time people try to do things with it.

Sam Minnée

unread,
Oct 3, 2012, 12:03:56 AM10/3/12
to silverst...@googlegroups.com

- Have a HTML equivalent for shortcodes, so you can do <div data-shortcode="YouTube XXXX">....</div> which TinyMCE wouldn't surround with <p> tags. Would probably need a plugin for TinyMCE to actually enter via the WYSIWYG form.

This sounds like a much better thing to dig into, particularly if the WYSIWYG editor had some support for this.

Hamish Friedlander

unread,
Oct 3, 2012, 12:05:44 AM10/3/12
to silverst...@googlegroups.com
Get out of my brain Sam!

Hamish Friedlander

--
You received this message because you are subscribed to the Google Groups "SilverStripe Core Development" group.
To post to this group, send email to silverst...@googlegroups.com.
To unsubscribe from this group, send email to silverstripe-d...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/silverstripe-dev?hl=en.

akbortoli

unread,
Oct 3, 2012, 12:17:48 AM10/3/12
to silverst...@googlegroups.com
Obviously you guys are friends, my point is not around my solution but yes to at this point in time shortcodes have problem with the Tinymce editor, and if that is a better solution we should definitely implement it, but it would generate an issue with end users that doesn't know HTML and they would need to remember to user [shortcode] for HTML content and <div data-shortcode="shortcode"></div> for WYSIWYG content.

akbortoli

unread,
Oct 3, 2012, 12:19:57 AM10/3/12
to silverst...@googlegroups.com
I can tell that other frameworks uses this solution of unauto-p shortcodes and they don't have any problem or complaint about it.

Sam Minnée

unread,
Oct 3, 2012, 12:23:02 AM10/3/12
to silverst...@googlegroups.com

> Obviously you guys are friends

Heh, yeah, we've worked together for a few years - Hamish is CTO of SilverStripe Ltd and I'm CEO. I was CTO a couple of years back.

> my point is not around my solution but yes to at this point in time shortcodes have problem with the Tinymce editor, and if that is a better solution we should definitely implement it, but it would generate an issue with end users that doesn't know HTML and they would need to remember to user [shortcode] for HTML content and <div data-shortcode="shortcode"></div> for WYSIWYG content.

I think we'd deprecate the shortcodes altogether. Shortcodes aren't useful in and of themselves; they're a simple way of adding an extension point to build plugins for the CMS content. I don't think it's the best one.

The <div data-shortcode="shortcode"></div> approach would only make sense if there was a plugin to the WYSIWYG editor for injecting them into the HTML. Presumably, the editor would give them some kind of styling. However, in that case you'd have a UI for inserting these into the content instead of having to remember shortcode syntax.

Sam Minnée

unread,
Oct 3, 2012, 12:23:28 AM10/3/12
to silverst...@googlegroups.com
Could you point to some examples in other frameworks?

akbortoli

unread,
Oct 3, 2012, 12:30:23 AM10/3/12
to silverst...@googlegroups.com
Frankly you guys are not helping at all, i'm just trying to help/contribute and get a solution for my problem, we are arguing for nothing.

How someone would take a company serious if they are not listening to developers and community?

Yes that are hundreds of cases but the only one to concern about is this two

[shortcode]Content[/shortcode]

that generates

<p>[shortcode]Content/[shortcode]</p>

and 

[shortcode]
Content
[/shortcode]

that generates

<p>[shortcode]</p>
<p>Content</p> (this <p> should be here)
<p>[/shortcode]</p>

as the user would only type in

[shortcode]Content[/shortcode]
or
[shortcode]
Content
[/shortcode]

akbortoli

unread,
Oct 3, 2012, 12:36:44 AM10/3/12
to silverst...@googlegroups.com
It's nice to try contributing to something and get replies like this...

End of discussion, not using it by option... Client requirement :/

Cheers guys!

Sam Minnée

unread,
Oct 3, 2012, 12:37:08 AM10/3/12
to silverst...@googlegroups.com

> Frankly you guys are not helping at all, i'm just trying to help/contribute and get a solution for my problem, we are arguing for nothing.
> How someone would take a company serious if they are not listening to developers and community?

I'm sorry you feel that way, I've done my best to explain the issues that I see with your solution. I've spent a good chunk of the afternoon trying to help you with this, but you consistently deny that there's an issue.

> Yes that are hundreds of cases but the only one to concern about is this two
>
> [shortcode]Content[/shortcode]
>
> that generates
>
> <p>[shortcode]Content/[shortcode]</p>
>
> and
>
> [shortcode]
> Content
> [/shortcode]
>
> that generates
>
> <p>[shortcode]</p>
> <p>Content</p> (this <p> should be here)
> <p>[/shortcode]</p>
>
> as the user would only type in
>
> [shortcode]Content[/shortcode]
> or
> [shortcode]
> Content
> [/shortcode]

WYSIWYG editors end up being much more fiddly than this: you have to deal with a wide variety of browsers, users copy/pasting content, entering content into an existing page and that content. The way to make a robust system is to address the kinds of edge cases that I raised.

akbortoli

unread,
Oct 3, 2012, 2:02:58 AM10/3/12
to silverst...@googlegroups.com
Sorry Sam if i didn't expressed myself quite well.

I'm not denying anything i'm not saying that my solution should be implemented, it was stupid of me to pull a request.

I've just made a solution that kind of worked for me, you actually pointed out some issues which i'm working on and trying to fix.

I'm just pointing out an issue with shortcodes and trying to find a solution for it, specially with you guys who created SS, you guys understand it more than anyone else .

I also spent most of my afternoon trying to fix this issue and explain to you that there is an issue with shortcodes and tinymce.

I'm trying to contribute with SS and get my job done, but you guys are just not getting what i'm saying or have a really close mind about shortcodes.

Thanks anyway.

akbortoli

unread,
Oct 3, 2012, 2:24:39 AM10/3/12
to silverst...@googlegroups.com
Before you say anything else, i know that Tinymce works as it should i know shorcodes works too, what i'm saying is that adding a shortcode to tinymce generates an issue, that's where the problem lives, tinymce adds a <p> around it and it's not good for HTML markup, specially if your shortcode returns a block element.

consider the following situation:

You have a simple page and the content of this page is a WYSIWYG content, how would you implement a nice little and easy to remember code/feature for the final client that doesn't know any HTML to implement which return a quite big chuck of HTML and inserts it wherever the end client added it into the editor. the best solution i know is by using shortcodes, is there anything else you could do?

the content may look like this:

long content

[shortcode]

rest of the content

keep it simple and easy to the client to use.

Martimiz

unread,
Oct 3, 2012, 4:51:27 AM10/3/12
to silverst...@googlegroups.com
Hi guys
Interesting - and very fast paced - discussion :-) I for one would be sorry to see the shortcodes go, as they are the one way for a user to place some dynamic stuff, like a small form, into the content area. And since there is no obligation to use them, why not keep them :-)

My question is, should we cater for every possible way a user could mess up?  Like shortcodes in headers, td, lists? I think not. First of all, most users that use the WYSIWYG, know after a while that it's not always absolutely reliable nor user-friendly.

You may approach any user in two ways: restrict or educate (though most time it'll be some balance of the two). So if you do allow your user to use shortcodes at all, you sort of automatically need to provide some education as to how. Just take it one step further and explain where and in what manner to place them (and to try again if things get weird). That is one thing.

The next is the complexity of the shortcodes used - and here it's really up to the developer. I tend to keep away from shortcodes where the user can add their own content - as it can lead to really unexpected outcomes. There are often other ways (like for instance providing a textfield for that). But anyway, if for example, the user knows how to keep the codes on a single line, then you could (and possibly should) effectively remove every other bit of HTML from the start and the finish of that line. That magic would work for me - and maybe it could even be optional…

Martine

Op woensdag 3 oktober 2012 04:58:40 UTC+2 schreef akbortoli het volgende:

Josua

unread,
Oct 3, 2012, 9:30:10 AM10/3/12
to silverst...@googlegroups.com
Hi all!

On many occasions the TinyMCE editor is not enough to make complex formats, so I'm developing a special system based on shortcodes.
The problem I have had, as akbortoli says, are <p>[shortcode]</ p>.
To remove them simply I have done an intelligent filtering (only surrounding the shortcodes) when save a record to the database using augmentWrite in a Decorator.
For now it works pretty well. :)
But surely, this functionality should be in the SilverStripe core.

Regards,
Jose

Anselm Christophersen

unread,
Oct 3, 2012, 10:27:51 AM10/3/12
to silverst...@googlegroups.com
I very much liked the idea of having a tinymce plugin handle insertion of shortcodes or macros, as I agree that it would be very beneficial to have this additional feature (for many of the reasons that akbortoli mentioned).
That would also be a lot more user friendly, e.g. you could choose "insert macro" in the menu, and could be presented with a list of available macros. Under the hood we could still use shortcodes.
If I'm not remembering wrongly there was an option like this when I used the CMS "Umbraco" quite some years ago - I think it was also using tinymce.

I had a look trying to find that kind of plugins for tinymce, but didn't have much luck. Does anyone know if something like this exist?
I think this would be a good idea for a module.

Thanks

Anselm
> --
> You received this message because you are subscribed to the Google Groups "SilverStripe Core Development" group.
> To view this discussion on the web visit https://groups.google.com/d/msg/silverstripe-dev/-/kQn4TyAZ66EJ.

Nathan Cox

unread,
Oct 3, 2012, 5:27:00 PM10/3/12
to silverst...@googlegroups.com
What I've been meaning to look at is a system similar to Insert Gallery
in WordPress.

They have a button that opens a media popup like SS3 has but you can
insert a gallery which gets put in the content's HTML as the [gallery]
shortcode but gets displayed in TinyMCE as a placeholder image. I think
it uses TinyMCE's onPostProcess() event. Focusing the placeholder gives
you buttons to edit or delete the gallery. Editing it sets properties on
the shortcode (eg [gallery columns="1"]).

It would be great to try distill that kind of functionality down to
something reusable. Like an Insert Google Map button that has an
address field in the popup.

Nathan Cox

unread,
Oct 3, 2012, 5:32:49 PM10/3/12
to silverst...@googlegroups.com
Although I forgot to mention that the problem with just having a
straight up button in TinyMCE is that most of the shortcodes I use are
page-specific (like the form one on a user defined form) and experience
tells me that SilverStripe isn't really made for page-specific TinyMCE
configurations.

Sam Minnée

unread,
Oct 3, 2012, 5:33:30 PM10/3/12
to silverst...@googlegroups.com
Whatever Simon did for oEmbed is probably useful here. Simon, can it be generalised? Although oEmbed insertion would still keep its dedicated UI in the media insertion dialog, there could be an "insert widget" feature that developers can add to from an API.

We would want to consider cross-over with the existing "insert snippet" tool, and with the widgets module. I would expect that users don't give a damn about the difference between a snippet and a widget, even though behind the scenes they're completely different.

Assuming it still used shortcodes to actually power the widget insertion on the front-end, we would have more control over the generated HTML, which would obviate my initial concerns about akbortoli's patch.

Again, Simon's work on the oEmbed inserter should be a useful model to follow.

Sam Minnée

unread,
Oct 3, 2012, 5:34:44 PM10/3/12
to silverst...@googlegroups.com
1) A "widgets" button with a popup containing page-specific widgets might be okay.  It would also give more room to explain what the different widgets are, and minimise toolbar clutter.

2) Is SS3 any better for page-specific TinyMCE configurations?  I know a lot of stuff was chucked out and rebuilt in that arena.

Nathan Cox

unread,
Oct 3, 2012, 5:56:19 PM10/3/12
to silverst...@googlegroups.com

On 4/10/2012 10:34 a.m., Sam Minn�e wrote:
> 1) A "widgets" button with a popup containing page-specific widgets
> might be okay. It would also give more room to explain what the
> different widgets are, and minimise toolbar clutter.
Yeah that's probably the way to go. Adds an extra click and we might
have to be double sure to point out to users that there can be different
things behind that button though.

>
> 2) Is SS3 any better for page-specific TinyMCE configurations? I know
> a lot of stuff was chucked out and rebuilt in that arena.
>
The basic idea is the same, although the magic of Entwine helped a lot
when I tried to do this in a module:
https://github.com/nathancox/silverstripe-customhtmleditorfield the
documentation only talks about editor-specific styles because I didn't
finish the configs but the gist was to store all tinyMCE configs in an
array and apply them to each editor based on a data-config-id attribute.

The issue was that configs don't get loaded via AJAX so it basically
only works if you open the edit page form directly but not if you
navigate to it. Was going to look at a tidy way to set all the configs
in LeftAndMain's init extension hook but that's as far as I got at the time.
Message has been deleted

akbortoli

unread,
Oct 3, 2012, 6:36:35 PM10/3/12
to silverst...@googlegroups.com
My solution didn't work out quite well.

I can see that we are getting some good ideias here, yes, WordPress does a really good job with shortcodes.

Another problem i ran into is that i can't add shortcodes inside another shortcode, it doesn't get parsed:

[shortcode1 attrs]Content [shortcode2 attrs] more content...[/shortcode1]

akbortoli

unread,
Oct 3, 2012, 7:12:35 PM10/3/12
to silverst...@googlegroups.com
Hey Sam, do you remember the job you guys done for http://mmnz.co.nz? Back in Jul 15, 2011... SS2.X

Jean-Fabien Barrois was working on it, they had the same problem with Tinymce and Shortcodes...

Sam Minnée

unread,
Oct 4, 2012, 1:05:55 AM10/4/12
to silverst...@googlegroups.com
> I can see that we are getting some good ideias here, yes, WordPress does a really good job with shortcodes.

I'm trying to find the <p> tag removal in http://core.trac.wordpress.org/browser/tags/3.4.2/wp-includes/shortcodes.php and I can't see it. Am I looking in the wrong place? My suspicion is that Wordpress handles this because it transforms content back into Markdown, rather than storing it in the DB as HTML. This does make the content much more amenable to shortcodes.

In any case, I like the idea of using shortcodes as the enabling technology of a widget UI, rather than expecting users to enter them manually. The discussion is going in an interesting direction on that. Regarding that, Simon says "the oEmbed hooks into onSaveContent/onBeforeSetContent should be fairly reusable" (I asked him on IRC)

> Another problem i ran into is that i can't add shortcodes inside another shortcode, it doesn't get parsed:
> [shortcode1 attrs]Content [shortcode2 attrs] more content...[/shortcode1]

Sounds like a bug to fix in the shortcode core code. I would expect that the way you'd do that is to pass the nested content snippet to the shortcode parser before executing the shortcode callback.
Reply all
Reply to author
Forward
0 new messages