Category item url have "amp" in them on 5.2.0-RC1

15 views
Skip to first unread message

Alexander Obuhovich

unread,
Jun 22, 2012, 3:29:31 PM6/22/12
to In-Portal Bugs
In-Portal apply "htmlspecialchars" function on all input that comes from Front-End user.

This is good, but when submitted link name is "good & bad", then it will become "good & bad" in database and produce "/category/good-amp-bad.html" url. Both "&" and ";" are restricted symbols and are stripped from url, but "amp" stays.

--
Best Regards,

http://www.in-portal.com
http://www.alex-time.com

Dmitry A.

unread,
Jun 22, 2012, 7:12:18 PM6/22/12
to in-port...@googlegroups.com
Interesting observation Alex!

I believe we should think about more unified rules/policies for storing data (escaped/non-escaped) so it can be followed through out the time.

Let's research on this more and see what we can find. Perhaps we can turn to other Frameworks to see how they handling it and then discuss. Can you please check on ZF2, and I'll check on Symfony or others.


DA

Dmitry A.

unread,
Jun 24, 2012, 1:42:58 PM6/24/12
to in-port...@googlegroups.com
So what are the news here?

Here is what I have found out:

1. No one is sanitizing ALL data by default. This should be done based on Application needs.

2. It's always a good practice to keep data in the same tables in single format. All escaped or unescaped.

3. We'll run into issues with searching in DB if we store escaped data.


My idea is that we should

1. Store data unescaped, but ALWAYS (on Front and Admin) escape when displaying it. I know you might say that additional load on PHP/CPU, but we can't just escape all the data in the DB or do mixture - doesn't make sense. We can escape with MySQL or something on SELET if possible.

2. We should give an option to Strip HTML tags on Input from Front-end for specific fields when needed. By default it should be Striped, but developer can overwrite this based on needed logic.

Example with Category name: we should strip HTML before saving so we don't end up with something like this (my test) in DB: <b>Sanitize & Clean<b/>


Please share what you have found and your thoughts of course.


DA

Phil

unread,
Jun 24, 2012, 4:45:39 PM6/24/12
to in-port...@googlegroups.com
I agree to store data unescaped, seems the more easy. No risk for super technical SQL injection?


Envoyé avec Sparrow

--
You received this message because you are subscribed to the Google Groups "In-Portal Bugs Team" group.
To view this discussion on the web visit https://groups.google.com/d/msg/in-portal-bugs/-/aDpCR5IzoQkJ.
To post to this group, send email to in-port...@googlegroups.com.
To unsubscribe from this group, send email to in-portal-bug...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/in-portal-bugs?hl=en.

Alexander Obuhovich

unread,
Jun 24, 2012, 4:59:32 PM6/24/12
to in-port...@googlegroups.com
But if you forgot to escape before displaying then reviewing front-end data from admin could result in session_id sent to other server via ajax by bad script injected in user data.

Dmitry A.

unread,
Jun 24, 2012, 5:27:53 PM6/24/12
to in-port...@googlegroups.com
I think we should Escape all this when displaying!

I believe I have explained that. Did I miss anything?

DA

Alexander Obuhovich

unread,
Jun 25, 2012, 2:58:10 AM6/25/12
to in-port...@googlegroups.com
I see, then use <inp2:prefix_Field name="FieldName" html_escape="1"/> instead of <inp2:prefix_Field name="FieldName"/> at all places.

Interesting thing about Field tag. I've just noticed, that it calls "htmlspecialchars" function internally (equivalent of "html_escape" parameter) for all calls made from Admin Console.

This was working like this all the time. This proves an idea, that we must do output escaping and not input escaping.



DA

--
You received this message because you are subscribed to the Google Groups "In-Portal Bugs Team" group.
To view this discussion on the web visit https://groups.google.com/d/msg/in-portal-bugs/-/1W7snutt1IAJ.

To post to this group, send email to in-port...@googlegroups.com.
To unsubscribe from this group, send email to in-portal-bug...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/in-portal-bugs?hl=en.

Dmitry A.

unread,
Jun 26, 2012, 4:41:35 PM6/26/12
to in-port...@googlegroups.com
Actually, I was thinking about auto-escaping in <inp2:prefix_Field name="FieldName"/> field so there is no need to add escape_html at all.

I know this might some as backwards compatibility issue, but my question is where specifically Escaped HTML can become an issue?

DA

Alexander Obuhovich

unread,
Jun 27, 2012, 5:16:33 AM6/27/12
to in-port...@googlegroups.com
Can't remember any one right now, but anyway we need to find each Field tag usage everywhere and decide if automatic html escaping can do any harm there.



DA

--
You received this message because you are subscribed to the Google Groups "In-Portal Bugs Team" group.
To view this discussion on the web visit https://groups.google.com/d/msg/in-portal-bugs/-/wMggJgKF0XkJ.

To post to this group, send email to in-port...@googlegroups.com.
To unsubscribe from this group, send email to in-portal-bug...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/in-portal-bugs?hl=en.

Dmitry A.

unread,
Jul 6, 2012, 1:35:10 AM7/6/12
to in-port...@googlegroups.com
Alex, would you please create a task here?

DA

Alexander Obuhovich

unread,
Jul 6, 2012, 4:56:29 AM7/6/12
to in-port...@googlegroups.com
Are you sure, that we have 100% solid plan place in the task?

From discussion I've understood, that we want:
  1. keep front-end user input as-is in database
  2. always do htmlspecialchars on output of Field tag result

If we try to do this we'll get double-escaping of data, that comes from Front-End (e.g. links, articles) that is escaped in database already.


On Fri, Jul 6, 2012 at 8:35 AM, Dmitry A. <dand...@gmail.com> wrote:
Alex, would you please create a task here?

DA

--
You received this message because you are subscribed to the Google Groups "In-Portal Bugs Team" group.
To view this discussion on the web visit https://groups.google.com/d/msg/in-portal-bugs/-/C5K8m6Xsjp0J.

To post to this group, send email to in-port...@googlegroups.com.
To unsubscribe from this group, send email to in-portal-bug...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/in-portal-bugs?hl=en.

Dmitry A.

unread,
Jul 9, 2012, 2:56:18 PM7/9/12
to in-port...@googlegroups.com
Double-escaping will happen ONLY on websites where it was used so we might want to have some sort of upgrade for this?

DA

Alexander Obuhovich

unread,
Jul 9, 2012, 3:37:26 PM7/9/12
to in-port...@googlegroups.com
We can't do any automatic upgrade system for database, because each text field in database doesn't have any sort of indicator from where it's data comes from (Front-End or Admin).

We can theoretically try to do a mass "htmlspecialchars_decode" on all text fields, which might have some unexpected effects. With such mass replace we can get mass data corruption too.


On Mon, Jul 9, 2012 at 9:56 PM, Dmitry A. <dand...@gmail.com> wrote:
Double-escaping will happen ONLY on websites where it was used so we might want to have some sort of upgrade for this?

DA

--
You received this message because you are subscribed to the Google Groups "In-Portal Bugs Team" group.
To view this discussion on the web visit https://groups.google.com/d/msg/in-portal-bugs/-/BcVB_stmoacJ.

To post to this group, send email to in-port...@googlegroups.com.
To unsubscribe from this group, send email to in-portal-bug...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/in-portal-bugs?hl=en.

Dmitry A.

unread,
Jul 9, 2012, 4:24:04 PM7/9/12
to in-port...@googlegroups.com
I think we can create a script or some sort of Tool similar to what we had for UTF-8 conversion which can be ran if needed separately from the installation/update at any point.

What do you think?

DA

Alexander Obuhovich

unread,
Sep 26, 2012, 7:32:18 AM9/26/12
to in-port...@googlegroups.com
Another interesting case, with this problem.

We use "next_template" url parameter to specify where users will be redirected after login/logout. This works in most cases, except ones, when "next_template" has some extra variables in it, e.g:
"external:/path/to/new-page.html?var1=val1&var2=val2".

In this case "&" first will be encoded into "&amp;" thanks to hHTTPQuery class (on Front-End only) and then into "&amp;amp;" if this code is used on the form:
<input type="hidden" name="next_template" value="<inp2:m_Get name="next_template" html_escape="1"/>"/>


With new version versions of In-Portal and it's new features need to use as much of them as possible grows and this htmlspecialchars problem grows even bigger. I propose that we need to address it as soon as we could.

To finalize problem has several places that require fixing:
  1. kHTTPQuery class - don't automatically encode all input that comes from Front-End
  2. Front-End template - find every use of inp2: tag, that is glued into HTML markup (e.g. <input value="_INP_TAG_HERE"/>) and add html_escape="1" to prevent " or ' from breaking down HTML tag
  3. Scan all text fields in database and do htmlspecialchars_decode function on them no matter what's inside. I suppose that function is clever enough to keep existing field value if nothing needs to be replaced.

DA

--
You received this message because you are subscribed to the Google Groups "In-Portal Bugs Team" group.
To view this discussion on the web visit https://groups.google.com/d/msg/in-portal-bugs/-/EsJ7m7T7dWEJ.

To post to this group, send email to in-port...@googlegroups.com.
To unsubscribe from this group, send email to in-portal-bug...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/in-portal-bugs?hl=en.

Alexander Obuhovich

unread,
Nov 5, 2012, 12:32:04 PM11/5/12
to Bugs In-Portal
Reply all
Reply to author
Forward
0 new messages