Process and technology questionnaire
4 views
Skip to first unread message
Florian Thiel
Mar 31, 2009, 8:40:16 AM3/31/09
to habari-dev
Hello habari developers,
I'm in the process of writing my diploma thesis on the prevention of
web application security vulnerabilities and I'd like to know a bit
about your fine project from the developer's point of view.
It would be great if you could take a couple
of minutes and think about the questions below. The questions are
mostly open-ended. Elaborate and skip questions at will. I'm sure
these questions
are not relevant for everybody on this list but I would like to be
rather inclusive and not only
address the project lead. I also know that part of these questions can
be answered by looking at your web page but I'd like to see the
developer's view and also uncover discrepancies which might be
present.
Thank you very much in advance. I will provide you with a link to the
results of my thesis when it's done.
Florian
The questions:
About technical aspects:
- Are you using a web application framework? Which one?
- Do you use explicit data modeling for all business objects in the
application?
- Do you have a specific layers for input/output validation/filtering?
(If applicable) What does the input/output layer do (respectively)?
How? Are you using external libraries? Why? Why not? (for HTML
sanitation. object-relational mappers, database abstractions with
prepared statements)?
- (If applicable) What responsibilities do the input/output layers
have, respectively?
- How do you ensure that all input passed through validation/
filtering? Do you have an API that must be used?
- Do you provide services to independently developed modules/
components? Is there a defined API?
- Which other external libraries do you use?
About the development process:
- Is there public documentation about the responsibilities of the
input/output layers?
- Is there public documentation about *when* input/output validation/
filtering should happen? (Like: "output filtering must always happen
in the method that renders the data")
- Do you have automatic tests for the whole system?
Bonus question:
- Do you do manual code review?
I'm in the process of writing my diploma thesis on the prevention of
web application security vulnerabilities and I'd like to know a bit
about your fine project from the developer's point of view.
It would be great if you could take a couple
of minutes and think about the questions below. The questions are
mostly open-ended. Elaborate and skip questions at will. I'm sure
these questions
are not relevant for everybody on this list but I would like to be
rather inclusive and not only
address the project lead. I also know that part of these questions can
be answered by looking at your web page but I'd like to see the
developer's view and also uncover discrepancies which might be
present.
Thank you very much in advance. I will provide you with a link to the
results of my thesis when it's done.
Florian
The questions:
About technical aspects:
- Are you using a web application framework? Which one?
- Do you use explicit data modeling for all business objects in the
application?
- Do you have a specific layers for input/output validation/filtering?
(If applicable) What does the input/output layer do (respectively)?
How? Are you using external libraries? Why? Why not? (for HTML
sanitation. object-relational mappers, database abstractions with
prepared statements)?
- (If applicable) What responsibilities do the input/output layers
have, respectively?
- How do you ensure that all input passed through validation/
filtering? Do you have an API that must be used?
- Do you provide services to independently developed modules/
components? Is there a defined API?
- Which other external libraries do you use?
About the development process:
- Is there public documentation about the responsibilities of the
input/output layers?
- Is there public documentation about *when* input/output validation/
filtering should happen? (Like: "output filtering must always happen
in the method that renders the data")
- Do you have automatic tests for the whole system?
Bonus question:
- Do you do manual code review?
Chris Meller
Apr 1, 2009, 11:08:56 PM4/1/09
to habar...@googlegroups.com
Since no one else has responded, I'll give a quick 5-minute response that'll hopefully gloss over enough to give you a start.
This kinda relates to the next question's ORM section.
The input filters are designed to make sure data is (at least 99%) safe for any use. Since we used prepared statements and bound parameters through PDO this is less of a SQL injection protection and more of an HTML / XSS type issue. The SuperGlobal class should ensure that you don't end up with unintentional JavaScript events in things like comments that could cause problems when re-displayed.
The SuperGlobal class automatically filters the PHP superglobal arrays and replaces them with instances of itself on each request. By default it should behave much like the originals, only providing pre-filtered (and supposedly safe) input. That makes it very easy to use external libraries or to write code not specifically using the Habari API, since it should behave just like any other PHP script.
Developers have to manually request the raw input (via something like $_POST['foo']->raw(), IIRC) to get anything that may contain content that's unsafe. That should be the only instance in which input has to be accessed through an API that's Habari-specific.
Other libraries are limited to PHP extensions that are deemed broadly available. I don't believe there are any other libraries utilized, save jQuery on the client-side.
This of course doesn't necessarily apply to plugins that aren't shipped with core. There are several in -extras that rely on external libraries.
I don't believe so, no. Our documentation needs serious work.
Nope. I think random beatings are our sole teaching technique right now.
There have been a handful of unit tests using a UnitTest class created by Owen quite a while ago. Recently we've started (multiple times?) to move to PHPUnit and encourage the increased use of unit tests in development and pre-release.
Only in that hopefully the person committing a patch is knowledgeable enough to adequately review the code being submitted. There's no formal code review, it's more of a "community policing" policy at present. As we (hopefully) grow and acquire more developers this policy may change to require a more formal review of all changes, but at present the project isn't really large enough to dedicate the kinds of resources required for formal review.
Hopefully my quick pass at answers will help, I'm sure others will have more detail and maybe clarifications or corrections to add. Thanks for asking and good luck with the thesis, please let us know if we can help further!
On Tue, Mar 31, 2009 at 8:40 AM, Florian Thiel <flo....@googlemail.com> wrote:
Hello habari developers,
I'm in the process of writing my diploma thesis on the prevention of
web application security vulnerabilities and I'd like to know a bit
about your fine project from the developer's point of view.
It would be great if you could take a couple
of minutes and think about the questions below. The questions are
mostly open-ended. Elaborate and skip questions at will. I'm sure
these questions
are not relevant for everybody on this list but I would like to be
rather inclusive and not only
address the project lead. I also know that part of these questions can
be answered by looking at your web page but I'd like to see the
developer's view and also uncover discrepancies which might be
present.
Thank you very much in advance. I will provide you with a link to the
results of my thesis when it's done.
Florian
The questions:
About technical aspects:
- Are you using a web application framework? Which one?
No, all code is original and written by Habari developers. It roughly follows an MVC pattern, but doesn't use an existing framework.
- Do you use explicit data modeling for all business objects in the
application?
This kinda relates to the next question's ORM section.
- Do you have a specific layers for input/output validation/filtering?
(If applicable) What does the input/output layer do (respectively)?
How? Are you using external libraries? Why? Why not? (for HTML
sanitation. object-relational mappers, database abstractions with
prepared statements)?
All input (through the PHP superglobals) is filtered by a custom PHP class. I believe it uses a good bit of regex and a whitelist of elements and attributes, but you should check the InputFilter and SuperGlobal classes to verify that.
No external PHP classes or libraries are used, mainly because we haven't found any that are suitably licensed. Personally I'd love to use something like HTMLPurifier, but it's GPL and so we can't use it in our ASL project.
Several of the data classes (like Post) use a rough ORM approach, implementing our QueryRecord class to provide simple ways to insert / update / delete records. I would say this is more of a laziness / convenience thing than a security thing, although it may have that side effect. In general I'd say we're all fairly wary of ORM's potential performance impacts, so there's a lot of manual SQL to be found.
We do use the PHP PDO database abstraction layer, mainly for prepared statements and bound parameters. Not only does it prevent a lot of security problems with SQL injection, but it can provide a performance boost in certain situations.
No external PHP classes or libraries are used, mainly because we haven't found any that are suitably licensed. Personally I'd love to use something like HTMLPurifier, but it's GPL and so we can't use it in our ASL project.
Several of the data classes (like Post) use a rough ORM approach, implementing our QueryRecord class to provide simple ways to insert / update / delete records. I would say this is more of a laziness / convenience thing than a security thing, although it may have that side effect. In general I'd say we're all fairly wary of ORM's potential performance impacts, so there's a lot of manual SQL to be found.
We do use the PHP PDO database abstraction layer, mainly for prepared statements and bound parameters. Not only does it prevent a lot of security problems with SQL injection, but it can provide a performance boost in certain situations.
- (If applicable) What responsibilities do the input/output layers
have, respectively?
The input filters are designed to make sure data is (at least 99%) safe for any use. Since we used prepared statements and bound parameters through PDO this is less of a SQL injection protection and more of an HTML / XSS type issue. The SuperGlobal class should ensure that you don't end up with unintentional JavaScript events in things like comments that could cause problems when re-displayed.
- How do you ensure that all input passed through validation/
filtering? Do you have an API that must be used?
The SuperGlobal class automatically filters the PHP superglobal arrays and replaces them with instances of itself on each request. By default it should behave much like the originals, only providing pre-filtered (and supposedly safe) input. That makes it very easy to use external libraries or to write code not specifically using the Habari API, since it should behave just like any other PHP script.
Developers have to manually request the raw input (via something like $_POST['foo']->raw(), IIRC) to get anything that may contain content that's unsafe. That should be the only instance in which input has to be accessed through an API that's Habari-specific.
- Do you provide services to independently developed modules/
components? Is there a defined API?
- Which other external libraries do you use?
Other libraries are limited to PHP extensions that are deemed broadly available. I don't believe there are any other libraries utilized, save jQuery on the client-side.
This of course doesn't necessarily apply to plugins that aren't shipped with core. There are several in -extras that rely on external libraries.
About the development process:
- Is there public documentation about the responsibilities of the
input/output layers?
I don't believe so, no. Our documentation needs serious work.
- Is there public documentation about *when* input/output validation/
filtering should happen? (Like: "output filtering must always happen
in the method that renders the data")
Nope. I think random beatings are our sole teaching technique right now.
- Do you have automatic tests for the whole system?
There have been a handful of unit tests using a UnitTest class created by Owen quite a while ago. Recently we've started (multiple times?) to move to PHPUnit and encourage the increased use of unit tests in development and pre-release.
Bonus question:
- Do you do manual code review?
Only in that hopefully the person committing a patch is knowledgeable enough to adequately review the code being submitted. There's no formal code review, it's more of a "community policing" policy at present. As we (hopefully) grow and acquire more developers this policy may change to require a more formal review of all changes, but at present the project isn't really large enough to dedicate the kinds of resources required for formal review.
Hopefully my quick pass at answers will help, I'm sure others will have more detail and maybe clarifications or corrections to add. Thanks for asking and good luck with the thesis, please let us know if we can help further!
Rich Bowen
Apr 16, 2009, 3:48:50 PM4/16/09
to habar...@googlegroups.com
The questions:
About technical aspects:
- Are you using a web application framework? Which one?
Cake
- Do you use explicit data modeling for all business objects in the
application?
Yes. But Cake pretty much handles that for you.
- Do you have a specific layers for input/output validation/filtering?
Yes. Again, Cake does a great job of this, but you have to define validation rules per field.
(If applicable) What does the input/output layer do (respectively)?
How? Are you using external libraries? Why? Why not? (for HTML
sanitation. object-relational mappers, database abstractions with
prepared statements)?
Cake gives you a way to define what constitutes valid data, and has a jillion built-in defined data types (date, email, etc)
- (If applicable) What responsibilities do the input/output layers
have, respectively?
- How do you ensure that all input passed through validation/
filtering? Do you have an API that must be used?
Cake handles this. Data must be validated before an object can be saved.
- Do you provide services to independently developed modules/
components? Is there a defined API?
No.
- Which other external libraries do you use?
JQuery. ImageMagick. FCKEditor.
About the development process:
- Is there public documentation about the responsibilities of the
input/output layers?
No.
- Is there public documentation about *when* input/output validation/
filtering should happen? (Like: "output filtering must always happen
in the method that renders the data")
Filtering happens when an object is saved.
- Do you have automatic tests for the whole system?
Um. Sort of. We're not real good about this.
Bonus question:
- Do you do manual code review?
Absolutely. Every code commit goes to a mailing list, and most of us read every diff. Stupid changes generate angry comments pretty quickly. It's a good system.
--
"That's what being alive IS ... It's being badly prepared for everything!
Because you only get one chance ... You only get one chance and then you die
and they don't let you go round again after you've got the hang of it!"
_The Bromeliad Trilogy_, Terry Pratchett
Owen Winkler
Apr 16, 2009, 4:19:52 PM4/16/09
to habar...@googlegroups.com
Rich Bowen wrote:
>>
>> The questions:
>>
>> About technical aspects:
>> - Are you using a web application framework? Which one?
>
> Cake
>>
>> The questions:
>>
>> About technical aspects:
>> - Are you using a web application framework? Which one?
>
> Cake
Was this supposed to be about our own projects, or just Habari?
In case it was unclear to others besides me, Habari doesn't use Cake.
Owen
Chris Meller
Apr 16, 2009, 5:43:46 PM4/16/09
to habar...@googlegroups.com
We should switch to Cake!
Sean Coates
Apr 16, 2009, 5:54:26 PM4/16/09
to habar...@googlegroups.com
> We should switch to Cake!
You're 15 days late.
S
Caius Durling
Apr 16, 2009, 5:57:11 PM4/16/09
to habar...@googlegroups.com
On 16 Apr 2009, at 22:43, Chris Meller wrote:
We should switch to Cake!
Sean Coates
Apr 16, 2009, 6:05:46 PM4/16/09
to habar...@googlegroups.com
> You're 15 days late.
America/Montreal
for you purists.
S
Joey Brooks
Apr 16, 2009, 6:04:49 PM4/16/09
to habar...@googlegroups.com
On Thu, Apr 16, 2009 at 5:57 PM, Caius Durling <ca...@caius.name> wrote:
> On 16 Apr 2009, at 22:43, Chris Meller wrote:
>
> We should switch to Cake!
> On 16 Apr 2009, at 22:43, Chris Meller wrote:
>
> We should switch to Cake!
+1. With a fork.
Thanks,
Joey Brooks
Milk Carton Designs || milkcartondesigns.com
Reply all
Reply to author
Forward
0 new messages